12 Mar

Why You Want to Develop in Production-Like Conditions, and How to Do it with Vagrant

Like many people developing with WordPress, I used to use XAMPP to run a Linux/Apache/MySQL stack under Windows. This beats developing directly off a remote server by a long stretch (in case anybody still does that), but it turns out to be quite a hassle as you wrestle with / vs \ in paths and other discrepancies between the Unix and Windows worlds. Developing in an environment as close to production as possible ends up being a much better choice. Popular free tools to do so are:

  • VirtualBox to run other operating systems within your own
  • Vagrant to manage development environments within VirtualBox
  • Puphpet, a wizard-like interface to help set up Vagrant through Puppet automation scripts.

Yes, these work somewhat like Russian dolls, as modern web development has a severe case of tools to set up tools to set up tools. It can get a little crazy at times, so you want to find the sweet spot where you gain efficiency without becoming a slave to your toolset (which is supposed to save you time in the first place). Getting this up and running is a big enabler for continuous integration, and why you would want to think and develop that way is the topic of a separate entry.

In this post I’ll share some practical details to contribute a little back to the open source community and not just consume its benefits passively.  The end result is a fully functional Unix environment serving a WordPress website locally on a Windows PC, across a whole local network.

This entry is written from the perspective of “Unix as a foreign language” from a Windows native. It does assume that you have read the basics about VirtualBox and Vagrant, and focuses on common roadblocks that I’ve encountered along the way. As always, the key to learning a whole new way of doing things is to break it down in small digestible chunks.

Installing VirtualBox + Vagrant + ConEmu

It took me hours to get the VirtualBox /Vagrant combo to fully run like I wanted, so for the benefit of others trying to get it up and running, here are things to watch for in order to run a Linux guest from a Windows host, though some of these things also apply if you use a Mac. I’m running CentOS as a guest OS, you may need to adjust your syntax slightly with other Linux builds but the below should mostly apply with Ubuntu et. al.

First of all, check that your BIOS is set up to with virtualization enabled, which is often not the case by default.

Second, VirtualBox can be a pain to install/update in Windows. One thing that can help is to install it after disabling network devices or unplugging your Ethernet cable (source). You may want to install it as an administrator and use bridged networking, for reasons explained further below.

Third, on Windows it is safer to install Vagrant off the root of one of your hard drives, in a directory without spaces (say D:\Vagrant). A few Vagrant settings help with troubleshooting and overall quality of life, such as picking a short friendly name for your VM (defaults tend to be very long and unwieldy), or triggering the VirtualBox UI for the guest OS to let you see what’s going on during its boot sequence. You’ll want to edit your parent Vagrantfile with these default customizations, and add a system environment variable pointing to your Vagrant directory (batch file). That way, these settings will apply across all boxes you may create manually or with a tool such as Puphpet.

Fourth, you’ll find you need to run a bunch of command line instructions to manage Vagrant then dive into your Linux host. Out of the box Windows is poorly equipped for that. After trying Console2, I settled on ConEmu which lets me run PowerShell in one tab (Windows host), vagrant ssh in a 2nd tab, the mysql command line in the third… (Understand though that it’s a console, not a shell.) I recommend using different color schemes in ConEmu for different tabs/environments, that way you’ll instantly see whether you’re in your Windows PowerShell vs. your Linux command line vs. MySQL. Fear not the command line, you’re gonna need it! Here are some shortcuts to make your life easier. It looks more complicated than it really is, just take it one step at a time as you bump into the need for new instructions.

Common Solutions to Post-Install Blues

At this point I’ll skip a bunch of install/setup steps already well documented in the links at the bottom of this entry. You should find the basics rather easy, but once you’re to the point where Vagrant is serving a website that you’re able to browse from Windows, several problems could still be in your way that I haven’t seen documented in one place, so here we go:

a) If PHP sessions can’t be written to disk, there are a couple different fixes so that Apache cooperates. That issue is easy to spot as PHP error messages will probably pollute your pages when you browse your brand new virtual local web server. Typically running this from your guest command line should do it:

sudo chown www-data:www-data /var/lib/php/session

b) If small file changes are not reflected by Apache, and/or some Javascript files served by your guest appear cut before their end, leading to a serious case of WTF (why is my local website not catching edits even though I’ve sync’ed a folder between host and guest? Is some sort of hidden caching or file corruption going on??): fear not, it’s not just you going crazy. Simply turn off EnableSendFile and restart Apache:

sudo nano /etc/httpd/conf/httpd.conf

sudo service httpd restart

I like to switch KeepAlive to On while I’m at it. This problem can also occur with Nginx, just google for the slightly different syntax. Update: newer versions of Vagrant apparently took care of the EnableSendFile default.

Note: Sudo lets you run commands as root (i.e. admin), while nano is a text editor that’s easier to use by noobs like us than vi (oh god).

c) If some npm packages can’t be fully installed (say some Grunt modules, a topic I’ll cover in a future entry), that happens for two reasons: 1) problems with symlinks 2) because Windows paths are limited to 260 characters (yes this is crazy), while Node dependencies are sometimes installed very deep into subdirectories. To solve these two issues in one swoop, in your guest OS move the node_modules directory off your shared folder (i.e. your project folder shared from the Windows file system) and create a symlink to it (source) before executing npm install. To do so from the guest command line (i.e. after vagrant ssh):

cd /var/www/yourproject (i.e. whatever’s your Vagrant shared folder)

mv node_modules /tmp/node_modules

ln -s /tmp/node_modules ./node_modules

If you run into a read-only or protocol error while creating the symlink, you need to enable symlinks in your VM and run VirtualBox as an administrator (source). To check that VirtualBox has symlinks enabled, open a Windows command line, move to the directory where VirtualBox is installed, and type (source):

VBoxManage getextradata YOURVMNAME enumerate

Because these issues have a way to cascade into a series of roadblocks, if Vagrant returns an error message when trying to relaunch it while VirtualBox is running as admin, it turns out you need to edit DCOM permissions  in component services for things to work. For me this still wouldn’t work… and I finally resolved the issue by reinstalling Virtualbox as an admin. Note that when you run VirtualBox as an admin, you also need to run Vagrant as an admin (in practice in my case: launch ConEmu as an admin). In other words: ugh!

d) If you can’t access MySQL directly from the host, say using a GUI like SQLYog:

  • Enable port forwarding in NAT mode, or switch VirtualBox to bridged mode from your Vagrantfile, then vagrant reload. Bridged networking is what the cool kids use because it makes your VM accessible to the outside world, and the first time you’ll get it working you’ll feel like you’re in Inception. (See setting up a Mac VM and access from tablets further below.)

sudo nano /etc/my.cnf

  • In that file, change binding to 0.0.0.0 and comment out skip-external-locking

sudo mysql

mysql>GRANT ALL PRIVILEGES ON *.* TO ‘root’@’%’ IDENTIFIED BY ‘pwdhere’ WITH GRANT OPTION;

mysql>FLUSH PRIVILEGES;

exit mysql then:

sudo service mysqld restart

sudo /etc/init.d/iptables stop

This last instruction turns off the guest firewall entirely. If that strikes you as a sledgehammer approach it is, feel free to find out how to open just port 3306 (the default for mysql) if you still want to run the guest OS firewall (of course you still have the Windows firewall standing between the internet and you). But the point is that your VM has likely the firewall blocking external access to mysql by default.

e) If you want to test your websites in Safari / MacOS, install OS X Mountain Lion with iAtkos ML2 and make sure to use bridged mode in your two virtual machines. Edit the hosts file in your brand new virtual Mac, and in my case I gained both extra testing capability and lots of cool points with my geek son who found this extremely cool! Screenshot to prove it did happen.

f) Better than editing hosts files on clients on your LAN, set up a local DNS server to point to your VM’s IP address (again, bridged mode FTW). I have a Synology NAS that does this very well. On most smartphones and tablets you can’t even edit the hosts file unless you root your device, so this trick comes very handy for easy mobile testing.

Final Thoughts

In my experience, you might end up having to destroy the VM (vagrant destroy) and start from a full fresh vagrant up. Breathe, this can happen, you may have to go through a couple of these slow (10+ minutes) full VM rebuilds, but it seems sometimes a fresh start is really the best way to get Vagrant working. Once you’re in a stable place, you’ll use vagrant suspend and vagrant resume whenever you need to turn the VM off and back on.

Yes, it can be a significant learning curve, but if you do any serious web development at all basic UNIX skills are a necessity, and once you get it all working, you’ll wonder how you survived without such a setup.

Useful Resources

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>