Faster Rails Development with Vagrant

Vagrant is an excellent tool for Rails development - you do not need to worry about installing dependencies, damaging your local workstation, or disparity between different machines and operating systems. You can quickly scale the size and shape of the virtual machine on and as-needed basis, and in a worst-case scenario you can destroy everything and start from a blank slate. However, virtual machines often come with a bit of a performance cost. In this post, I will detail some of the tips and tricks for improving the performance of your Rails application during development with Vagrant.

Disable Log Files

Let's be honest - when is the last time you actually opened up the Rails development or test log files on disk? They usually just cloud grep results, but what you may not know is that log files are actually a huge performance bottleneck in virtualized environments! But why?

If you have not read Mitchell's blog post on Comparing Filesystem Performance in Virtual Machines, go read that now. Transmitting information to and from a virtual machine by means of a shared folder is by far one of the largest bottlenecks in working with Vagrant. But what does that have to do with Rails?

By default, Rails is incredibly chatty. Every request for every resource is logged. Even with great gems like quiet_assets and lograge, Rails logs can quickly reach gigabytes of text through regular use. Each time you load a page in the browser or run rake test, you are potentially generating hundreds of lines of log data. Since the logs are stored at the root of your repository, they are shared with the virtual machine by default!

Assuming you have a Rails application running in a Vagrant virtual machine with a standard setup, you are potentially wasting hundreds of cycles waiting for log files to be generated on the guest, transferred to the host, and written to disk for each request. Furthermore, your test suite could be suffering from horrible performance because of the poor I/O in writing the test log!

A standard "hello world" Rails application with the default Rails setup, has an average request time of ~3.83s from my host.

Each time a request is made to the Rails application, a log entry of about 30 lines is added to log/development.log. This file must be synchronized over the shared folder, onto the host machine, all delaying the response from the Rails server.

Disabling Rails logging is actually a bit trickier than one might think. In the Rails API documentation, there exists is a configuration option in the Application namespace for setting and configuring the logger:

# config/development.rb
MyApp::Application.configure do
  config.logger = # ...?
end

Your first instinct might be to set this value to nil or false. For some reason, that does not work. Instead of not logging, Rails still creates the log file but uses timestamped Ruby logs instead of the formatted Rails logs. Setting the value to a path (like /dev/null) will cause no logging to happen (even in the local terminal), which is not an ideal experience. In order to keep logging in the terminal but disable it on disk, you must use the following option:

MyApp::Application.configure do
  config.logger = ActiveSupport::Logger.new(nil)
end

The advantage to this approach is that it works well with other logging gems and still provides output in the terminal. This allows you to keep a terminal tab open with your server to see logs, but eliminates the overhead of writing logs to disk and sharing them with the host.

With the Rails logging disabled, the new response time is ~0.41s.

As an alternative approach, you can also set the value to a path on disk that is outside the shared folder path on the guest machine. In this way, you can still have Rails write logs to disk, but those logs will not be shared with the host, drastically increasing performance:

MyApp::Application.configure do
  config.logger = ActiveSupport::Logger.new("/path/outside/of/shared/folders")
end

For incredibly faster tests, this technique can also be applied to config/test.rb to prevent writing logs during test runs. As a general rule, I always disable logging during the test suite. Even without the overhead of a virtual machine, I can usually shave 10-15 seconds off a test run by reducing the disk I/O.

Use RSync, NFS, and SMB

The default mechanism for shared folders with Vagrant is convenient and is the "least common denominator" among all systems. If your system supports newer technologies, you can leverage them for an added speed boost. The downside is that some of these methods require specific software is installed on the guest and host in order to function properly.

RSync

As of Vagrant 1.5, you can use RSync shared folders which will provide a dramatic increase in disk I/O between the host and guest. To use RSync for shared folders, simply add the following to your Vagrantfile:

config.vm.synced_folder ".", "/vagrant", type: "rsync"

Then run:

$ vagrant rsync-auto

and Vagrant will automatically watch and sync files to and from the virtual machine. For a full list of the configuration options, please see the Vagrant RSync documentation.

NFS

If you are running on OS X or a Linux that supports NFS folders, you can also use NFS shared folders. I only recommend using this option if you are familiar with NFS. Given you have an NFS server installed, add the following to your Vagrantfile:

config.vm.synced_folder ".", "/vagrant", type: "nfs"
config.vm.network "private_network", ip: "192.168.50.1"

In order for NFS to work, the machine must have a statically assigned IP address as you can see above. The given address must be available on the network, so please adjust accordingly. For a full list of the configuration options, please see the Vagrant NFS documentation.

SMB

If you are running on Windows that supports SMB shares, you can use SMB. I only recommend using this option if you are familiar with SMB. Given you have an NFS server installed, add the following to your Vagrantfile:

config.vm.synced_folder ".", "/vagrant", type: "smb"

For a full list of the configuration options, please see the Vagrant SMB documentation.

Combined with other tweaks, using RSync/NFS/SMB can drastically increase the speed of Rails development with Vagrant.

Use all CPUs and half your RAM

Most people are turned off by the idea of allocating all the CPUs to a virtual machine, but here is the thing about CPUs - they are schedulers, not resource constraints. By restricting your virtual machine to less than the full capacity of your CPU(s), you are severely restricting the performance, especially if you are performing complex operations or a large number of database queries.

Memory, on the other hand, is a resource constraint. Once you have allocated memory, it cannot be used by other processes until it is unallocated. As such, we need to more tightly restrict the amount of memory we give to our virtual machines.

If you know how many CPU(s) and how much memory your machine has, great! Just hardcode that in your Vagrantfile:

config.vm.provider "virtualbox" do |v|
  v.memory = 4096
  v.cpus = 2
end

If you do not know how many CPU(s) and RAM you have, or if you would like Vagrant to automatically calculate that for you, you will need to do a bit of trickery. Here is a modified snippet from Stefan Wrobel's How to make Vagrant performance not suck that will set your memory to half of the system memory and use all the CPUs:

config.vm.provider "virtualbox" do |v|
  host = RbConfig::CONFIG["host_os"]

  if host =~ /darwin/ # OS X
    # sysctl returns bytes, convert to MB
    v.memory = `sysctl -n hw.memsize`.to_i / 1024 / 1024 / 2
    v.cpus = `sysctl -n hw.ncpu`.to_i
  elsif host =~ /linux/ # Linux
    # meminfo returns kilobytes, convert to MB
    v.memory = `grep 'MemTotal' /proc/meminfo | sed -e 's/MemTotal://' -e 's/ kB//'`.to_i / 1024 / 2
    v.cpus = `nproc`.to_i
  end
end

While this solution is a bit ugly, it is functional across all non-Windows systems (sorry, but I do not know enough about Windows to know how to obtain those values). If you want a quick-and-easy solution, you can also install the vagrant-faster gem which handles a lot of this configuration automatically.

Disclaimer: This technique assumes you are only running one virtual machine simultaneously. If you are running multiple virtual machines, these values will need to be adjusted accordingly.

Use anything but Webrick

Rail's default web server, Webrick, has a number of problems which make it perform very poorly in virtualized environments. One of the biggest reasons for the poor performance is the native settings which performs reverse DNS lookups by default. For more information on the problem, please read the bug.

In general, do not use Webrick for your development web server. It performs horribly, and it is very unlikely a parody of your production environment (recall that one of the founding principles of Vagrant is to match your production environment as closely as possible). While it is possible to disable reverse DNS lookups in Webrick, I would highly recommend switching to a more performant web server like thin or puma. These servers are easily installed and have proven to be more stable and performant than the native Webrick implementation:

(Credit: http://www.madebymarket.com/blog/dev/ruby-web-benchmark-report.html)

Installing one of these alternative servers is as easy as adding a line in your Gemfile:

gem "puma"

And run the bundle command to install:

$ bundle install

Now when you run bundle exec rails server (or bin/rails if you are into that), the better server will be used automatically!

Do you have other tips or tricks to speed up your Rails development environment with Vagrant? Leave a comment or link below and I will update the post. I hope you enjoyed this post and look forward hearing stories about your increased productivity!