Unit Testing Chef Cookbooks

Okay, now that I'm done ranting about how to Unit test, let's move onto Chef.

I spoke at Chef Summit a few months ago and received a lot of questions about ChefSpec. It's very difficult to demonstrate the value in a Unit test when everyone is thinking at a higher level (acceptance testing).

Let's say I have a simple cookbook that just installs apache:

package value_for_platform(
  %w(centos redhat suse fedora) => {
    'default' => 'httpd'
  },
  %w(ubuntu debian) => {
    'default' => 'apache2'
  }
)

And if we were to test this with ChefSpec and Fauxhai:

require 'spec_helper'

describe 'my_cookbook::default' do
  platforms = {
    'ubuntu' => {
      'package'  => 'apache2',
      'versions' => ['10.04', '12.04']
    },
    'debian' => {
      'package'  => 'apache2',
      'versions' => ['6.0.5']
    },
    'centos' => {
      'package'  => 'httpd',
      'versions' => ['5.8', '6.0', '6.2', '6.3']
    },
    'redhat' => {
      'package'  => 'httpd',
      'versions' => ['5.8', '6.3']
    }
  }

  platforms.each do |platform, (package, versions)|
    versions.each do |version|
      context "On #{platform} #{version}" do
        before do
          Fauxhai.mock(platform: platform, version: version)
        end

        let(:chef_run) { ChefSpec::ChefRunner.new.converge('my_cookbook::default') }

        it 'installs the apache2 package' do
          chef_run.should install_package package
        end
      end
    end
  end
end

Here's the part that's difficult to grasp - this isn't checking that the package was installed! This is checking that Chef was instructed to install a package. In other words:

chef_run.should install_package('foo') => expect(chef_run).to install_package('foo')

This is more than a semantic difference - it's a fundamental different way of thinking. We need to start thinking about messages, not results (in unit tests).

In other words, your test is "Did I tell Chef to install the package?", not "Did Chef install the package?". Chef already tests for that! If you tell Chef to perform an operation (such as installing a package), it will fail the Chef run if it doesn't succeed. There's no reason to check that package exists - if the Chef run completed, it's there.

So then why unit test at all?

Answer: regression. If you tell Chef to install a package, it will install a package... But what if you don't? Or, what if you accidentally change the way a file is written out during a refactor? That's what your unit tests will catch!

Consider our last example - I want to write out a template for an apache site:

package value_for_platform(
  %w(centos redhat suse fedora) => {
    'default' => 'httpd'
  },
  %w(ubuntu debian) => {
    'default' => 'apache2'
  }
)

template "#{node['apache']['dir']}/sites-avaliable/my-site.conf" do
  source    'apache2/sites/my-site.conf.erb'
  mode      '0755'
end

This is a perfectly valid Chef recipe. You can upload it, run chef-client and it will fail. Can you spot why? Let's write a unit test to catch the error before it hits production:

describe 'my_cookbook::default' do
  platforms = {
    'ubuntu' => {
      'package'  => 'apache2',
      'versions' => ['10.04', '12.04'],
      'dir' => '/etc/apache2'
    },
    'debian' => {
      'package'  => 'apache2',
      'versions' => ['6.0.5'],
      ,'dir' => '/etc/apache2'
    },
    'centos' => {
      'package'  => 'httpd',
      'versions' => ['5.8', '6.0', '6.2', '6.3'],
      'dir' => '/var/httpd'
    },
    'redhat' => {
      'package'  => 'httpd',
      'versions' => ['5.8', '6.3'],
      'dir' => '/var/httpd'
    }
  }

  platforms.each do |platform, (package, versions, dir)|
    versions.each do |version|
      context "On #{platform} #{version}" do
        before do
          Fauxhai.mock(platform: platform, version: version)
        end

        let(:chef_run) { ChefSpec::ChefRunner.new.converge('my_cookbook::default') }

        it 'installs the apache2 package' do
          chef_run.should install_package package
        end

        it 'create my-site.conf' do
          file = File.join(dir, 'sites-available', 'my-site.conf')
          chef_run.should create_file file
        end
      end
    end
  end
end

The tests fail, and you may still be scratching your head... I spelled "available" incorrectly in "sites-available"!

It's just as easy to accidentally remove a line when refactoring, spell a username wrong, or change file permissions to the wrong mode. In most of these cases, the Chef run will still complete successful. They are hidden time bombs.

You could remove a line that writes out a template. All your existing servers will continue to function (because the file was already written out in the past). New bootstraps will fail and you'll be left scratching your head.

About Seth