Berkshelf Workflow

There are only two fundamental assumptions for working with Berkshelf:

1. Each cookbook is a uniquely packaged and versioned artifact
2. You have a centralized artifact store that exposes a dependency API and/or is indexable by the Berkshelf API

Each cookbook is it own unit of infrastructure and should be treated as such. There are certainly situations where two or more cookbooks need to be iterated on in tandem, but those situations should be few and far between. In this article, I will discuss the procedure by which our team handles such situations using Berkshelf and the "GitHub flow".

Internally on the Release Engineering team at CHEF, we commonly experience this type of issue in our continuous integration (CI) environment. We use the community omnibus cookbook to setup builders for our CI, but we have an internal cookbook called "opscode-ci" that wraps that cookbook and tunes attributes, etc. The opscode-ci cookbook has a metadata.rb that looks something like:

name 'opscode-ci'
depends 'omnibus', '~> 3.0'

As you know, this metadata entry tells Berkshelf to pull the latest published artifact of the omnibus cookbook from the CHEF community site that satisfies the ~> 3.0 constraint. When we want to make changes to the omnibus cookbook, we want that update to coincide with an update of the opscode-ci cookbook. First, we create a new branch off of the latest master branch in the omnibus cookbook.

~/cookbooks/omnibus | $ git checkout -b sethvargo/add_new_feature

As part of our workflow, we prefix branches with our GitHub username. This is especially useful because it helps track down the "owner" of a branch without diving into commits. This tidbit is unrelated to the Berkshelf workflow.

Next, we make the appropriate changes to our branch, push the code to GitHub, and collaborate via Pull Requests. We run the automated test suite included in the omnibus cookbook at each stage of this process, adding more test coverage as necessary.

~/cookbooks/omnibus | $ git add .
~/cookbooks/omnibus | $ git commit -m "Make awesome changes"
~/cookbooks/omnibus | $ git push

After we are generally happy with the changes, we want to test those changes against opscode-ci, before merging to master or publishing a new version. In order to acheive this, we use Berkshelf to temporarily point at the GitHub location using Berkshelf:

source 'https://api.berkshelf.com'
metadata
+
+ cookbook 'omnibus', github: 'opscode-cookbooks/omnibus', branch: 'sethvargo/add_new_feature'

Note that we leave the existing metadata.rb intact, adding this entry to the end of the existing Berksfile in the cookbook. Next we tell Berkshelf to pull in the branch of our omnibus cookbook.

~/cookbooks/opscode-ci | $ berks update omnibus

This will update the Berksfile.lock (which we checkin to version control) and download the cookbook from GitHub into the cookbook store. Next we use Test Kitchen to converge a test node. Test Kitchen will automatically detect the presence of a Berksfile and pull in the GitHub version of the omnibus cookbook. Next we conduct all of our testing. In this particular example of the opscode-ci cookbook, we have both an automated test suite and manual smoke tests. If we find a bug in the omnibus cookbook changes that was not caught in testing, we:

1. Add a new test for it in the omnibus cookbook (because it should have been caught earlier)
2. Commit and push to master from the omnibus cookbook
3. Run berks update omnibus from opscode-ci to pull in the newer version from GitHub
4. Repeat as necessary

Ideally we will never find a bug this late in the game, but we are humans, and mistakes happen. Hopefully those edge cases are few-and-far between. When we are satisfied with our changes to the omnibus cookbook, we tag a new version following semver, push the tag to GitHub, and publish a packaged artifact to the community site.

Back in the opscode-ci cookbook, we remove the temporary line in our Berksfile:

-
- cookbook 'omnibus', github: 'opscode-cookbooks/omnibus'

If necessary, we bump the required minimum version constraint in the metadata.rb and run berks update omnibus. This will update the Berksfile.lock to point at the correct version of the omnibus cookbook on the community site.

~/cookbooks/opscode-ci | $ berks update omnibus

We check the Berksfile.lock into version control too:

~/cookbooks/opscode-ci | $ git add Berksfile.lock
~/cookbooks/opscode-ci | $ git commit -m "Update omnibus to vX.Y"

Finally, we cut a new release of the opscode-ci cookbook and push it to our internal private artifact store. At this point, we may choose to deploy (which is a simple berks upload), or we may choose to wait for a bigger changeset.

Important notes:

• We never publish an unfinished artifact
• We never publish a cookbook that depends on an SCM location
• We heavily use Test Kitchen + VMWare Fusion for local testing

Further reading:

• Jamie Winsor's The Berkshelf Way
• Jamie Winsor's The Berkshelf Vision
• Berksfile Magic
• Using Berkshelf on AWS Opsworks
• Moving to Individual Cookbooks with Berkshelf