Updates

New Chef Cookbook Release

Aaron Kalin on May 16, 2017

I'm pleased to announce the release of our re-written Chef Cookbook to the Chef Supermarket. This was the result of many months of very hard work and help from a few of the team members and the Chef Community. In this post I'll detail what has changed and for the super technical folks out there, I'll do a deep dive into the technical challenges of this cookbook.

What's new?

Simply put, we've replaced the usage of the fog gem with our official ruby client. This means you'll now be using the v2 api since the fog gem has not yet been updated from the v1 api. It also means the dependencies have dropped significantly in the cookbook. We no longer require specific build packages per platform, etc. It also helps pave the way for some exciting changes in the future for this cookbook.

The other thing you'll notice since it is related to changing from API v1 to v2 is we no longer require a username/password or even the older API v1 tokens. From now on you'll need to generate a v2 API account token to use this cookbook. How do you obtain one? You can do this programmatically by following this guide or via the website by following this guide. Make sure you are generating an account access token. User tokens are not supported at this time.

One more subtle change is the terminology for the resource. What we used to call "zone" is now "domain" for example. They are the same thing, but the naming is important for the cookbook update. We also now support updating a given DNS record which I'll give some examples of below.

One final technical note here is that our cookbook now requires Chef Client 12.8 or newer. The reason for this will be noted in the technical detail section below. If you're not on Chef 12 yet, you'll want to be there soon because Chef 11 will be deprecated soon.

I want to use this right now!

Of course you do! After you have generated your v2 API account token via one of the two methods mentioned earlier, you'll simply need to add the dnsimple dependency to your cookbook's metadata.rb like so:

cookbook 'dnsimple'

Once you have done that, you're ready to start using our custom resource! Easy, yeah?

This is where you'll want to write a recipe which will update your records via Chef. Let's say I just bought dnsimple.io and wanted to point that domain to a new server. Here is how I'd write the default.rb recipe:

dnsimple_record 'webserver' do
  domain 'dnsimple.io'
  type 'A'
  content '1.2.3.4'
  ttl 3600
  access_token 'your_access_token_goes_here'
  action :create
end

What this does is create a new 'A' record on the apex (which is the domain itself, no subdomain) with the value '1.2.3.4' and a default TTL of 3600. I've omitted my access token for obvious security reasons, but as an operator, you'll likely want to secure this important credential via something like chef-vault.

Maybe you want this record to always update itself in case you re-build the server or move it to another provider. Not a problem!

dnsimple_record 'webserver' do
  domain 'dnsimple.io'
  type 'A'
  content '1.2.3.4'
  ttl 3600
  access_token 'your_access_token_goes_here'
  action :update
end

What about a new mailserver you just setup with Chef? This will search for the new node in Chef and update it if it should ever change or get rebuilt.

dnsimple_record 'new_mailserver' do
  domain 'dnsimple.io'
  type 'MX'
  priority 10
  content search(:node, 'roles:mailserver')[0]['ipaddress']
  ttl 3600
  access_token 'your_access_token_goes_here'
  action [:create, :update]
end

That is really how simple it is to automate your DNS records with Chef and DNSimple.

Technical Deep Dive

For the curious out there, I want to dive into the technical challenges of doing this re-write. We originally intended to use the all new Chef 12.5 custom resource model, but we quickly realized at least for an API client this was an issue.

Who's mock is it anyway?

One of the first hurdles we had to clear was getting our new resource testable with the dnsimple gem via ChefSpec. I generally like this path for creating new resources because its a much faster feedback loop than the integration level Test Kitchen. While it is a fast path, our first hurdle to clear was how to stop the dnsimple gem from making outbound connections to our API.

We had a few options here with one of them being using something like webmock or vcr which tries to simulate a real outbound call using fixture files. Going this route now meant we had to re-create or copy over a bunch of already existing mocks from our library. We're now managing the mocks on top of managing our interface to our own library. Seem ridiculous? We thought so too.

The other option we had was to go the route of dependency injection, which is a well-known technique for testing at the edges of objects. The short explanation of this techique is that you "inject" a test double into your subject under test, which in this case is our new resource. While this sounds simple enough to do, Chef does not seem to have an easy way to do this due in part to how their resource DSL is made. We had to fall back to a more advanced technique for resource building where we subclass provider and resource in chef directly via ruby. Beware of going this route in Chef as it's largely undocumented right now in terms of how you need to implement their interface without the magical help of their newer 12.5+ syntax.

To help handle the possibility that our test double could eventually lie to us, we enabled a newer rspec feature to verify instance doubles which will raise exceptions if the doubled class changes from underneath us. If you take a closer look at the pull request you'll see a pretty detailed progression of how we tried the first method of mocking and ultimately went with dependency injection. This method of testing also revealed a lot about our API design as it took 8 lines of test doubles just to get to a zone record for example. However, we now had a very fast testing loop and a rather high degree of confidence that we were integrating our API correctly. No test fixtures to manage, ever.

InSpec for fun and profit

This time around to help facilitate our integration level verification, we leveraged the awesome new InSpec from Chef. However, there isn't a built-in control to verify an API based resource. After all, we're commanding an API (ours) to make DNS changes. How do we know they worked? This is where custom inspec libraries came into use for us.

If you take a look inside the test/integration folder of the cookbook, you'll notice two basic tests. One for creating a record and another for updating. Why not destroy? I'll get to that later. For now look into one of the folders and notice we have a libraries folder which allows us to build a custom inspec control. Think of these like custom matchers for ChefSpec because they're very similar. We wrote one that leveraged our gem to make the API calls to verify our changes were correct on the other side. One thing to note about using these is that you must provide a inspec.yml file for Inspec to pickup on the library. It's also good to note that libraries currently cannot be shared across controls and tests. This is something Chef is looking into as I think our cookbook is a fantastic use case for this.

One other small detail we had to handle was passing the secret API token for our Sandbox Environment to our Inspec and Chef test cookbook runs. This is where Inspec Profile Attributes and environment variables come in handy. Using the encrypted variables in Travis-CI we were able to pass in the sensitive API token for testing via these variables which are then utilized in the kitchen configuration along with a test domain of our choosing to make the integration tests more flexible overall.

Test. Rinse. Repeat.

One final thing we had to do was make sure our integration testing environment was reset every time we went through and updated or modified records and tested them with Inspec. The way we did this was to test the delete action of our new resource by way of a reset recipe in the test cookbook. If this recipe is not run at the start of the test, it will fail the Chef run and thus break our builds due to a record not existing or already existing for us. You can see this recipe in the .kitchen.yml file of our cookbook repository and under the test/cookbook directory which has great examples of how to use the new resource along with how we reset the testing environment. I can even verify the changes occured by looking at the account activity in our sandbox environment.

Go check it out!

This cookbook rewrite was a bigger task than I ever imagined and we learned a great deal along the way of how Chef has vastly improved the experience for making custom resources in Chef. There is still more work to be done and I'll be working with them to improve this method of writing resources. It's not the easy path, but I'd like to make it easier in the future. Please go check out this cookbook for yourself and again if you need any support from us, please email support to get assistance from us. We'd love to know how you are automating your domains with us!

Share on Twitter and Facebook