DevOps and Chef at DNSimple
Aaron Kalin
on
Managing and sending out changes to over 50 servers is difficult. Each of these servers are distributed out across 5 data centers in 3 separate countries spanning the globe. How do you coordinate changes across all those systems? We use Chef, a configuration management tool to help make this difficult problem a lot easier for the DNSimple team to tackle.
This post will the first of many in a series about how we are using Chef and some of its tools to manage our growing and changing infrastructure. Using Chef isn't just about managing the servers themselves, but also making sure your team can help manage those chages too. We'll also detail how our drive to foster a DevOps culture helped us contribute to Chef and how you can use this contribution at your company.
DevOps at DNSimple
Culture & Communication
We're a small team of 5 at DNSimple distributed around the globe. While our team size does not afford us the luxury of having a large, dedicated team of system administrators. We have embraced this limitation by creating a DevOps Culture at DNSimple. It's a constraint that has helped us be a lot more resourceful with the limited development resources we have.
One of the primary goals of a DevOps culture is to foster communication between the ops and development teams. We have written about how we work remotely as a company and have built a great remote culture. As a small team, we have shared responibility over everything that extends into server operations. Everyone is responsible for what they ship and services like PagerDuty to respond to server outages is also shared by the team in case of a problem. It's the job of myself and Jacobo to communicate everything infrastructure to the rest of the team. If they can't understand our infrastructure or feel confident that a deploy will work, we need to work harder to make sure that is not the case.
Using Tools
Tools can help create and support workflows for your team to do their jobs more effectively. They aren't the one thing that will create a DevOps culture, but they can help support it. As we mentioned above, we use Chef everywhere in our infrastructure along with Test-Kitchen to make sure we can reliably test our changes before deploying to production. We have also begun converting a lot of our Chef cookbooks to use Berkshelf and making sure it can integrate with our workflow.
More recently, we have started to use the new Chef Development Kit to start unifying a lot of these tools and keep everyone's environment more consistent. We are always looking for more ways to make it easier for the other developers on the team to feel more confident when working with Chef. This need for accessibility and productivity gave us the drive to contribute to Chef DK recently.
A new Chef DK feature
Early on when I started with DNSimple, one of the problems we faced was generating new cookbooks consistently. Since we had just begun migrating cookbooks to Berkshelf and leveraging the Wrapper Cookbook Pattern. One of the pain points we hit was consistently generating cookbooks with all the things we cared about. The right licensing, testing tools, and plugins where all tedious to get right the first time. We definitely tried to use the Berkshelf cookbook generator, but it always required more tweaking past generation and did not allow custom generator templates to make things easier to create.
Around this time, Chef DK 0.1.0 released and I went digging into the internals to find out how it generated cookbooks. Much to my surprise(and delight) I noticed they were actually using a built-in chef cookbook that was called upon by Chef to generate the cookbook. It's Chef running a cookbook to generate a cookbook. Is your mind blown? Mine was! Despite this amazing discovery, I also noticed that the cookbook name and path were both hard-coded into the library, making it difficult to for us to make our own generator cookbook. This is when I created a pull request to at least make it possible to change where the template cookbook was located.
As of a few days ago, the release of Chef DK 0.2.0 now includes a new --generator-cookbook feature to let you point at a directory containing a generator cookbook. How do you use this new feature? I'll give you a quick tutorial.
A quick, custom generator cookbook
Requirements: Chef DK 0.2.0 installed
You can get started quickly by simply copying the current 'skeleton' cookbook hidden within the Chef DK. Here is an example command from a linux/unix system cp -R /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-dk-0.2.0/lib/chef-dk/skeletons/code_generator . which copies the cookbook to your current directory. That huge path is to get to the current version of the Chef DK gem and the hidden cookbook inside.
Something to note in case you want to make a new generator cookbook from scratch or heavily modify this one. You will have to make sure the cookbook name is code_generator along with all the recipes by name that it is expecting depending on the type of generator you are calling. Cookbook, cookbook file, and lwrp are a few examples, but you can look in the recipes directory to see what is currently used. This may change in the future with more pull requests to open up control of what recipes within the generator cookbook are called, but for now, you'll have to stick with what it is expecting.
Something else to note is that the generator cookbook is just that, a cookbook. You can use Chef's resources that you typically use with Chef Solo (no search or data bags) to generate the cookbook you want to create. Dig into the templates or files directories of the skeleton cookbook and go crazy. As long as your recipe calls it, it will get generated.
Finally, to get the chef generate command to call your new skeleton cookbook just use the new --generator-cookbook command with a path to where your code_generator cookbook resides. In my quick example, I have it on the Desktop so I'll run chef generate cookbook dnsimple --generator-cookbook '~/Desktop' and it will build a new cookbook because it needs to find a code_generator folder in the Desktop folder to load it. Please note you can use the short-form -g option if you don't like typing out the --generator-cookbook option.
To make sure everyone at DNSimple can use this template cookbook, we are storing it in our main Chef repository along side our data bags, roles, and environments. It'll also contain some instructions in the README on how to perform the command I illustrated above. It's not the most friendly command to run, but this can later be wrapped with something like Thor to make it easier.
Conclusion
We have much more to cover on the topics of Chef and DevOps at DNSimple. You also got to see a little more about infrastructure at DNSimple along with how we are contributing to not just help our team, but improve the Chef community as well.
Aaron Kalin
Software and Server maintainer by day, board and video game geek by night.
We think domain management should be easy.
That's why we continue building DNSimple.
