Auto-publishing a static site to CloudFront using Travis

At DNSimple we love automation, and we use Travis to test almost any piece of code we write. And not just the code.

A few years ago we started to use Travis to build and package our Erlang and Go code after a successful build. More recently, we experimented on using Travis to auto-deploy our static-generated sites to CloudFront.

I must confess I was a little bit skeptical at the beginning (I'm definitely not a fan of continuous deployment in general), but the result of the experiment was successful and we'll likely expand it to the rest of our static sites. So here's the instructions, in case you want to try the same deployment system for your sites.

Background

Before starting, a little bit of background information. We currently have a few static sites, generated using Jekyll or Nanoc from a Markdown repo: the blog, the developer documentation and the support site.

All these sites are deployed to Amazon CloudFront (over an Amazon S3 bucket) to be able to serve them via HTTPS and using a CDN. For the deploy we have been using the s3_website gem for quite a while. This gem provides a configurable library to publish the content of a folder to AWS, with a bunch of extra settings such as redirects or CloudFront distributions. The AWS keys are securely stored in two environment variables that are securely read by the gem at runtime during the deploy.

All the static sites are tested using Travis. This may be surprising to you, as you may be wondering what you could possibly test on a static site. I'll talk about that in a minute.

Deployment flow

The deployment flow using Travis is quite simple.

1. Changes to are committed to the GitHub repo
2. Travis runs the tests
3. If tests are successful, Travis generates the static site
4. If the generation is successful, Travis deployes the site using the s3_website gem and given settings

Configuration

In this section I'll talk about the various configs.

Configuring the site

This particular configuration largely depends on your static generator tool. There are a number of static site generators today: Jekyll, Hugo, Nanoc and many more.

All these tools provide you a single command to build your site to a local folder. In the case of Jekyll the command is:

$ jekyll build

The command will generate the site in a folder called _site. You can indeed change the folder passing the name as --destination. For Nanoc the command is nanoc compile.

For simplicity, we generally wrap this command into a custom Rake task called compile we can call with $ rake compile. Here's a simple example:

desc "Compile the site"
task :compile => [:clean] do
  puts "Compiling site"
  out = `nanoc compile`

  if $?.to_i == 0
    puts  "Compilation succeeded"
  else
    abort "Compilation failed: #{$?.to_i}\n" +
          "#{out}\n"
  end
end

task :clean do
  FileUtils.rm_r('output') if File.exist?('output')
end

Configuring the tests

Having a test suite for your static site is important for two reasons:

1. You can rely on Travis for publishing your site (as I'll show later on)
2. You can test the correct compilation of the site, and you can also create specific tests to enforce publishing conventions

For a very simple test suite, you can check our public dnsimple-developer repo. The tests are in a folder called _test, to avoid conflicts with the static sites (names prefixed with _ are generally not compiled as content).

The test suite consists in a super simple Minitest helper:

# _test/test_helper.rb

require 'rubygems'
require 'bundler/setup'
require 'minitest/autorun'
require 'minitest/reporters'

Minitest::Reporters.use!

and one single test suite that enforces some conventions:

# _test/content_test.rb

require 'test_helper'

describe "Content" do

  BannedCharacters = [
    "\u201C", # U+201C
    "\u201D", # U+201D
    "\u201E", # U+201E
    "\u2018", # U+2018
    "\u2019", # U+2019
    "\u201A", # U+201A
    "\u201B", # U+201B
  ]

  it "is sucessful" do
    regexp   = /[#{BannedCharacters.join}]/
    affected = []

    Dir["content/*"].each do |path|
      next unless File.file?(path)
      File.readlines(path).each do |line|
        affected << [path, line] if regexp.match(line)
      end
    end

    message  = "#{affected.size} lines contain banned characters:\n"
    affected.each_with_index do |(path, line), index|
      message << "#{("%d." % (index+1))} #{path} -> #{line}"
    end

    assert_equal 0, affected.size, message
  end
end

Specifically, we don't want the content to include any "Fancy curly quote". These are not valid ASCII chars, and they are generally the result of cutting-and-pasting from rich text editors.

Of course, you can add as many tests as you want. We also test that the site compiles correctly, but we generally rely on the Rake tasks and exit code for this (more later in the deploy configuration).

Last but not least, we also have a Rake test task we use to run the tests:

# Rakefile

Rake::TestTask.new do |t|
  t.libs << "_test"
  t.test_files = FileList["_test/*_test.rb"]
  t.verbose = true
end

Configuring the deployment

As previously mentioned, for the deployment we use the s3_website gem. You can write your custom deployment procedure if you don't like this gem or if you need to deploy to a different service, but for S3 this gem works just fine.

Follow the setup instructions to configure the s3_website tool. You will have to generate an s3_website.yml in your root folder containing the deployment settings.

WARNING: make sure to not include your Amazon credentials in the file, especially if the repo is public. You can use environment variables to pass the settings at runtime.

s3_id: <%= ENV['S3_ID'] %>
s3_secret: <%= ENV['S3_SECRET'] %>

As we did for the other tasks, we generally package all the deployment instructions into a single Rake task called publish. The task depends on the compile task we previously defined.

desc "Publish to S3"
task :publish => :compile do
  puts "Publishing to S3"
  puts `s3_website push`
  puts "Published"
end

At this point, you should have three Rake tasks:

• $ rake test runs the test suite
• $ rake compile compiles the site into the output folder
• $ rake publish compiles the site and publishes it

All tasks should return an exit code 0 if (and only if) the task completes successfully.

Configuring Travis

Now we have all the tasks we need to compile, test and deploy the site. We just need to instruct Travis to deploy the site for us.

In the .travis.yml file we use the after_success: hook to tell Travis what to do after a successful build.

after_success:
- test $TRAVIS_PULL_REQUEST == "false" && test $TRAVIS_BRANCH == "master" && rake publish

Specifically, if the build is not a pull request, the branch is master, and the build is successful, then go ahead and deploy the site using the $ rake publish command. The test conditions ensure we don't deploy a pull request, or a branch which is not the main branch.

To make sure the tests are automatically run by Travis, we configure the default Rake task to run the tests and to try to compile the site:

# Rakefile

task :default => [:test, :compile]

In this way, if the compilation fails (e.g. because of a syntax error in the site), or a test fails, the site will not be deployed.

As mentioned, we used environment variables to store the Amazon credentials for the deploy. You can safely store them in Travis using the repository setting section.

Here's an example of a successful build and deploy via Travis.

Example

You can see a live example on our public dnsimple-developer repo.

• The Rakefile
• The _test folder
• The Travis deployment code

Wrapping up

It is worth mentioning that Travis has a large number of pre-defined deployment recipes you can use to deploy your code to a service. However, in our case we needed a customized version as we've been using certain settings for a while.

Deploying your static generated website using Travis is very simple. You can use Rake tasks to wrap the various recipes, and keep your Travis configuration files independent from the underlying service.

In general, you need:

• A task to run the tests
• A task to publish the site
• A task to compile the site
• A default task that runs the tests and the compilation, and exists with a non-zero status code on failure

You can use any type of programming language, static site generator, and target service. In our case the example shows you how to use Ruby (and Nanoc) to compile a static-generated site, run a Ruby (Minitest) test suite and deploy the site to Amazon CloudFront using the s3_website gem.

Share on Twitter and Facebook

Simone Carletti

Italian software developer, a PADI scuba instructor and a former professional sommelier. I make awesome code and troll Anthony for fun and profit.