API v2 Architecture and Hanami

Our current API (v2) is now in General Availability, and it deprecates the previous version (v1).

While our main flagship application is built with Ruby on Rails, the team wanted to develop a revised version of the DNSimple API with a separate context from Rails to reduce the tight coupling between the web UI and the API.

At that time our CTO Simone was prototyping the new version of the API with Sinatra. We often meet for a coffee and hack together from time to time. I wasn't working at DNSimple yet. One day, over a coffee, Simone showed me the branch of the API prototype and said to me: what if we try to build it with Hanami? In an hour we had an initial prototype up and running.

Hanami was in its very early stage (it was still called Lotus), but as an experiment we also decided to benchmark the new API prototype. We implemented the exact same features in both Sinatra and Hanami, and we ran some benchmarks to measure basic cases: the fetch of a single domain, a 404 HTTP error, and a failed authentication. Both Simone and I didn't know what to expect.

Surprisingly, Hanami was performing very much like Sinatra. Simone took note of the results, and added them to a git commit.

commit 913002c510f7cbf876cad87ba0354121cfa57fd7
Author: Simone Carletti <weppos@weppos.net>
Date:   Fri Jan 16 14:11:38 2015 +0100

    Lotus-based API test

    Here's the results of the benchmarks with Lotus and Sinatra.

    ## 200 request

        ##### Lotus

        ➜  Desktop  wrk -t2 -d30s -H 'Authorization: Basic
    ZXhhbXBsZUBleGFtcGxlLmNvbTpzZWNyZXQ='
    http://127.0.0.1:9292/api/lotus/5/domains/3
        Running 30s test @ http://127.0.0.1:9292/api/lotus/5/domains/3
          2 threads and 10 connections
          Thread Stats   Avg      Stdev     Max   +/- Stdev
            Latency   995.65ms   53.10ms   1.30s    95.65%
            Req/Sec     4.76      0.48     5.00     78.26%
          300 requests in 30.04s, 190.14KB read
        Requests/sec:      9.99
        Transfer/sec:      6.33KB

        ##### Sinatra

        ➜  Desktop  wrk -t2 -d30s -H 'Authorization: Basic
    ZXhhbXBsZUBleGFtcGxlLmNvbTpzZWNyZXQ='
    http://127.0.0.1:9292/api/v2/5/domains/3
        Running 30s test @ http://127.0.0.1:9292/api/v2/5/domains/3
          2 threads and 10 connections
          Thread Stats   Avg      Stdev     Max   +/- Stdev
            Latency   992.24ms   57.63ms   1.05s    97.50%
            Req/Sec     4.50      1.44     7.00     45.00%
          300 requests in 30.05s, 189.84KB read
        Requests/sec:      9.98
        Transfer/sec:      6.32KB

        ## 404 request

        ##### Lotus

        ➜  Desktop  wrk -t2 -d30s -H 'Authorization: Basic
    ZXhhbXBsZUBleGFtcGxlLmNvbTpzZWNyZXQ='
    http://127.0.0.1:9292/api/lotus/5/domains/3000
        Running 30s test @ http://127.0.0.1:9292/api/lotus/5/domains/3000
          2 threads and 10 connections
          Thread Stats   Avg      Stdev     Max   +/- Stdev
            Latency   988.60ms   51.32ms   1.12s    89.29%
            Req/Sec     4.45      1.26     8.00     51.19%
          301 requests in 30.05s, 74.96KB read
          Non-2xx or 3xx responses: 301
        Requests/sec:     10.02
        Transfer/sec:      2.49KB

        ##### Sinatra

        ➜  Desktop  wrk -t2 -d30s -H 'Authorization: Basic
    ZXhhbXBsZUBleGFtcGxlLmNvbTpzZWNyZXQ='
    http://127.0.0.1:9292/api/v2/5/domains/3000
        Running 30s test @ http://127.0.0.1:9292/api/v2/5/domains/3000
          2 threads and 10 connections
          Thread Stats   Avg      Stdev     Max   +/- Stdev
            Latency     1.32s   336.40ms   2.37s    84.00%
            Req/Sec     3.69      1.41     6.00     73.33%
          235 requests in 30.04s, 69.08KB read
          Non-2xx or 3xx responses: 235
        Requests/sec:      7.82
        Transfer/sec:      2.30KB

        ##### No auth request

        ##### Lotus

        ➜  Desktop  wrk -t2 -d30s -H 'Authorization: Basic
    ZXhhbXBsZUBleGFtcGxlLmNvbTpzZWNyZXQ='
    http://127.0.0.1:9292/api/lotus/5/domains/3
        Running 30s test @ http://127.0.0.1:9292/api/lotus/5/domains/3
          2 threads and 10 connections
          Thread Stats   Avg      Stdev     Max   +/- Stdev
            Latency   278.73ms   26.62ms 332.84ms   64.71%
            Req/Sec    17.35      0.65    18.00     47.06%
          1047 requests in 30.03s, 665.62KB read
        Requests/sec:     34.86
        Transfer/sec:     22.16KB

        ##### Sinatra

        ➜  Desktop  wrk -t2 -d30s -H 'Authorization: Basic
    ZXhhbXBsZUBleGFtcGxlLmNvbTpzZWNyZXQ='
    http://127.0.0.1:9292/api/v2/5/domains/3
        Running 30s test @ http://127.0.0.1:9292/api/v2/5/domains/3
          2 threads and 10 connections
          Thread Stats   Avg      Stdev     Max   +/- Stdev
            Latency   298.15ms   30.56ms 352.42ms   53.41%
            Req/Sec    16.61      1.24    20.00     46.59%
          1028 requests in 30.04s, 653.54KB read
        Requests/sec:     34.22
        Transfer/sec:     21.76KB

We merged the test branch into the prototype, and Simone decided to continue developing the API v2 using Hanami. The architecture used today in production for our API v2 still contains fragments from that commit.

The API Application

The new API is built only with two Hanami components: the router and the actions. They are both compatible with Rack, a protocol for Ruby web applications. Thanks to this protocol, we can mount this application inside the flagship application.

# config/routes.rb
scope as: 'api', ... do
  # ...
  mount Api::V2::App.new, at: '/v2'
end

The /v2 path prefix is delegated to Api::V2::App, which has its own set of routes.

The application itself is really simple:

require_relative 'router'

module Api::V2
  class App
    attr_reader :router

    def initialize
      @router = Router.new(
        namespace: Api::V2::Controllers,
        routes: 'app/api/api/v2/routes.rb'
      )
    end

    def call(env)
      router.call(env)
    end
  end
end

The Router

The router is responsible for accepting incoming HTTP requests and dispatching them to the proper action. If the requested path is unknown, it returns a Not Found error (404).

For any request that sends a JSON payload, the router parses the payload. If the payload invalid, the parser raise an exception that is turned into a generic Client Error (400), otherwise the router passes that payload to the action as parameters.

require 'hanami/router'

module Api::V2
  class Router < Hanami::Router
    PARSERS   = [:json].freeze
    NOT_FOUND = ->(_) { [404, { ... }, ['{"message":"Not Found"}']] }

    def initialize(namespace:, routes:)
      # ...
      super(
        namespace:   namespace,
        parsers:     PARSERS,
        default_app: NOT_FOUND,
        # rubocop:disable Security/Eval
        &eval(File.read(routes))
      )
    end

    def call(env)
      # instrumentation code
      super
    end
  end
end

Actions

Shared Configurations

Hanami actions are objects. It's possible to share common code across all the actions of an application. Behaviors like rendering logic, authentication, rate limiting, error handling, and precondition checks are all elegantly configured in one place and can be used as needed in the actions.

Hanami::Controller.configure do
  # ...

  prepare do
    include Accept
    include Rendering
    include Errors
    include NotFoundHandler
    include AccountIdCheck
    include Authentication
    include Subscription
    include Throttling
    include Features

    # ...
  end
end

The prepare block is evaluated when Hanami::Action mixin is included during the boot process. With this technique, each single action will include the specified modules.

This code organization is designed to visually understand which behaviors a single action exposes. Each module implements one and only one responsibility. Some of them ignore the rest of the other modules, while a few of them are dependent on each other. For instance Subscription depends on Errors.

Let's have a closer look at one of them: Authentication.

module Api::V2
  module Authentication
    module Skip
      private

      # Empty method
      def authenticate!
      end
    end

    def self.included(base)
      base.class_eval do
        before :authenticate!
      end
    end

    private

    def authenticate!
      # authentication code ...
    end
  end
end

When Authentication is included in an action, it enables a callback that checks if the current request is authenticated or not. This enables authentication for all actions.

But what if we want to skip the check for a single action? We include Skip which overrides the original #authenticate! with a no-op method. When the before callback invokes this method the authentication check is not performed.

module Api::V2
  module Controllers::Oauth
    class AccessToken
      include Hanami::Action
      include Authentication::Skip

      # ...
    end
  end
end

An Example Action

The result of this design is a clear set of actions, each of them has a single goal that is understandable at the first glance.

module Api::V2
  module Controllers::Webhooks
    class Create
      include Hanami::Action
      require_feature! Feature::WEBHOOKS

      def call(params)
        @result = WebhookCreateCommand.execute(...)
        @webhook = @result.data

        if @result.successful?
          render Serializers::WebhookSerializer.new(@webhook), 201
        else
          render Serializers::ErrorSerializer.new(@result.error, @webhook), 400
        end
      end
    end
  end
end

For each endpoint we use a specific type object that we call a command: if the operation has a failing outcome, we render an error, otherwise we return a successful result and serialize the object.

For more about our command objects you can read the post Why we ended up not using Rails for our new JSON API.

Conclusion

The API v2 architecture is the result of two years of work. We were able to deploy a maintainable solution without sacrificing performance. This was possible by the flexibility of Ruby, the elegance of Hanami, and a maintainable architecture.

Share on Twitter and Facebook

Luca Guidi

Former astronaut, soccer player, superhero. All at the age of 10. For some reason now I write code.