Why we ended up not using Rails for our new JSON API
Luca Guidi
on
When we initially launched our API v1, we knew that it was just the first step towards our vision of an extensive domain management automation API to support our platform for DNS and domain automation. API v1 was designed as a bridge between the original API v0 and a new set of redesigned API. With years of API v1 under the belt, we had a clear picture of what worked and what didn't, and we were ready to start working on a new, completely redesigned API.
We decided to not evolve that system, but to start fresh with a new architecture.
API v1 was built with a classic Ruby on Rails app: complex business logic spread across models and controllers, rendering logic delegated to respond_to, and ActiveRecord serializers. This structure was inherited from API v0, where we adopted the standard Rails way of serving different responses from the same controller that was quite common back in 2010.
This structure, however, introduces several maintenance and scalability issues. Here are some problems we found and how we solved them.
Chaotic Code Organization
Without a clear guidance from the framework, and with no explicit team guidelines, the code for complex use cases was randomly scattered across controllers and models. The domain registration system had a different organization from the domain renewal, which had a different system than domain transfers, and so on. As you can imagine, this made maintenance hard in a large codebase like ours.
Here's an example of how the registrations controller used to be back in 2011:
class DomainRegistrationsController < ApplicationController
helper :domains
before_filter :require_user
before_filter :setup_name_and_domain
before_filter :require_subscription
before_filter :setup_number_of_years
before_filter :setup_registration_price
def setup_name_and_domain
# ...
end
protected :setup_name_and_domain
def setup_number_of_years
# ...
end
protected :setup_number_of_years
def setup_registration_price
# ...
end
protected :setup_registration_price
# TODO: refactor this and update - duplicate code
def create
# ...
end
private
def process_registration(payment_type)
# ...
end
def register
if @domain.register(@number_of_years, params[:extended_attribute])
if @domain.registering?
DomainNotifier.domain_registering(@domain).deliver
elsif !@domain.registered?
DomainNotifier.failed_domain_registration(@domain).deliver
end
# TODO: if the domain is registering, then what happens here?
if params[:privacy] == '1'
if @wpps_supported
@domain.enable_wpps
DomainNotifier.whois_privacy_protection_purchased(@domain).deliver
else
DomainNotifier.whois_privacy_protection_unavailable(@domain).deliver
end
end
# TODO: if the domain is registering and fails then this will result
# in an offering being used with no domain delivered.
@offer.use! if @offer
respond_to do |format|
format.html { redirect_to domains_path }
format.json { render :json => @domain.to_json(:except => 'powerdns_domain_id'), :location => domain_path(@domain), :status => :created }
format.xml { render :xml => @domain.to_xml(:except => 'powerdns_domain_id'), :location => domain_path(@domain), :status => :created }
end
else
@domain.registration_failed!
respond_to do |format|
format.html { render :action => 'new' }
format.json { render :json => {:name => @domain.name, :errors => @domain.errors.full_messages }, :status => 422 }
format.xml { render :xml => {:name => @domain.name, :errors => @domain.errors.full_messages }, :status => 422 }
end
end
end
# TODO: refactor!
def setup_contact
if @domain.registrant == nil
if params[:contact]
base_attributes = {:user_id => current_user.id, :state_province_choice => 'S'}
base_attributes[:email_address] = current_user.email if params[:contact][:email_address].blank?
contact_attributes = base_attributes.merge(params[:contact])
@contact = @domain.build_registrant(contact_attributes)
unless @contact.save
respond_to do |format|
format.html { render :action => 'new' }
format.json { render :json => {:name => @domain.name, :errors => @domain.errors.full_messages + @contact.errors.full_messages }, :status => 422 }
format.xml { render :xml => {:name => @domain.name, :errors => @domain.errors.full_messages + @contact.errors.full_messages }, :status => 422 }
end
return false
end
elsif current_user.default_contact
@domain.registrant = current_user.default_contact
else
@domain.errors.add(:base, "A registrant is required")
respond_to do |format|
format.html { render :action => 'new' }
format.json { render :json => {:name => @domain.name, :errors => @domain.errors.full_messages }, :status => 422 }
format.xml { render :xml => {:name => @domain.name, :errors => @domain.errors.full_messages }, :status => 422 }
end
return false
end
end
logger.info "Contact setup complete"
return true
end
# ...
end
Pretty messy, isn't it?
As preparatory work for the new API, we extracted the code from models and controllers into objects we called commands. We picked this name as we originally thought we would implement the command design pattern, but it never happened and the name never changed. Today, we would probably call them interactors or operations.
These objects implement the common logic for each use case in our system. We have one object for the login use case, one for the forgotten password, one for the signup, etc.
This abstraction alone simplifies a lot the maintenance.
All the code is predictably organized.
class DomainRegistrationsController < ApplicationController
# ...
def create
@result = DomainRegisterCommand.execute(command_context,
this_account,
domain_params,
contact_params)
@domain = @result.domain
if @result.successful? && @result.processing?
create_registering
elsif @result.successful?
create_successful
else
create_failed
end
end
end
class DomainRegisterCommand
include Command::Command
def execute(account, domain_name, domains_params, contact_params)
# ...
end
private
def register(domain, extended_attributes, registrar_premium_price)
# ...
end
end
We didn't removed the complexity of this workflow, we just split it in smaller manageable objects. Now testing a feature is easier as we can focus only on the behavior of a single command object at the time.
In 2014 we also started to experiment our own version of service objects, that we formalized in 2016.
If you want to learn more about the evolution of our Rails architecture you can check out the slides and video of the talk Developing and maintaining a platform with Rails and Hanami that Simone presented at RailsConf 2016.
Tight Coupling
In Rails, when an action uses respond_to, the context for the web UI and the JSON API is the same: they share the same set of instance variables used for rendering.
That means you can't easily change one of these instance variables for a component (eg the UI), without affecting the behavior of the other component (eg API). Consider that a public JSON API is versioned, so you can't change it without breaking the entire ecosystem. We soon reached a state of code rigidity, where it was hard or impossible to modify the code for certain actions.
class RecordsController < ApplicationController
# ...
def create
@result = RecordCreateCommand.execute(command_context, @domain.zone, param_record_type, record_params)
@record = @result.data
respond_to do |format|
if @result.exists?
format.html { redirect_to domain_records_url(@domain), notice: @result.warning }
format.json { render json: { message: @result.warning }, status: 400 }
elsif @result.successful?
format.html { redirect_to domain_records_url(@domain) }
format.json { render json: @record, status: 201 }
else
prepare_complex_records(@record, param_record_type)
format.html { render_new }
format.json { render json: { message: @result.error, errors: @record.errors }, status: 400 }
end
end
end
end
The most obvious solution to reduce complexity was to remove responsibilities from these actions. That's why we decided to implement the new API as a standalone Hanami application mounted inside Rails. A change in the web UI (Rails) isn't reflected in the API (Hanami) and viceversa.
When we'll sunset, API v1 we can simplify that action to:
class RecordsController < ApplicationController
# ...
def create
@result = RecordCreateCommand.execute(command_context, @domain.zone, param_record_type, record_params)
@record = @result.data
respond_to do |format|
if @result.exists?
redirect_to domain_records_url(@domain), notice: @result.warning
elsif @result.successful?
render json: @record, status: 201
else
prepare_complex_records(@record, param_record_type)
render_new
end
end
end
end
With the command objects already in place, it became much simpler to implement the new endpoints of the API. Each new endpoint is implemented with an action. It has the role of accepting the input and invoking the command related to the current use case.
Here's an API v2 action:
module Api::V2
module Controllers::ZonesRecords
class Create
include Hanami::Action
def call(params)
@zone = DomainFinder.find!(
params[:zone_id], authentication_context).zone
@result = RecordCreateCommand.execute(command_context, @zone,
params[:type], ZoneRecordParams.new(params, [:regions]))
@record = @result.data
if @result.successful? && !@result.exists
render Serializers::RecordSerializer.new(@record), 201
elsif @result.exists
error(400, I18n.t("api.zone_records.already_exists"))
else
render Serializers::ErrorSerializer.new(@result.error, @record), 400
end
end
end
end
end
It's still complex, but it has a high cohesion. The purpose is to not eliminate the complexity of the model domain (cause you can't), but to make it maintanable.
Implicit Serializations
The way ActiveRecord serializes a model into JSON is straightforward: it dumps all the attributes. Again, this is easy, but as before, it comes with a cost: you can't change a database column without breaking the JSON API backwards compatibility.
To solve this problem we had to introduce a new layer of serializers. They have the role of translating ActiveRecord attributes into a stable set of key/value pairs for JSON.
In DNSimple we have the concept of contact, which is a person or a company who registers a domain. For a contact serialization we return the email information in the payload, but the corresponding database column is email_address. The ContactSerializer helps to resolve this mismatch with a stable public name: the email key.
module Api::V2
module Serializers
class ContactSerializer < ElementSerializer
attributes :id, :account_id,
:label, :first_name, :last_name,
:job_title, :organization_name, :email,
:phone, :fax,
:address1, :address2, :city,
:state_province, :postal_code, :country,
:created_at, :updated_at
def email
object.email_address
end
end
end
end
Even a value can be subject to accidental changes. At one point, Ruby changed the way that JSON.generate serializes timestamps. Without an explicit control on the timestamps format, a change in Ruby (or another library), can suddenly change the output of the API.
require 'json'
# Ruby 1.9
JSON.generate(time: Time.now)
# => "{\"time\":\"Tue Jan 17 10:25:37 +0100 2017\"}"
# Ruby 2.0
JSON.generate(time: Time.now)
# => "{\"time\":\"2017-01-17 10:25:37 +0100\"}"
Conclusion
While the API v1 implementation had a short "time to market", it hid unfortunate surprises down the road. In order to make it easier to evolve your systems, it is good to remember to create team guidelines for code structure, avoid tight coupling with the current framework, and take control of your code via explicitness of intents.
This post only scratches the surface of the code changes behind API v2, and the reasons why we went through this journey. You can read more about the changes in API v2 in the API v2 announcement post. I also encourage you to check out the talk The Great 3 Year API Redesign that Anthony presented at CodeMash 2017, where he went through the story of the DNSimple API v2, explaining some of the decisions we took over the last 3 years that shaped the development of the API as well as challenges we faced.
Luca Guidi
Former astronaut, soccer player, superhero. All at the age of 10. For some reason now I write code.
We think domain management should be easy.
That's why we continue building DNSimple.
