Learning to Fail Fast at DNSimple
Amelia Aronsohn
on
As many readers of the blog here at DNSimple know, we are a fans of making rapid changes and iterating. Our idea is to move quickly and add lots of small changes that add up to big improvements for you, our customers. On the operations side of the fence sometimes we can lose sight of that during a large technical challenge.
Recently I took the reins of a large project that's been in process for longer than we would like. It's not that we hadn't managed to ship on it, it's just that we kept running into edges and iterating on it past a point of reasonable conclusion. We had forgotten to fail fast and fail often.
Failing Fast
One of the strategies of agile styles of developments, and of rapid iterative systems, is the idea that you shouldn't just release rapidly but "fail fast, fail often".
It is an often criticized concept meaning you release, measure, and consider problems along the way to learning important lessons. If something is not working, you make the call to pivot to a different course before you become too invested or layer too much over something that will not work.
The idea is to learn as fast as you deploy.
Sunk Cost Fallacy
For me, one of the biggest unspoken lessons of fail fast, fail often is avoiding the sunk cost fallacy; the sunk cost fallacy is a form of loss aversion that people often have because you have invested time, effort, and money in a process. It is easy to feel an obligation to see your project though, because you have personally invested time and effort in the process. The simple fact is that resources are spent and cannot be recovered, no matter if the initiative will pay off or not.
Logically the amount of resources invested in something shouldn't be a deciding factor in how valuable or useful it is, but we become emotionally invested in its success—regardless of it's potential for success. I've personally been drawn into the sunk cost fallacy and have become adverse to it due to the consequences of remaining committed to projects or systems that no longer have a positive gain.
Learning our lessons
After making hard choices and sitting down with the whole team, we discussed what we wanted to accomplish. We had become bogged down in what we were trying to accomplish. Regardless of the fact that the system had "failed" we needed to focus on what we had learned from the implementation.
There were many lessons learned and technical methods we can reuse elsewhere. Once we were able to refocus and generalize the scope with the lessons learned, we were able to attack the project with a new light and in more focused measures. I worked to implement a second system and started targeting problem points early on. This time around, since I was working in smaller units, it was easy to identify that I was hitting a show stopper and quickly pivot to a different solution that deployed quickly and has offered success.
Our project isn't done, but we've been able to make progress rapidly, and quickly test the units of this project to measure success and growth.
Amelia Aronsohn
Kaizen junkie, list enthusiast, automation obsessor, unrepentant otaku, constantly impressed by how amazing technology is.
We think domain management should be easy.
That's why we continue building DNSimple.
