Addressing Empty Non-Terminal Compliance

At DNSimple, our authoritative name servers are powered by a custom-built DNS server called erldns, written in Erlang and released open source. It's the result of years of investment in operating reliable, efficient DNS infrastructure tailored to our needs.

For a long time, however, erldns has been failing to correctly implement a subtle part of DNS behavior: support for Empty Non-Terminals (ENTs). When we first built the name server, we overlooked this detail.

The non-compliance only became evident once we began running other authoritative DNS implementations in parallel with erldns. For example, we used PowerDNS for features such as secondary DNS via AXFR, and for integrating third-party providers through our DNS Control Plane.

We've now addressed this longstanding defect. With the release of erldns version 9, ENT compliance is correctly implemented. Version 9 has been running in production across all DNSimple authoritative servers since November. After a deprecation period, we enabled the new ENT behavior at the beginning of this month.

What are Empty Non-Terminals?

This section dives a bit deeper into DNS internals. If you're passionate about DNS behavior and standards, this is for you.

In DNS, an Empty Non-Terminal (ENT) is an intermediate domain name in a DNS hierarchy that exists only because it is a path to a deeper name, but has no records of its own.

Here's a simple example:

$ORIGIN example.com.
example.com.                       IN  A   192.168.1.1
srv1.retails.network.example.com.  IN  A   192.168.1.10
srv2.retails.network.example.com.  IN  A   192.168.1.20

In this zone, both network.example.com and retails.network.example.com are Empty Non-Terminals. They have no records themselves, but exist because of the srv1 and srv2 records beneath them.

ENTs become particularly important when wildcards are involved, as defined in RFC 1034, Section 3.1 and clarified in RFC 4592, Section 2.2.2. Proper handling of ENTs is critical for:

• Correct wildcard record behavior
• DNSSEC compliance (incorrect ENT handling can break NSEC chains or signatures)

For a comprehensive explanation of ENTs, including how they interact with wildcards and troubleshooting guidance, see our support article: What Are Empty Non-Terminals (ENT)?

The problem with our implementation

In previous versions of erldns, ENT responses were not compliant with RFC 4592. The server would incorrectly return wildcard matches even when the queried name existed as an ENT.

For instance:

$ORIGIN example.com.
*.prod.example.com.        IN  A   192.168.0.11
app.us.prod.example.com.   IN  A   192.168.0.41

Querying us.prod.example.com would return:

$ dig @ns1.dnsimple.com us.prod.example.com A +short
192.168.0.11

But, according to the RFC, us.prod.example.com exists as an ENT (because of app.us.prod.example.com), and should return NOERROR with no records, not a wildcard response. This discrepancy created inconsistencies, especially in zones that used both wildcards and DNSSEC.

We noticed the issue while working with other name servers, like PowerDNS, which does handle ENTs correctly. Customers relying on erldns behavior found they needed to modify their zones to maintain the same resolution outcome under PowerDNS. This inconsistency made it clear that our implementation needed to be corrected to meet DNS standards.

While ENT-related scenarios are statistically rare, DNS resolvers must behave predictably and follow published standards. This is particularly important for DNSSEC-compliant zones. For a detailed explanation of how wildcards interact with ENTs and the implications, see Wildcards and Empty Non-Terminals in our support article.

Fixing it in Erldns 9

To address the issue, we restructured how erldns internally stores and queries DNS data.

The fix centered around changes to how we use Erlang Term Storage (ETS). Previously, we stored records in a flat structure, which made it difficult to detect ENT situations.

Since version 8, FQDNs are split into label components and stored like this:

{[<<"com">>, <<"example">>], [<<"a">>, <<"b">>, <<"c">>], A}

This new structure makes it easier to perform pattern matching using Erlang's support for improper lists. For example, to detect whether a.example.com has children (indicating it is an ENT), we now use:

{[<<"com">>, <<"example">>], [<<"a">> | '_'], A}

This lets erldns determine ENT presence without having to traverse the full tree of domain names. The change also reduces unnecessary lookups and improves performance. You can review the changes in this pull request, as well as the relevant source code.

With this new structure in place, erldns now behaves correctly when handling wildcards and DNSSEC in the presence of Empty Non-Terminals.

Since version 9, you can set the configuration erldns.zones.rfc_compliant_ent flag as true to enable the new RFC-compliant resolution behaviour.

Rolling out the change

Fixing the issue in the code was just part of the problem. We then had to roll out the change, and since this would introduce breaking behavior for zones relying on the incorrect wildcard matching, we needed to notify all impacted customers and help them migrate.

This step was initially identified in our shape, then the task was discarded as we overlooked the problem. Ironically, while writing this post, we actually realized we had implemented an incomplete fix, and had to go back to the drawing board and restart the whole notification process. As this blog post's publishing date can testify, it took a while.

And it was not a perfect ride. Along the way, we learned some valuable lessons. Looking back, there were two areas where we could have done better:

1. We didn't consider that ENTs in a zone can appear and disappear in the blink of an eye. There are some curious cases, such as the one related to the provisioning of Let's Encrypt challenges for issuing certificates, that would create short-living ENTs. When we extracted the list of affected customers, we considered a point-in-time scenario (and notified only affected customers) rather than an evolving scenario (and notifying everyone).

2. While the rollout was anticipated, when the time came, we could have done a better job of communicating it to customers. We didn't have an article dedicated to ENTs (it was immediately written the day after), and for some customers it was a bit difficult to follow along the implications.

Preparing an effective deprecation process appears to be a mastery you keep improving each time you exercise it. But I guess this is a topic for another blog post.

Wrapping up

With the release of version 9, erldns now implements Empty Non-Terminal support correctly. This brings us into full compliance with DNS standards, resolves inconsistencies with wildcard behavior, and supports more reliable DNSSEC integration.

We're glad to have closed this long-standing gap in our implementation. We want to extend our sincere thanks to all our customers for their patience and understanding throughout this transition. Your feedback and collaboration during the rollout process were invaluable.

If you have any questions or encounter issues related to Empty Non-Terminals, we've prepared comprehensive guides to help:

• How to Fix Empty Responses from Empty Non-Terminals
• Troubleshooting Empty Non-Terminal Issues

You can also always reach out to our support team, we're here to help.

If you're not using DNSimple yet, give us a try free for 30 days, and experience our best-in-class DNS management for yourself. Have more questions about how we can help meet your domain and DNS management needs? Get in touch, and we'll be happy to help.

Share on Twitter and Facebook

Simone Carletti

Italian software developer, a PADI scuba instructor and a former professional sommelier. I make awesome code and troll Anthony for fun and profit.