Skip to content
allanninal.dev/build
Collaborate

Part 3 of 7 · Domain renewal watcher series ~5 min read

How forgotten domains get discovered

The domains that lapse are almost never the main one. They are the campaign domain from 2021 that still has an MX record, the one an agency bought, the one a former employee expensed. This post is about finding them before a renewal date does.

Key takeaways

  • Three sources: certificate transparency, your own DNS, and card charges.
  • Certificate logs are public and find any domain that has ever had a public certificate.
  • Your own zones find domains you are pointing at things without owning the record of.
  • Card charges find the registrar relationships nobody documented.
  • A discovered domain is a question, never an automatic addition.

Three sources

Three discovery sources converging on candidate domainsThree boxes stacked on the left. Certificate logs, which are public and searchable, labelled public. Your own DNS zones and records, labelled internal. And Card charges showing registrar names, labelled billing. All three converge on Candidate domains not in the register. Below that, connected by a downward arrow, is Ask a person: is this ours or not? A note says none of these adds a domain automatically and a candidate is a question.Certificate logspublic, searchablepublicYour own DNSzones and recordsinternalCard chargesregistrar namesbillingCandidate domainsnot in the registerAsk a personours, or not?None of these adds a domain automatically. A candidate is a question.
Fig 1. The three discovery sources. Each finds a different kind of forgotten domain, and between them they cover most of how domains go missing from a register.
  • Networking
  • Security & identity
  • Analytics
  • People

Certificate transparency

Every publicly trusted certificate issued since about 2018 is logged in a public, append-only, searchable set of logs. That means any hostname the business has ever put a certificate on is discoverable, by anybody, including you.

Searching those logs for your organisation name and for known domains turns up subdomains and sibling domains routinely. It is particularly good at finding the agency-registered ones, because an agency that set up a site also set up a certificate, and the certificate is public whether or not the domain is on your list.

It is worth being clear-eyed about the other side of that: the same search works for anybody, which is a reason to know what is in there rather than a reason not to look.

Your own DNS

The second source is inside your own infrastructure. A hosted zone for a domain you do not have in the register is a domain you are actively serving and not tracking. So is a CNAME in a zone you do control pointing at a hostname on a domain you do not.

This one finds a specific and dangerous category: domains that other things depend on. A redirect that has been in place for four years, a mail routing record, an SPF include. Those lapse quietly and take something with them.

Card charges

If the subscription audit bot from Day 80 is running, it already knows about every recurring charge, and registrar charges are among the easiest to recognise. A charge to a registrar for an amount that looks like a domain renewal, with no corresponding domain in the register, is a strong candidate.

It is also the only source that finds domains with no public presence at all — one bought defensively, one held for a project that never launched — which the other two sources cannot see by construction.

What happens to a candidate

How a discovered candidate domain is confirmed or dismissedA vertical chain of five steps entered by a box labelled A candidate domain, found but not registered. Step one looks it up at the registry over RDAP before asking anybody, collecting the registrar and expiry. Step two asks whether anything depends on it, checking for MX records, redirects and certificates; if so it exits to Flag as load-bearing and asks urgently. Step three asks a person, showing what it points at; a negative exits to Not ours, which is recorded so it is never raised again. Step four adds it to the register with an internal owner. Step five is Now it is watched, like everything else. A note says not ours is recorded so the same candidate is not raised every week forever.AWS ACCOUNTA candidate domainfound, not registeredLook it upregistry, before askingRDAPregistrar, expiryDoes anything depend on it?MX, redirect, certFlag as load-bearingask urgentlyyesAsk a personwith what it points atNot oursrecord, never ask againnoAdd to the registerwith an internal ownerNow it is watchedlike everything else'Not ours' is recorded, so the same candidate is not raised every week forever.
Fig 2. What happens to a discovered domain. Looking it up before asking means the question carries the expiry and the registrar, which is usually enough for somebody to recognise it immediately.
  • App integration
  • Machine learning
  • Networking
  • Security & identity
  • Management
  • Analytics
  • People
  • Outside AWS

Why ask rather than add

Because discovery produces false positives with certainty. A certificate log search on your organisation name will find domains belonging to businesses with similar names, a customer’s subdomain that you host, and a domain a partner set up that genuinely is not yours to renew.

Adding those automatically produces a register with things in it that nobody can act on, and a register that people learn to ignore. So a candidate is a question with the evidence attached — “found in a certificate issued for shop.example.net, registered at Registrar X, expires March, MX points at your mail provider” — which is usually enough for somebody to answer in five seconds.

Next: the status codes, which frequently matter more than the date.

All posts