allanninal.dev/build

Part 3 of 7 · Lead enricher series ~7 min read

How a lead gets enriched

A clean lead is still a thin one. You have a name, a phone, an email, and a country — but not what the person actually does, how big their company is, or what industry they’re in. A lot of that is hiding in plain sight in the email domain. This post is about the enrichment stage: taking the company from the domain, looking it up against public firmographic data, and attaching size, industry, and location to the record — carefully, cached, and always with a note of where each field came from so nothing is silently invented.

Key takeaways

  • Enrichment turns a clean email domain into a company profile: name, size band, industry, and location, attached to the lead.
  • The work is a single firmographic API lookup keyed on the domain, with the API key held in Secrets Manager and never in code.
  • Results are cached per domain in DynamoDB, so the same company is never paid for twice in a short window — the cache is what keeps the bill low.
  • Free and personal email domains (gmail, outlook) are recognised and skipped, not looked up — there’s no company behind them.
  • Every enriched field records its source and the lookup date; a field the provider can’t supply is left blank, never invented.

From a domain to a company

After normalisation you have a clean lead but a thin one: a real name, a formatted phone, a validated email, a country. What you don’t have is any sense of who this is in business terms — what company they’re from, how big it is, what it does. For a B2B enquiry, that context is most of what makes a lead useful, and a surprising amount of it is recoverable from one field you already cleaned: the email domain.

The enrich stage takes the domain — acme-widgets.co.uk — and looks it up against a public firmographic data provider. The provider returns a structured company record: the legal name, a headcount band, an industry classification, and a headquarters location. The enricher attaches those to the lead, each one tagged with where it came from and when it was fetched. That’s the whole job: complete the record from public data, carefully and traceably.

The lookup, and the key behind it

The lookup itself is a single authenticated API call. The provider’s key never appears in the function code or the environment — it lives in Secrets Manager, and the enrich function fetches it at runtime with a tightly scoped IAM permission to read that one secret. Rotating the key, or swapping providers entirely, is a change in one place with no redeploy of the pipeline.

Two things happen before any call is made, and both exist to save money:

  • Personal domains are skipped. If the domain is a known free or personal-email host — gmail.com, outlook.com, yahoo, icloud, and a maintained list of others — there is no company to look up, so the enricher doesn’t try. The lead keeps its clean fields and is simply marked “personal email, no firmographics,” and the API call is never spent.
  • The cache is checked. Companies don’t change overnight, and the same domains recur constantly — several people from the same firm enquire over a month. So every successful lookup is cached in DynamoDB, keyed by domain, with a time-to-live. A second lead from acme-widgets.co.uk within the cache window reuses the stored profile and costs nothing. At small-business volume this cache absorbs a large share of the traffic, which is exactly why the enrichment line on the bill is dollars and not tens of dollars.
The enrich stage: skip personal domains, check the cache, look up once, attach the profile A clean lead with the domain acme-widgets.co.uk enters a dotted AWS account container holding the ler-enrich Lambda. First a decision diamond: is this a personal or free email domain? A Yes arrow marks the lead personal email, no firmographics, and exits — no lookup. A No arrow goes to a second decision: is the domain already in the cache? A Hit arrow reads the stored profile from a DynamoDB enrichment cache box and goes straight to the output. A Miss arrow goes to a Firmographic API call box, which reads its key from a Secrets Manager box and calls the external provider; the result is written back into the cache and on to the output. The output box, Enriched record, lists company Acme Widgets Ltd, size 50 to 200, industry industrial supplies, location Birmingham, each tagged with source and date. A note reads: a field the provider can’t supply is left blank, never invented; the cache is what keeps the bill low. Clean lead acme-widgets.co.uk ler-enrich — AWS personal domain? yes no firmographics no in cache? Enrichment cache DynamoDB · per domain hit miss Firmographic API call writes to cache Secrets Manager the API key Enriched record Acme Widgets Ltd 50–200 · supplies Birmingham · sourced A field the provider can’t supply is left blank, never invented. The cache is what keeps the bill low.
Fig 3. Enrichment skips personal domains, checks the per-domain cache, and only pays for a fresh lookup on a cache miss. The key lives in Secrets Manager; every field carries its source.

Attaching the profile, honestly

When a lookup returns, the enricher doesn’t just dump the provider’s fields onto the lead. It maps them to a small, fixed schema — company name, size band, industry, location — and tags each value with two things: the source (“firmographic provider”) and the date it was fetched. That provenance matters later. When the dedupe stage compares this lead’s company against an existing record, it can tell a human-entered company name from an auto-enriched one. And when the push stage decides whether to fill a CRM field, it knows it’s filling a gap with sourced data, not overwriting something a rep typed.

A field the provider can’t supply is left blank. If there’s no headcount for a tiny one-person consultancy, the size band stays empty rather than being filled with a default or a guess. The same restraint runs through the whole system: an empty field is honest, a fabricated one is a landmine that surfaces months later as a wrong number in a report.

When the lookup fails

External APIs are not always there. The provider might rate-limit, time out, or be down entirely. The enricher treats an enrichment failure as a soft failure, never a hard one: the lead keeps its clean normalised fields, is marked “enrichment pending,” and continues through dedupe and the junk filter so a real enquiry isn’t held hostage to a third party’s uptime. The pending lookup is queued for a retry, and an EventBridge-scheduled backfill (covered in the engineering reference) sweeps up anything still un-enriched once the provider recovers. A lead never gets stuck, and it never gets dropped just because the enrichment step couldn’t complete.

Why this shape

  • One key, one place. The provider key lives in Secrets Manager with a scoped read permission; rotating or swapping it touches nothing else.
  • Cache before you call. A per-domain DynamoDB cache absorbs repeat companies, turning a per-lead cost into a per-new-company one.
  • Don’t pay for nothing. Personal and free domains are skipped outright — there’s no company there to find.
  • Provenance on every field. Each enriched value carries its source and date, so downstream stages can tell sourced data from human input.
  • Enrichment failures are soft. A provider outage marks the lead pending and lets it continue; a scheduled backfill finishes the job later.
All posts