Duplicate contact merger
A small system that finds probable duplicate contacts, scores each pair against evidence rather than a similarity score, proposes a merge showing precisely which field wins, and keeps enough to undo it. It merges nothing automatically above the certain band. Seven posts on the same system — one diagram at a time — with a cost breakdown and an engineering reference at the end.
- 01
A duplicate contact merger on AWS for a few dollars a month
The whole system on one page — a blocker, a scorer and a proposer — with the asymmetry that makes automatic merging almost always the wrong choice.
- 02
How candidate pairs get found
Why every-pair comparison is impossible at any real scale, the four blocking keys worth using, why multiple keys beat one good one, and the duplicates blocking will still miss.
- 03
How a pair gets scored
Why evidence beats similarity, how rarity weights agreement, the three signals that argue actively against a merge, and the narrow band that merges automatically.
- 04
How a merge gets proposed
What a proposal shows field by field, the rules that decide which value wins, why showing losses matters more than showing the result, and the related records a merge has to move.
- 05
How a merge gets undone
What has to be stored to make a merge reversible, how long to keep it, the changes that genuinely cannot be undone, and the numbers worth watching.
- 06
What the duplicate contact merger costs
About $2 a month. One Bedrock read per thousand contacts is the only line that grows; the queue, the table, the mail and the storage are rounding errors. Plus the three ways the bill could surprise you.
- 07
Engineering reference: the duplicate contact merger architecture
Same system, drawn purely for engineers. Service names, region, Lambda inventory, IAM scopes, the schemas and the exact model id.
Frequently asked questions
- What is a duplicate contact merger?
- A small serverless system that finds records likely to be the same person or company, scores each candidate pair, proposes a merge showing what would change, and executes only what a person confirms — keeping enough history to reverse it.
- Does it merge automatically?
- Only in a narrow certain band: an exact match on a strong identifier with no conflicting data. Everything else is proposed to a person, because a wrong merge is much harder to explain to a customer than a duplicate.
- Why is a wrong merge worse than a duplicate?
- A duplicate means somebody gets two emails. A wrong merge means one customer’s order history is attached to another customer’s record, which is a data protection problem as well as an embarrassing one.
- Can a merge be undone?
- Yes, for a retention window you set. The system stores both original records in full, so reversing is restoring rather than reconstructing. After the window it becomes irreversible and the report says which merges have passed it.
- What does it cost to run?
- A few dollars a month for a database of tens of thousands of contacts. See part six.