Engineering reference: the duplicate contact merger architecture

Key takeaways

Single region, single account. Every resource is regional; nothing is global except the IAM roles.
3 Lambda functions, each with its own execution role. No shared role, no wildcards on resources.
3 DynamoDB tables, each keyed so the concurrency story is a condition expression rather than a lock.
One Bedrock model, called once, with a JSON schema it must fill or leave null.
Nothing always-on: no instance, no container, no provisioned capacity.

The system, by service name

Fig 1. The same shape as Part 1 with the service names filled in. Nothing here is new; it is the same three groups, named.

Compute
Storage
Database
App integration
People

Region and account

Region: us-east-1. Chosen because SES inbound receipt rules exist in only a subset of regions and this one has the widest Bedrock model availability. If your data has to stay elsewhere, check both constraints before moving: inbound SES is the binding one.
Account: one. This is a small system, and a separate account per environment costs more in wiring than it saves. A dev and a prod stack in the same account, with distinct resource prefixes, is the right size here.
Everything is regional. The only global resources are the IAM roles and policies. There is no CloudFront, no global table and no cross-region replication, because nothing here has a latency or durability requirement that would justify them.

Lambda inventory

Function	Trigger	Does	Timeout / memory
`dm-block`	EventBridge weekly	Computes four keys per record, forms blocks, caps oversized ones	300s / 3008 MB
`dm-score`	SQS block queue	Rarity-weighted evidence for and against; writes candidate pairs	60s / 1024 MB
`dm-merge`	Function URL	Builds the proposal; on confirmation writes the undo bundle then merges	60s / 1024 MB

Splitting this into separate functions is not about modularity. It is that only one of them needs Bedrock permissions and only one is reachable from the public internet, and neither of those is true if it is one handler behind a router.

IAM, scoped

Role	Allowed	On
`dm-block-role`	`secretsmanager:GetSecretValue`, `dynamodb:PutItem`	The CRM read credential; the pairs table
`dm-score-role`	`dynamodb:Query`/`UpdateItem`	Pairs and decisions
`dm-merge-role`	`s3:PutObject`, `secretsmanager:GetSecretValue`, `dynamodb:PutItem`	The undo prefix; the CRM write credential; the merges table

No role has a Resource: “*” on anything that writes, and every GetSecretValue grant names a single secret arn. That is why there is more than one secret rather than one JSON blob with everything in it.

DynamoDB schemas

Table: pairs

PK   pair_key          S   min(id_a,id_b)#max(id_a,id_b)
     score             N   4.8
     terms             L   [{field, agreed, rarity, weight}]
     against           L   [{reason, strength}]
     band              S   automatic | propose | record_only
     blocked_by        L   which keys produced this pair
     ttl               N   epoch, +30 days

The pair key is order-independent, so A-B and B-A are one row. Getting that
wrong doubles every count and proposes each pair twice.

Table: decisions

PK   pair_key          S   min#max
     decision          S   merged | not_a_duplicate
     decided_by        S   a person, or ’automatic’
     decided_at        S   2026-08-05T11:02:00Z

No TTL, ever. A pair somebody has declared separate must never be proposed
again — that is the single feature that keeps the review worth opening.

Table: merges

PK   merge_id          S   mrg_2026_08_05_c41a
     kept_id           S   the surviving record
     merged_id         S   the record that was absorbed
     bundle_key        S   s3://undo/mrg_....json
     moved             L   [{type, id, from, to}]
     undoable_until    S   2026-11-03
     undone_at         S   or null

`moved` is the complete list of related records repointed, in both
directions, which is what makes an undo a restore rather than a guess.

Inbound and outbound

The CRM read credential and the write credential are separate secrets. Only dm-merge can read the write credential, and only on a confirmed proposal.
Confirmation links are signed, scoped to one pair, single-use, and expire after thirty days — the same window as the pairs table, so a stale link cannot act on a recomputed pair.
The undo bundle is written before the merge, not after. A merge that fails part way through leaves related records split across both, and the bundle is what puts them back.
Merges are serialised per record. Two proposals involving the same record cannot execute concurrently; a conditional write on the record id enforces it.

The model call

There is no model in this system. Blocking is key computation, scoring is weighted evidence, and both need to be identical between runs.
An embedding-based similarity is the obvious alternative and it produces the thing this design specifically avoids: a single opaque number instead of visible evidence.
The proposal has to be explainable in the four seconds somebody spends on it, and “same phone, same rare surname, different first name” is explainable in a way that 0.88 is not.
Rarity weighting gives most of what a learned model would, from a count query over your own data, and it is stable between runs.
The cost page assumes none, which is why compute is the only variable band.

Things worth knowing before you build it

Make the pair key order-independent. A-B and B-A as separate rows doubles every count and proposes each pair twice.
Cap block sizes. One placeholder phone number shared by four hundred records reintroduces the quadratic problem blocking exists to solve.
Never expire the not-a-duplicate decisions. Re-proposing rejected pairs is what makes people stop opening the review.
Store full snapshots, not diffs. An undo applied to a record somebody has since edited must restore, not reconstruct.
Write the undo bundle before the merge runs, so a partial failure is recoverable with the same mechanism.

That is the whole system. Seven posts, one diagram at a time, and nothing in it that needs a server.

All posts