Engineering reference: the search rank reporter architecture

Key takeaways

Single region, single account. Every resource is regional; nothing is global except the IAM roles.
3 Lambda functions, each with its own execution role. No shared role, no wildcards on resources.
2 DynamoDB tables, each keyed so the concurrency story is a condition expression rather than a lock.
One Bedrock model, called once, with a JSON schema it must fill or leave null.
Nothing always-on: no instance, no container, no provisioned capacity.

The system, by service name

Fig 1. The same shape as Part 1 with the service names filled in. Nothing here is new; it is the same three groups, named.

Compute
Storage
Database
App integration
Analytics

Region and account

Region: us-east-1. Chosen because SES inbound receipt rules exist in only a subset of regions and this one has the widest Bedrock model availability. If your data has to stay elsewhere, check both constraints before moving: inbound SES is the binding one.
Account: one. This is a small system, and a separate account per environment costs more in wiring than it saves. A dev and a prod stack in the same account, with distinct resource prefixes, is the right size here.
Everything is regional. The only global resources are the IAM roles and policies. There is no CloudFront, no global table and no cross-region replication, because nothing here has a latency or durability requirement that would justify them.

Lambda inventory

Function	Trigger	Does	Timeout / memory
`sr-fetch`	EventBridge weekly	Paginated fetch by query and by page; writes raw rows to S3	120s / 1024 MB
`sr-group`	S3 ObjectCreated	Applies the ordered rules; writes themed totals	60s / 1024 MB
`sr-report`	EventBridge weekly + monthly	Comparisons, page attribution, the report page	30s / 512 MB

Splitting this into separate functions is not about modularity. It is that only one of them needs Bedrock permissions and only one is reachable from the public internet, and neither of those is true if it is one handler behind a router.

IAM, scoped

Role	Allowed	On
`sr-fetch-role`	`s3:PutObject`, `secretsmanager:GetSecretValue`	The raw prefix; the search API credential only
`sr-group-role`	`s3:GetObject`, `dynamodb:PutItem`, `secretsmanager:GetSecretValue`	The raw prefix; the history table; the Sheets credential
`sr-report-role`	`dynamodb:Query`, `ses:SendEmail`	History, read; one verified identity

No role has a Resource: “*” on anything that writes, and every GetSecretValue grant names a single secret arn. That is why there is more than one secret rather than one JSON blob with everything in it.

DynamoDB schemas

Table: history

PK   theme             S   emergency-callouts
SK   week_ending       S   2026-07-26
     impressions       N   4100
     clicks            N   340
     position          N   7.8    — impression-weighted, never a simple mean
     pages             L   [{page, clicks, share}]
     rules_version     S   v7     — which rule set produced this row

`rules_version` is what makes a regroup honest: a row grouped under v7 and
one grouped under v4 are not directly comparable, and the report says so
if a comparison would cross a version boundary.

Table: themes

PK   theme             S   emergency-callouts
     label             S   Emergency callouts
     order             N   1      — the report order, fixed
     patterns          L   ordered match rules
     volume_floor      N   500    — impressions below which nothing is reported
     kind              S   service | brand | competitor | unmatched

Brand and competitor themes are matched BEFORE any service theme, which is
enforced by `kind` rather than by rule ordering, so a rule edit cannot
accidentally let a brand query into a service theme.

Inbound and outbound

Read-only against the search property. The credential has no write scope and there is nothing in the API to write anyway.
Nothing is scraped. No simulated searches, no proxy fleet, no positions from a datacentre — all of which produce a worse number and most of which breach somebody’s terms.
Raw rows land in S3 before anything is grouped, and the S3 event is what fires grouping. That means a regroup of history is the same code path as a fresh fetch.
The fetch always asks for a completed week ending at least three days ago, because recent data is still moving.

The model call

There is no model in this system, and that is the design. Automatic query clustering is the obvious use and it is the wrong one.
Clusters shift between runs. Same site, slightly different queries, slightly different groups — and every trend built on them is quietly meaningless.
Rules are stable by construction. The same rule set produces the same groups forever, which is what makes an eight-week comparison mean anything.
If you want help writing rules, use a model interactively, once, on the unmatched bucket. Then write the rules down and let the code apply them.
The cost page assumes none, which is why there is no read band.

Things worth knowing before you build it

Store raw rows, not grouped totals. Every theme rule refinement is worthless if it cannot be applied to history.
Weight position by impressions. A simple mean across queries reports a theme that ranks second as ranking thirty-ninth.
Match brand before anything else. Brand queries containing a service word will otherwise inflate that service and mask a real decline.
Check for row-limit truncation. A response exactly at the limit is almost certainly cut off, and the long tail is where the interesting queries are.
Never claim a cause. State that a page changed on a date and let a person weigh it; a confident wrong cause costs a fortnight of reverting something that was fine.

That is the whole system. Seven posts, one diagram at a time, and nothing in it that needs a server.

All posts