Engineering reference: the cost anomaly alerter architecture

Key takeaways

Single region, single account. Every resource is regional; nothing is global except the IAM roles.
3 Lambda functions, each with its own execution role. No shared role, no wildcards on resources.
2 DynamoDB tables, each keyed so the concurrency story is a condition expression rather than a lock.
One Bedrock model, called once, with a JSON schema it must fill or leave null.
Nothing always-on: no instance, no container, no provisioned capacity.

The system, by service name

Fig 1. The same shape as Part 1 with the service names filled in. Nothing here is new; it is the same three groups, named.

Compute
Database
App integration
Management

Region and account

Region: us-east-1. Chosen because SES inbound receipt rules exist in only a subset of regions and this one has the widest Bedrock model availability. If your data has to stay elsewhere, check both constraints before moving: inbound SES is the binding one.
Account: one. This is a small system, and a separate account per environment costs more in wiring than it saves. A dev and a prod stack in the same account, with distinct resource prefixes, is the right size here.
Everything is regional. The only global resources are the IAM roles and policies. There is no CloudFront, no global table and no cross-region replication, because nothing here has a latency or durability requirement that would justify them.

Lambda inventory

Function	Trigger	Does	Timeout / memory
`ca-fetch`	EventBridge daily 09:00	One grouped Cost Explorer query for the last three days	60s / 512 MB
`ca-compare`	SQS fetched queue	Money floor, step and ramp checks, new services and falls	30s / 512 MB
`ca-attribute`	SQS anomaly queue + EventBridge hourly	Resource level, usage metrics, tags; the fast proxy checks	60s / 512 MB

Splitting this into separate functions is not about modularity. It is that only one of them needs Bedrock permissions and only one is reachable from the public internet, and neither of those is true if it is one handler behind a router.

IAM, scoped

Role	Allowed	On
`ca-fetch-role`	`ce:GetCostAndUsage`, `dynamodb:PutItem`	Cost Explorer, read; the daily table
`ca-compare-role`	`dynamodb:Query`, `sqs:SendMessage`	The daily table; the anomaly queue
`ca-attribute-role`	`ce:GetCostAndUsageWithResources`, `cloudwatch:GetMetricData`, `tag:GetResources`, `ses:SendEmail`	Read-only across all three; one verified identity

No role has a Resource: “*” on anything that writes, and every GetSecretValue grant names a single secret arn. That is why there is more than one secret rather than one JSON blob with everything in it.

DynamoDB schemas

Table: daily

PK   service           S   AWSLambda
SK   date              S   2026-08-02
     amount            N   32.14
     amount_provisional BOOL false  — true until it has settled 3 days
     day_of_week       N   2
     first_seen        BOOL false   — true on a service’s first day
     ttl               N   epoch, +120 days

120 days is chosen for the ramp check, which compares a 7-day window
against the 21 before it and wants a quarter of context around that.

Table: resources

PK   service_date      S   AWSLambda#2026-08-02
SK   resource          S   arn:...:function:image-resize
     amount            N   31.02
     usage_metric      S   Invocations
     usage_value       N   400218
     usage_normal      N   2010
     tags              M   {owner: platform, env: prod}
     untagged          BOOL false

`usage_normal` is what makes the message a bug report. Without it the
sentence is ’this function cost £31’, which is an observation.

Inbound and outbound

One grouped Cost Explorer query per run, not one per service. The API is charged per request and a per-service loop multiplies the bill by thirty for identical data.
Resource-level costs come from a separate, more expensive call made only for the services that actually moved — typically zero or one per day.
Usage metrics are read hourly for three services only: Lambda invocations, S3 requests and NAT gateway bytes. Those three cover almost every runaway that can matter inside a day.
No write permission anywhere. The execution roles have no lambda:Delete, no ec2:Stop, no s3:Delete. That is enforced by the policy rather than by the code not calling them.

The model call

There is no model in this system. Comparisons, projections and attribution are arithmetic and lookups.
The tempting use is generating an explanation of a spike, which would produce a confident guess about a cause the data does not contain.
The message is a template with the resource, the usage figures and the projection substituted, which also means it reads the same every time and gets scanned quickly.
Stating two facts side by side — the invocation count and that the function writes where it is triggered from — lets a person recognise recursion without the system claiming it.
The cost page assumes none, which is why the API requests are the only unusual line.

Things worth knowing before you build it

Group the Cost Explorer query. It is charged per request, and a per-service loop is thirty times the cost for the same answer.
Re-fetch the last three days. Yesterday’s figure is provisional and a restatement will otherwise leave a wrong number in the baseline forever.
Add the ramp check. A step change is easy and the expensive surprises are almost always something growing eight per cent a day that never trips a day-on-day comparison.
Put a money floor in front of every relative test. A hundredfold increase on four pence is four pence.
Do not grant write permissions, and say why in the policy. The next person to work on this will want to add remediation, and the reasoning should be somewhere they will find it.

That is the whole system. Seven posts, one diagram at a time, and nothing in it that needs a server.

All posts