Engineering reference: the incident postmortem collector architecture

Key takeaways

Single region, single account. Every resource is regional; nothing is global except the IAM roles.
3 Lambda functions, each with its own execution role. No shared role, no wildcards on resources.
2 DynamoDB tables, each keyed so the concurrency story is a condition expression rather than a lock.
One Bedrock model, called once, with a JSON schema it must fill or leave null.
Nothing always-on: no instance, no container, no provisioned capacity.

The system, by service name

Fig 1. The same shape as Part 1 with the service names filled in. Nothing here is new; it is the same three groups, named.

Compute
Database
App integration
Networking
Management
People

Region and account

Region: us-east-1. Chosen because SES inbound receipt rules exist in only a subset of regions and this one has the widest Bedrock model availability. If your data has to stay elsewhere, check both constraints before moving: inbound SES is the binding one.
Account: one. This is a small system, and a separate account per environment costs more in wiring than it saves. A dev and a prod stack in the same account, with distinct resource prefixes, is the right size here.
Everything is regional. The only global resources are the IAM roles and policies. There is no CloudFront, no global table and no cross-region replication, because nothing here has a latency or durability requirement that would justify them.

Lambda inventory

Function	Trigger	Does	Timeout / memory
`ip-capture`	Function URL, from the channel and event sources	Appends timestamped entries; posts matching past incidents when one is declared	10s / 512 MB
`ip-publish`	API, when the write-up is finalised	Stores the edited timeline and conditions; expires the raw channel; creates the actions	60s / 1024 MB
`ip-chase`	EventBridge, weekly	Nudges owners at 30 and 90 days; escalates once; reports the open-and-overdue count	60s / 512 MB

Splitting this into separate functions is not about modularity. It is that only one of them needs Bedrock permissions and only one is reachable from the public internet, and neither of those is true if it is one handler behind a router.

IAM, scoped

Role	Allowed	On
`ip-capture-role`	`dynamodb:PutItem`, `dynamodb:Query`	Incidents; read for matching
`ip-publish-role`	`dynamodb:UpdateItem`, `dynamodb:PutItem`	Both tables
`ip-chase-role`	`dynamodb:Query`, `dynamodb:UpdateItem`, `ses:SendEmail`	Actions; one verified identity

No role has a Resource: “*” on anything that writes, and every GetSecretValue grant names a single secret arn. That is why there is more than one secret rather than one JSON blob with everything in it.

DynamoDB schemas

Table: incidents

PK   incident_id       S
SK   entry_at          S   one item per timeline entry
     kind              S   message | alert | deploy | config | belief
     text              S
     who               S   in the timeline only, never in conditions
     kept              BOOL survives the edit; the rest expires
     ttl               N   set on unkept entries at publish time

Metadata item (SK = ’#meta’):
     systems           L   picklist at declaration; drives matching
     alert_signature   S   drives matching
     impact            S   who, how, how long
     conditions        L   from a controlled vocabulary
     went_well         L
     near_miss         BOOL a short form, not a full write-up

Table: actions

PK   incident_id       S
SK   action_id         S
     text              S
     condition         S   which contributing condition it addresses
     owner             S   a person who was in the room and said yes
     due               S   a date they believed
     state             S   open | done | declined | escalated
     declined_reason   S   required when declined; shown on recurrence
     closed_at         S

There is no path that leaves an action open indefinitely: the chase
function moves everything to done, declined or escalated.

Inbound and outbound

The channel is the timeline. Nobody is asked to log anything during an incident; messages are captured as they are sent.
Automatic events are interleaved with the conversation, so a deploy twelve minutes before an alert is visible without anybody looking for it.
Systems are a picklist at declaration, one field, because it is the basis for matching and free text does not match.
Conditions come from a controlled vocabulary, extensible but curated. It is the only way the recurrence chart in Part 5 exists.

The model call

There is no model in this system, deliberately. This is the one place in this series where the absence is a value judgement rather than a cost one.
The tempting use is drafting the write-up from the channel. It would produce a fluent, plausible document and remove the thinking that is the entire purpose.
A second tempting use is identifying causes. Naming a cause is the analysis, and outsourcing it produces a confident answer nobody argued with.
The narrowest defensible use is suggesting which past incidents might be related, alongside the structural matching, with the match reason shown.
The cost page assumes none, which is why messaging is the whole variable.

Things worth knowing before you build it

Capture the timeline during the incident. It is the one part that cannot be reconstructed, and hindsight deletes exactly the parts that matter.
Do not put a root cause field on the template. The field creates the answer, and the answer it creates is the last thing that changed.
Close every action, including as declined with a reason. An open list that grows teaches everybody that postmortems produce nothing.
Match past incidents on systems and alerts rather than on text. Write-up wording varies too much for search to work.
Show the declined actions when an incident recurs. It is uncomfortable and it is what makes declining an honest decision rather than a quiet one.

That is the whole system. Seven posts, one diagram at a time, and nothing in it that needs a server.

All posts