Engineering reference: the review request sender architecture

The architecture, for engineers

This is the same system as Part 1 with the friendly labels removed and the real resources named. Everything is in one region, eu-west-2 (London), in one account. There is no API Gateway, no NAT Gateway, and nothing always-on; the only inbound surface is one Lambda Function URL, the timed send is driven by per-request EventBridge schedules, outbound messages go through SES and SNS, and work is buffered on a single SQS queue with a dead-letter queue.

Lambda functions

Five functions, all Python 3.14 on arm64, all with CloudWatch Logs at 7-day retention. Each does one job and hands off; the SQS queue (rrs-work, with rrs-work-dlq as its dead-letter queue after five attempts) decouples the timed send from the slower model and messaging calls, and gives the send path free retries.

• rrs-webhook — the only public surface. Backs the single Lambda Function URL and handles all three inbound event types from the tool. For a completion: verifies the signature, guards one request per job against rrs-requests with a conditional write on the completion id, records the job in rrs-jobs, checks rrs-optout and eligibility, captures the signal from the event and rrs-customers, and creates a one-off EventBridge schedule for the computed due time. For a private-feedback submission: writes it to rrs-audit and enqueues an escalation. For an opt-out: writes rrs-optout. Nothing slow happens here.
• rrs-sender — SQS-triggered when a due schedule fires. Re-checks rrs-optout, once-only state, and quiet hours (rescheduling if the moment has drifted). Makes the first Bedrock call to grade the captured signal, branches on the band, and for a sendable branch makes the second Bedrock call to compose, injects the review or feedback link, validates the draft, sends one ask via SES or SNS, and writes state to rrs-requests (asked / diverted) and rrs-audit.
• rrs-escalator — SQS-triggered. Builds the handover (customer + job + original signal + grade + any feedback text), de-duplicates against open escalations, and emails the owner inbox via SES so a person can put a real complaint right.
• rrs-sync — scheduled. Pulls the customer sheet from the source and upserts rows into rrs-customers, including the prior-sentiment history the gate reads when no fresh signal exists.
• rrs-sweep — scheduled safety net. Queries rrs-requests on the state GSI for requests whose due time has passed but which never reached a terminal state — a misfired or dropped schedule — and reschedules or escalates them, so no request is silently lost. It never re-sends a request already marked asked or diverted.

Data model and exactly-once

Five DynamoDB tables, all on-demand, hold every bit of state; the design leans on conditional writes and a single terminal state per request to make the whole thing safe to retry.

• rrs-jobs — PK customer_id, SK job_ts; one item per completed job with the raw completion payload and the signal captured at the door. It’s the record of what finished and when.
• rrs-requests — PK request_id (derived deterministically from the completion id, which is what makes the one-per-job guard work). Each item carries the due time, the channel, the grade once assigned, and a state that moves scheduled → asked / diverted / suppressed and never back. A GSI on state plus due time lets the sweep find stragglers cheaply, and a TTL on a retention timestamp expires old requests automatically.
• rrs-customers — PK customer_id; the directory mirrored from the sheet, including contact details and prior sentiment.
• rrs-optout — PK contact (email or E.164 mobile); the suppression list, checked at the door and again at send time before every message.
• rrs-audit — PK request_id, SK ts, append-only; every grade, every message, every escalation, and the facts each was built from, for when someone asks “why did this customer get that?”.

Exactly-once rests on two moves. The one request per job guard is a conditional write to rrs-requests keyed on the completion id: the first completion wins, retries and re-marks no-op. The one send per request guard is a conditional state transition in rrs-sender: it only sends if the request is still scheduled, and flips it to a terminal state in the same conditional write, so a duplicate SQS delivery or a sweep racing the live path can never send twice. The EventBridge schedules are named from the request id, so re-creating one is idempotent too. Anything that still fails five times lands in rrs-work-dlq for inspection rather than being retried forever or lost.

Messaging, schedules, and the model

• Function URL. One, on rrs-webhook, with provider signature verification in-function; AuthType NONE at the edge because authenticity is enforced by the shared secret, not by IAM. No API Gateway.
• SES and SNS. SES sends the review ask and the private-feedback ask by email from a verified domain with DKIM, and sends the escalation handover to the owner inbox; SNS sends the ask by SMS where that’s the customer’s channel. Every send re-checks rrs-optout first.
• EventBridge Scheduler. One one-off schedule per request — an at() expression at the computed due time, named from the request id, targeting rrs-work — plus two recurring rules: rrs-sweep at rate(15 minutes) gated to opening hours, and rrs-sync at rate(15 minutes).
• Secrets Manager. Two secrets — the webhook signing secret and the review-platform / messaging API key — fetched at call time, never in env vars or the sheet.
• Bedrock. Model id anthropic.claude-haiku-4-5 via the Global cross-Region inference profile, invoked only by rrs-sender: once to grade the signal into a band, once to compose the ask. Both calls are handed only the facts they need, and neither ever sees or writes a link.

IAM scope and region

Each function gets its own execution role scoped to exactly what it touches, no wildcards. rrs-webhook can read rrs-customers and rrs-optout, conditionally write rrs-requests and rrs-jobs, read the signing secret, create schedules, and send to rrs-work — it cannot call Bedrock, SES, or SNS. rrs-sender is the only role with bedrock:InvokeModel, scoped to the one Haiku profile; it can send via SES and SNS and write rrs-requests and rrs-audit, but cannot delete from any table. rrs-escalator can send via SES and read its inputs only. The scheduled functions hold the narrow sheet and table permissions they need and no inbound surface at all. Everything runs in eu-west-2; the only cross-Region path is Bedrock’s Global inference profile, which routes the model call for capacity and is not a data store. An AWS Budgets alarm watches the monthly spend — with SMS and Bedrock the lines most likely to move, it’s the cheapest early warning that volume, or a loop, is running hot.

That’s the whole system: a completion webhook, a per-request schedule, two small model calls to grade and to phrase, and a firm rule that only happy customers are ever pointed at a public review — drawn for engineers, but built on the same idea the friendly diagram opened with. Ask once, ask well, ask only the people who are glad, and catch everyone else in private before a bad day becomes a bad rating.

All posts