Skip to content

Part 7 of 7 · Delivery exception handler series ~7 min read

Engineering reference: the delivery exception handler architecture

The first six posts are for the person deciding whether to build this. This one is for the person building it. Same system, no analogies: the services by name, the functions, the two tables, the status normalisation, and how the silence sweep works.

Key takeaways

  • Single region, single account. Every resource is regional; nothing is global except the IAM roles.
  • 3 Lambda functions, each with its own execution role. No shared role, no wildcards on resources.
  • 2 DynamoDB tables, each keyed so the concurrency story is a condition expression rather than a lock.
  • One Bedrock model, called once, with a JSON schema it must fill or leave null.
  • Nothing always-on: no instance, no container, no provisioned capacity.

The system, by service name

The delivery exception handler drawn with AWS service namesThree boxes across the top outside the AWS account. Carrier APIs, providing webhooks and polling. The order system, read only. And SES and the queue, reaching customers and staff. Inside the account, three groups. A Function URL for webhooks and EventBridge running an hourly silence sweep. Three Lambda functions named ingest, sweep and notify. And two DynamoDB tables named shipments and exceptions. A note gives the region as us-east-1, one account, and states that thresholds are derived weekly from delivered shipments' own scan gaps.AWS ACCOUNTCarrier APIswebhooks and pollingThe order systemread onlySES and the queuecustomers, and staffFunction URL + EventBridgewebhooks,hourly silence sweepLambda x3ingest, sweep, notifyDynamoDB x2shipments, exceptionsingroundsoutus-east-1. One account. Thresholds derived weekly from delivered shipments' own scan gaps.
Fig 1. The same shape as Part 1 with the service names filled in. Nothing here is new; it is the same three groups, named.
  • Compute
  • Database
  • App integration
  • Networking

Region and account

  • Region: us-east-1. Chosen because SES inbound receipt rules exist in only a subset of regions and this one has the widest Bedrock model availability. If your data has to stay elsewhere, check both constraints before moving: inbound SES is the binding one.
  • Account: one. This is a small system, and a separate account per environment costs more in wiring than it saves. A dev and a prod stack in the same account, with distinct resource prefixes, is the right size here.
  • Everything is regional. The only global resources are the IAM roles and policies. There is no CloudFront, no global table and no cross-region replication, because nothing here has a latency or durability requirement that would justify them.

Lambda inventory

FunctionTriggerDoesTimeout / memory
de-ingestFunction URL, carrier webhooksNormalises the status, updates last_event_at, raises event-based exceptions10s / 512 MB
de-sweepEventBridge, hourlyFinds shipments past their silence threshold; runs triage; sets recheck times300s / 1024 MB
de-notifySQSSends the four-line message and the promised follow-up, even with no news15s / 512 MB

Splitting this into separate functions is not about modularity. It is that only one of them needs Bedrock permissions and only one is reachable from the public internet, and neither of those is true if it is one handler behind a router.

IAM, scoped

RoleAllowedOn
de-ingest-roledynamodb:UpdateItem, dynamodb:PutItemShipments and exceptions
de-sweep-roledynamodb:Query, dynamodb:UpdateItem, sqs:SendMessageBoth tables; the notify queue
de-notify-roleses:SendEmail, dynamodb:UpdateItemOne verified identity; exceptions

No role has a Resource: “*” on anything that writes, and every GetSecretValue grant names a single secret arn. That is why there is more than one secret rather than one JSON blob with everything in it.

DynamoDB schemas

Table: shipments

PK   tracking_ref      S
     order_id          S
     carrier           S
     service           S   the threshold is keyed on carrier#service
     despatched_at     S
     last_event_at     S   the field the silence sweep queries
     last_status_raw   S   the carrier’s own code and text, kept
     last_status       S   normalised: in_transit | attempted | address | ...
     delivered_at      S   set once; the sweep stops looking
     ttl               N   epoch, +90 days after delivery

GSI1: last_status + last_event_at  — the silence sweep is one query
per active status band, not a scan across every shipment.

Table: exceptions

PK   tracking_ref      S
SK   raised_at         S
     kind              S   silence | attempted | address | refused
                           | never_collected | damaged
     cause             S   internal | customer_data | carrier
     recheck_at        S   set for self-resolving kinds
     contacted_at      S   null if we deliberately did not contact
     promised_update   S   the date we said we would write again
     resolved_at       S
     resolution        S   delivered | reshipped | refunded | traced_lost

`cause` is assigned at triage and is what makes the aggregate report
separate internal failures from carrier ones.

Inbound and outbound

  • Webhooks where available, polling where not. Poll intervals are proportional to the service’s typical scan gap and stop entirely once delivered.
  • The silence sweep is a GSI query, not a scan. Shipments are indexed by status and last event time, so the sweep touches only what could be overdue.
  • Thresholds are recomputed weekly from the observed scan gaps on delivered shipments, per carrier and service, so a carrier changing their scanning practice does not produce a wave of false exceptions.
  • Order data is read, never written. This system has no write access to the order system.

The model call

  • There is no model in this system. Status normalisation is a mapping table and detection is a timestamp comparison.
  • The tempting use is predicting which shipments will fail. It would produce a score over the same data the thresholds already use, with less explainability.
  • A second tempting use is drafting the customer message. The four lines are a template with two substitutions, and Part 4 is about why the wording should not vary.
  • Classifying free-text carrier notes into causes is defensible where a carrier sends prose rather than codes, and the raw text is kept beside it.
  • The cost page assumes none, which is why tracking writes are the only meaningful variable.

Things worth knowing before you build it

  • Derive silence thresholds from your own delivered shipments per carrier and service. A single global threshold either floods the queue or catches nothing.
  • Keep the raw carrier status next to the normalised one. New codes appear and the mapping will be wrong sometimes.
  • Give never-collected a tighter threshold and its own category. It is usually the most common exception and it is entirely internal.
  • Hold self-resolving exceptions with a recheck time rather than discarding them, or the second failed attempt looks like the first.
  • Never write a predicted delivery date into a customer message. A promised communication date is a promise you control.

That is the whole system. Seven posts, one diagram at a time, and nothing in it that needs a server.

All posts