Skip to content
allanninal.dev/build
Collaborate

Part 7 of 7 · Waste collection verifier series ~7 min read

Engineering reference: the waste collection verifier architecture

The first six posts are for the person deciding whether to build this. This one is for the person building it. Same system, no analogies: the services by name, the functions, the two tables, the expected-versus-actual join, and the one model call per invoice.

Key takeaways

  • Single region, single account. Every resource is regional; nothing is global except the IAM roles.
  • 4 Lambda functions, each with its own execution role. No shared role, no wildcards on resources.
  • 2 DynamoDB tables, each keyed so the concurrency story is a condition expression rather than a lock.
  • One Bedrock model, called once, with a JSON schema it must fill or leave null.
  • Nothing always-on: no instance, no container, no provisioned capacity.

The system, by service name

The waste collection verifier drawn with AWS service namesThree boxes across the top outside the AWS account. Contracts and variation letters. The morning check, three buttons and one photo. And Invoices, monthly, itemised and often scanned. Inside the account, three groups. S3 and EventBridge for uploads, schedules and windows. Four Lambda functions named terms, check, invoice and pursue. And two DynamoDB tables named sites and events. A note gives the region as us-east-1, one account, and states that expected collections are generated ahead and actuals arrive against them.AWS ACCOUNTContractsand variationlettersThe morning checkthree buttons,one photoInvoicesmonthly, itemised,often scannedS3 + EventBridgeuploads, schedules,windowsLambda x4terms, check,invoice, pursueDynamoDB x2sites, eventsingroundsoutus-east-1. One account. Expected collections are generated ahead; actuals arrive against them.
Fig 1. The same shape as Part 1 with the service names filled in. Nothing here is new; it is the same three groups, named.
  • Compute
  • Storage
  • Database
  • Front-end & mobile

Region and account

  • Region: us-east-1. Chosen because SES inbound receipt rules exist in only a subset of regions and this one has the widest Bedrock model availability. If your data has to stay elsewhere, check both constraints before moving: inbound SES is the binding one.
  • Account: one. This is a small system, and a separate account per environment costs more in wiring than it saves. A dev and a prod stack in the same account, with distinct resource prefixes, is the right size here.
  • Everything is regional. The only global resources are the IAM roles and policies. There is no CloudFront, no global table and no cross-region replication, because nothing here has a latency or durability requirement that would justify them.

Lambda inventory

FunctionTriggerDoesTimeout / memory
wc-termsS3 upload of a contract or variationOne model call; writes the schedule, the rate card, the windows and the renewal date60s / 1024 MB
wc-checkAPI, from the morning checkMarks each expected collection done, missed or partial; stores the resized photograph10s / 512 MB
wc-invoiceS3 upload of an invoiceOne model call to line items; matches every line to a rate; opens queries for the rest120s / 2048 MB
wc-pursueEventBridge, dailyFires reporting and challenge windows, chases credits, and warns on the renewal notice date60s / 512 MB

Splitting this into separate functions is not about modularity. It is that only one of them needs Bedrock permissions and only one is reachable from the public internet, and neither of those is true if it is one handler behind a router.

IAM, scoped

RoleAllowedOn
wc-terms-rolebedrock:InvokeModel, s3:GetObject, dynamodb:PutItemOne model id; the contracts prefix; sites
wc-check-roledynamodb:UpdateItem, s3:PutObjectEvents; the evidence prefix
wc-invoice-rolebedrock:InvokeModel, s3:GetObject, dynamodb:Query, dynamodb:PutItemOne model id; the invoices prefix; both tables
wc-pursue-roledynamodb:Query, dynamodb:UpdateItem, ses:SendEmailEvents; one verified identity

No role has a Resource: “*” on anything that writes, and every GetSecretValue grant names a single secret arn. That is why there is more than one secret rather than one JSON blob with everything in it.

DynamoDB schemas

Table: sites

PK   site_id           S
SK   ’#terms’          S   one item per site
     containers        L   [{stream, size, count, days}]
     rates             M   standing, lift, per_tonne, named surcharges
     escalation        M   {window, cap, index}
     report_window_h   N   hours to report a miss — usually 24 or 48
     challenge_days    N   days to challenge an invoice line
     term_end          S
     notice_months     N   the field that decides whether you have a choice
     notice_by         S   term_end minus notice_months, precomputed
     source_key        S   the contract PDF

notice_by is stored rather than derived, because it is the one date
that has to fire even if nobody ever opens this record again.

Table: events

PK   site_id#period    S
SK   due_at#stream     S   one item per expected collection
     expected          BOOL generated ahead from the schedule
     state             S   done | missed | partial | no_access
     fill              S   low | half | full | overflow
     photo_key         S   only on exceptions, and on a sample
     reported_at       S   inside report_window_h, or the claim is dead
     report_ref        S   the contractor’s reference
     charged           N   from the invoice line that matched
     credit_expected   N
     credit_received   N
     query_state       S   none | raised | answered | credited | rejected

Expected collections are written ahead of time. A missing actual is
then a row that stayed in expected state, not an absence of data.

Inbound and outbound

  • Expected collections are generated forward from the schedule, monthly. Detecting a miss becomes a query for rows nobody updated, which is far more reliable than inferring absence.
  • The morning check is one request and is silent when everything is normal. Notifications only ever come from exceptions.
  • Photographs are kept on exceptions and on a small random sample of normal days, which is what makes a challenge credible.
  • Invoices arrive by upload or by mailbox, and both land in the same function. Nothing here depends on a contractor portal or an integration that could be withdrawn.

The model call

  • Two calls, both per document. One per contract or variation letter, one per invoice. Nothing per collection.
  • A more capable model for invoices, deliberately. Multi-page scanned itemised documents are where cheap extraction quietly produces wrong quantities.
  • A JSON schema, with nulls allowed. A missing notice period becomes a question to a human, because guessing that field can cost three years.
  • Reconciliation is arithmetic, not a model judgement. Every discrepancy has to be stateable to a contractor in one sentence with a line reference.
  • No image model on the evidence photographs. They are for a human at the point of dispute, and a fill level from a tap is more reliable than one from a picture of a bin.

Things worth knowing before you build it

  • Generate expected collections ahead of time. Detecting a miss as a row that never got updated is reliable; inferring it from the absence of a record is not.
  • Store the notice date, not just the term end. The decision point is six to nine months before the date everybody has in their head, and it is the most expensive field in the contract.
  • Report misses within the contractual window, which is usually 24 or 48 hours. Almost every failed credit claim failed on timing rather than on merit.
  • Cut collections against your busiest measured month, never the average. A frequency reduction reversed in November is reinstated at a worse rate.
  • Keep the exception photographs and expire the rest. The evidence that matters is small, and keeping everything at full resolution is the only way to make this system cost real money.

That is the whole system. Seven posts, one diagram at a time, and nothing in it that needs a server.

All posts