Skip to content
allanninal.dev/build
Collaborate

Part 4 of 7 · Data request handler series ~5 min read

How the data gets gathered

Gathering looks like the bulk of the work and is mostly a fan-out. The part that actually determines whether the response is complete is knowing what to ask, which means having written down every system that holds anything about a person — and almost nobody has.

Key takeaways

  • The system register is the product. Everything else is a fan-out over it.
  • Ask every system, including the ones you expect to hold nothing.
  • Record “nothing held” explicitly; an unasked system is not an empty one.
  • Systems with no export API get a dated task for a named person.
  • Backups and logs hold personal data and are frequently forgotten.

The register

One row per system that could hold anything about a person: what it holds, how to ask it, who owns it, and whether the ask is automatic or manual. Building it is a couple of hours of asking around, and it is the entire difference between a response that is complete and one that is merely plausible.

SystemHoldsAsk
E-commerceOrders, addresses, payment referencesAPI
Email marketingAddress, subscription state, opensAPI
HelpdeskTickets, conversations, internal notesAPI
AccountingInvoices, payment recordsManual export
Booking toolAppointments, notesManual
Shared driveWhatever somebody filed thereSearch, manual
BackupsEverything, historicallyDocumented, not searched

The last two rows are the ones that get left out and both matter. A shared drive with a folder of customer correspondence is holding personal data whether or not anybody thinks of it as a system, and backups hold everything that was ever deleted, which is a fact that has to be stated rather than searched.

Asking

How data is gathered from every system for a requestA vertical chain of five steps entered by a box labelled A verified request, with the clock running. Step one fans out to every system from the register, always all of them. Step two asks whether the system can be asked automatically because an API exists; if not it exits to A dated task assigned to a named person. Step three fetches what it holds, searching by every identifier. Step four asks whether anything was found; nothing exits to Record nothing held, which is an answer rather than a gap. Step five adds it to the package with its source. A note says every system produces an answer including nothing, and a missing answer is not one.AWS ACCOUNTA verified requestclock runningFan out to every systemfrom the registerSystem registerevery one, alwaysAutomatic?an API existsA dated taskto a named personnoFetch what it holdsby every identifierAnything found?Record 'nothing held'an answer, not a gapnoAdd to the packagewith its sourceEvery system produces an answer, including nothing. A missing answer is not one.
Fig 1. How the gathering runs. The explicit nothing-held record is what lets the package say it is complete rather than hoping.
  • Database
  • App integration
  • Machine learning
  • Security & identity
  • Management
  • Analytics
  • People

By every identifier

A person is in different systems under different keys: an email address in one, a customer number in another, a phone number in a third, and quite possibly a second email address they used once. Searching each system by only the identifier the request arrived with will miss data that is genuinely held.

So the gather step starts by assembling every identifier known for that person — from the matching work in the duplicate merger, if that exists — and searches each system by all of them. It is the difference between a response that is complete and one that is complete for one email address.

Nothing held is an answer

Recording an explicit “asked, holds nothing” per system is what lets the package say, credibly, that these seven systems were checked and three of them hold nothing. Without it the package contains four sections and no evidence that the other three were considered at all.

It also catches the specific failure where a system’s API silently returns empty because the query was wrong: a system that returns nothing for every request, month after month, in a business with customers, is worth checking rather than trusting.

The manual ones

How systems without an export API are handled in a data requestA horizontal row of five boxes. No API: about half of them. A dated task: to a named person. Due in three days: well inside the clock. They attach it: or say nothing held. Same record: as an automatic answer. A note says giving them three days of a thirty-day clock is what keeps the review unhurried.HALF OF EVERY REGISTER IS MANUALNo APIabout half of themA dated taskto a named personDue in 3 dayswell inside the clockThey attach itor say nothing heldSame recordas an automatic answerGiving them three days of a thirty-day clock is what keeps the review unhurried.
Fig 2. How manual systems are handled. Setting internal deadlines well inside the statutory one is what preserves time for the review, which is where errors are expensive.
  • Machine learning
  • Security & identity
  • Management
  • People

Three days is deliberately aggressive against a thirty-day statutory window, and the reason is that the review in Part 5 is the part that needs unhurried attention. A package assembled on day four leaves three weeks of margin; one assembled on day twenty-four leaves a rushed review, which is exactly when something that should have been redacted is not.

Next: the review and the package.

All posts