Skip to content
allanninal.dev/build
Collaborate

Part 4 of 7 · Allergen checker series ~5 min read

Why the system never says safe

This is the design decision that everything else in the system exists to support, and it is the one that will be argued with, because a green tick is exactly what everybody wants.

Key takeaways

  • Cross-contamination is a kitchen fact and is not in any database.
  • Severity varies by person and changes what counts as acceptable.
  • Output declared and unknown, and let a person handle the rest.
  • A green tick would be trusted more than it deserves, which is the harm.
  • The interface should make the conversation easier, not shorter.

What is not in the data

Three inputs to an allergen question and which are available in dataThree boxes stacked on the left. Declared ingredients, which this system knows, labelled in the data. Cross-contamination from a shared fryer, boards or airborne flour, labelled not in the data. And This person's severity, whether trace or a large amount, labelled not in the data. All three converge on The actual question: can this person eat this? And that leads down to A person answers it, with the data to hand. A note says two of the three inputs are outside any system, hence no verdict.Declared ingredientsthis system knowsin the dataCross-contaminationshared fryer, boards,airborne flournot in the dataThis person's severitytrace, or a lotnot in the dataThe actual questioncan this person eat this?A person answers itwith the data to handTwo of the three inputs are outside any system. Hence no verdict.
Fig 1. The three inputs to the real question and where each lives. A system holding one of the three should not be answering the question.
  • Compute
  • Machine learning
  • Security & identity
  • Management
  • People

Cross-contamination

A dish whose ingredients contain no gluten, prepared on a board that had bread on it, in a kitchen where flour is in the air, cooked in a fryer shared with battered fish, is not a gluten-free dish. None of that is in a specification and none of it can be.

Some kitchens have separate equipment and procedures for this and some do not, and that is a matter of physical arrangement and training. A system can record what a kitchen claims about its procedures, and a claim is not the same as a measurement.

Severity

For one person an allergen means discomfort; for another a trace is a medical emergency. The same dish with the same “may contain” statement is a reasonable choice for one and an unacceptable risk for the other, and neither the system nor the menu knows which person is asking.

Why a green tick is harmful

The specific problem with a safety verdict

  • It will be trusted more than the underlying data justifies. That is what a tick does.
  • It ends the conversation that should be happening between the customer and the kitchen.
  • It cannot express severity, so it is either too cautious for most people or not cautious enough for some.
  • It hides the unknowns. A dish with four unknown allergens can only honestly produce a question mark.
  • It moves responsibility to a system that cannot hold it.
  • The alternative costs nothing: show what is declared, show what is unknown, and let a person talk to a person.

The last line is the practical point. Refusing to produce a verdict does not make the system less useful; it makes it useful in a different way, by ensuring that whoever is having the conversation has current, complete, honestly-caveated information in front of them.

What the interface shows

What an allergen lookup shows when a customer asks about a dishA vertical chain of five steps entered by a box labelled A customer asks about a dish. Step one shows Contains, named first. Step two shows May contain, named distinctly. Step three shows Unknown, named prominently. Step four shows a Kitchen note covering things like a shared fryer. Step five says please check with the kitchen. A note says five things and no verdict, and the last line is not a disclaimer but the instruction.AWS ACCOUNTA customer asksabout a dishContainsnamed, firstMay containnamed, distinctlyUnknownnamed, prominentlyKitchen noteshared fryer, etc'Please check withthe kitchen'Five things, no verdict. The last line is not a disclaimer; it is the instruction.
Fig 2. What a member of staff sees when a question is asked. Every element is a fact and none of them is a conclusion.
  • Compute
  • App integration
  • Security & identity
  • Management
  • People

The kitchen note

A short, honest, per-site statement about the kitchen’s actual arrangements: whether there is a shared fryer, whether flour is used in the same space, whether separate equipment exists for particular allergens.

It is written once by whoever runs the kitchen, reviewed when the kitchen changes, and it is the most useful single sentence in the whole interface because it addresses the part the specifications cannot.

Making the conversation easier

The measure of success here is not that fewer questions get asked. It is that when a question is asked, the person answering has accurate current information, knows which parts of it are incomplete, and is not guessing.

A system that reduced the number of conversations would be doing harm. One that makes each conversation better informed is doing the whole of what software can honestly do here.

Next: the substitution nobody recorded.

All posts