Skip to content
allanninal.dev/build
Collaborate

Part 3 of 7 · Cold chain monitor series ~6 min read

How an excursion is told from a breach

Every chilled unit goes out of range several times a day, because people open doors. A monitoring system that treats each of those as a failure produces an alarm nobody believes within about a week.

Key takeaways

  • A single reading out of range is not an event. Duration is.
  • Consecutive time out of range triggers the alarm; cumulative time triggers a review.
  • Different units get different rules; a walk-in is not a display fridge.
  • The alarm carries the last few hours, because trajectory is the diagnosis.
  • Fewer, louder alarms beat more, softer ones. Every time.

Two clocks

How consecutive and cumulative time out of range are judged separatelyA vertical chain of five steps entered by a box labelled A reading out of range, above or below. Step one starts the consecutive clock if it is not already running. Step two asks whether the next reading is back in range; if so it exits to Excursion closed, logged and added to the cumulative total. Step three asks whether it is past the limit, ten minutes for this unit; if not it exits to Still watching, with no alarm yet. Step four declares a breach, alarms and escalates. Step five asks whether cumulative time today is over forty-five minutes, drawing on a side box describing it as a separate rule for many small excursions; if so it exits to a Review flag rather than an alarm. A note says consecutive time wakes somebody up and cumulative time appears in the morning.AWS ACCOUNTA reading out of rangeabove or belowStart the consecutive clockif not already runningBack in range?next readingExcursion closedlogged, added to cumulativeyesPast the limit?10 min for this unitStill watchingno alarm yetnoBreachalarm, and escalateCumulative today?over 45 min?A separate rulemany small excursionsReview flagnot an alarmyesConsecutive time wakes somebody up. Cumulative time appears in the morning.
Fig 1. The two duration rules and their different consequences. Separating them is what lets the system catch a door left ajar all afternoon without alarming at three in the morning.
  • Compute
  • App integration
  • Machine learning
  • Management
  • Analytics

Why cumulative matters too

A door propped open for two minutes every ten minutes all afternoon never triggers a consecutive rule and does real damage. It is a common and genuinely bad pattern — usually somebody restocking with the door wedged — and only a cumulative measure catches it.

It is a review flag rather than an alarm because the right response is a conversation the next morning, not a call at four in the afternoon. Matching the urgency of the response to the urgency of the problem is what keeps both channels credible.

Different units, different rules

UnitRangeBreach afterWhy
Walk-in freezer−22 to −16 °C20 minutesLarge thermal mass; slow to change either way
Under-counter fridge1 to 5 °C10 minutesSmall; warms quickly
Open display chiller1 to 7 °C20 minutesConstantly disturbed; a tight rule is unworkable
Vaccine fridge2 to 8 °C5 minutesThe stock is not replaceable
Delivery vehicle−20 to −15 °C30 minutesDoors open at every drop

The fourth row is the useful contrast. The rule is not about the physics of the unit; it is about what is in it and what happens if you are wrong. A vaccine fridge gets a tight rule because the consequence of a missed breach is severe and irreversible.

Every one of these numbers is a judgement that belongs to whoever is responsible for the stock, written in configuration where they can see it, not embedded in code. The system applies rules; it does not choose them.

Fewer, louder

A month of cold chain alarms under threshold rules versus duration rulesA stacked bar chart with two bars. Two series: real breaches in the month in red, and alarms that were door openings in grey. Threshold alarms produce three real breaches and two hundred and fourteen door openings. Duration rules produce three real breaches and four false alarms. A note says the same three breaches appear in both, and only one of these systems still gets answered at three in the morning.0100200300400~217Threshold alarms~7Duration rulesReal breaches in the monthAlarms that were door openingsSame three breaches. One of these systems still gets answered at 3am.
Fig 2. A month of alarms under two rule designs. The three that matter are identical; what differs is whether anybody is still listening when they fire.

This chart is the entire argument of the post. Alarm fatigue is not a failure of discipline by the person on call; it is the predictable result of a system that cried wolf two hundred times and then once did not.

The alarm carries history

A message saying “freezer 2 is at minus nine” is much less useful than one showing the last six hours. Was it minus eighteen an hour ago and rising steadily, which is a compressor? Or has it been hovering around minus fifteen all evening, which is a door seal?

The trajectory determines what somebody does when they get out of bed, and including it in the alarm itself — a small chart, not a link to a dashboard behind a login — is the difference between a useful alarm and one that requires a laptop.

Next: the sensor that is lying.

All posts