Skip to content
allanninal.dev/build
Collaborate

Part 5 of 7 · Uptime status poster series ~5 min read

How an incident gets closed

The automated part of this system produces a colour. Everything customers actually value about a status page is the sentence next to it, and that sentence is a commercial communication written under pressure — which is exactly why it is drafted in advance and published by a person.

Key takeaways

  • The status changes automatically; the note is drafted and published by a person.
  • Three drafts exist in advance, so nobody writes prose during an incident.
  • Updates on a fixed cadence even when there is nothing new, because silence reads as worse.
  • A resolution says what was affected, for how long, and whether anything needs redoing.
  • Publish uptime monthly per journey. A single site-wide figure hides the checkout outage.

Drafts written in advance

Nobody writes well at 14:08 with a broken checkout and a phone ringing. So the three notes that cover almost every incident are written calmly in advance, stored, and presented for one-tap publication with the affected journey filled in.

The three drafts

  • Investigating. “We are aware that {journey} is not working correctly and are investigating. We will update this page within 30 minutes.”
  • Identified. “We have identified the cause of the problem with {journey} and are working on a fix. We will update this page within 30 minutes.”
  • Resolved. “{journey} is working normally again. The problem lasted from {start} to {end}. {impact}”
  • Nothing else is templated. Anything more specific is written in the moment, because a specific claim made from a template is how a status page says something wrong.

The thirty-minute promise in the first two is load-bearing and it is a commitment the system then enforces: a reminder fires at twenty-five minutes so that the promise is kept even when the incident is absorbing everybody.

Updating when there is nothing to say

How updates are posted during an open incidentA vertical chain of five steps entered by a box labelled An open incident, published. Step one waits twenty-five minutes since the last update. Step two asks whether there is anything new; if so it exits to Post it, specific and written in the moment. Step three posts anyway, saying still working on it. Step four repeats every thirty minutes until resolved. Step five is Resolved, with the impact stated. A note says silence during an outage reads as worse than the outage, and an empty update is not empty.AWS ACCOUNTAn open incidentpublished25 minutes elapsedsince the last updateAnything new?Post itspecific, written nowyesPost anyway'still working on it'Repeat every 30until resolvedResolvedwith the impactSilence during an outage reads as worse than the outage. An empty update is not empty.
Fig 1. The update cadence during an open incident. Posting nothing new is still worth posting, because the alternative is read as nobody being on it.
  • App integration
  • Machine learning
  • Management
  • Front-end & mobile

The resolution note

The one that matters commercially, and it has to answer three questions a customer actually has: what was affected, for how long, and is there anything they need to do.

The third is the one most status pages omit and the one that generates support contacts. “Orders placed between 14:06 and 14:31 may not have gone through; please check your order history or contact us” prevents a hundred emails. “The issue has been resolved” generates them.

Monthly numbers

Monthly uptime published per journey rather than as one figureA horizontal row of five boxes. Home: one hundred per cent. Search: ninety-nine point nine eight per cent. Login: ninety-nine point nine nine per cent. Checkout: ninety-nine point four one per cent. Site-wide: would say ninety-nine point eight five per cent. A note says the single site-wide figure hides the only outage that cost anybody money.PER-JOURNEY UPTIME, NOT ONE NUMBERHome100%Search99.98%Login99.99%Checkout99.41%Site-widewould say 99.85%The single site-wide figure hides the only outage that cost anybody money.
Fig 2. Why uptime is published per journey. A blended figure averages the journey that matters against three that were fine, and reports a good month.
  • Machine learning
  • Security & identity
  • Management
  • Analytics

Publishing four numbers instead of one is slightly more embarrassing and much more honest. The checkout figure is the one a customer cares about, and averaging it against three journeys that were fine produces a headline that is technically true and practically misleading.

What not to publish

Response times, error rates and internal component status all belong on an internal dashboard rather than a public page. A status page has one audience with one question: can I do the thing I came to do. Adding graphs invites a reader to interpret data they have no context for, and during an incident that produces speculation rather than patience.

Next: what all of this costs to run.

All posts