Skip to content
allanninal.dev/build
Collaborate

Part 4 of 7 · Age check logger series ~5 min read

When a quiet month is the warning

A refusals book with nine entries in four weeks reads as a quiet month, and nobody is ever asked to explain a quiet month. Divide the nine by the number of times the till asked the question and it stops being quiet.

Key takeaways

  • Count refusals per thousand prompts. A raw count mostly measures how busy the store is.
  • Compare a late shift with late shifts. Evenings refuse more because evenings are younger.
  • Watch ID checks as well as refusals. A person who never asks cannot refuse.
  • Require enough prompts before judging anybody, and below that say nothing.
  • A flag is a conversation with a manager, not a verdict and not a notification.

Four stores, one question asked 11,846 times

Refusals per thousand age prompts at four stores over four weeksFour vertical bars showing refusals per thousand age prompts over four weeks. Station Road, two point six. Market Street, twenty point five. Hill Road, eighteen point two. Canal Side, seventeen point zero. A note says Station Road handled three thousand four hundred and eighteen prompts, the most in the group, and recorded nine refusals, the fewest, and that its book was also one of the tidiest.010203040~2.6Station Road~20.5Market Street~18.2Hill Road~17Canal SideRefusals per 1,000 age prompts, four weeksStation Road handled 3,418 prompts, the most in the group, and recorded 9 refusals, the fewest. Its book was also one of the tidiest.
Fig 1. The same four weeks as a rate. Station Road asked the question more often than any other store and refused at less than a sixth of the rate of the next lowest.

Why the count hid it

Nine refusals in four weeks is an unremarkable number on its own. Canal Side recorded 44 and nobody thought Canal Side had a problem; they thought it had a school nearby, which it does. A count mixes up how busy a store is, who lives near it and how carefully its staff ask, and only the last of those is something a manager can change.

Dividing by prompts takes the first of those out. Station Road had more prompts than anywhere else in the group, 3,418, so its nine refusals came to 2.6 per thousand while the other three stores sat between 17.0 and 20.5. Nobody had worked it out, because the prompts lived in the till and the refusals lived in a book.

The same arithmetic sets the floor for judging anybody. At the other stores’ late-shift rate, three hundred prompts should produce about six refusals, so a zero from three hundred means something. Fifty prompts should produce about one, and a zero from fifty means nothing at all. That is where the 300 in the next diagram comes from, and why a part-timer with a quiet month is carried forward rather than judged.

Like with like

Evenings refuse more than mornings in every store, because the people trying to buy alcohol at nine at night are younger than the people buying it at nine in the morning. So a rate is only ever compared with the same kind of shift — weekday or weekend, day or late — and a person who only works mornings is never measured against an evening rate.

Across the other three stores, late shifts handled 4,806 prompts and refused 103, or 21.4 per thousand; their day shifts refused at 14.9. Station Road’s day shifts, at 6.1 per thousand from 1,471 prompts, were low but not absurd. Its late shifts handled 1,947 prompts and refused none. At the other stores’ late-shift rate that would have been about 42.

Flagging a person, carefully

Deciding whether one login's refusal pattern should be flaggedA vertical chain inside an AWS account container, entered from a box on the left labelled Four weeks of one login's prompts, answers. Enough prompts, meaning at least three hundred on this kind of shift, with a side exit reading no: Say nothing, carried into next month. A shared login, meaning prompts at two stores at once or no gap all day, with a side exit reading yes: Fix the login, because a rate for a shift is not a person. Far below the shift, comparing refusals and checks against the group, with a side exit reading yes: Flag, a conversation this week. The final step is Reviewed, with the numbers kept and nothing to say. A note says most months most logins reach the last box, and a flag means volume and shift cannot explain the numbers, and a manager should find out what can.AWS ACCOUNTFour weeksof one login'sprompts, answersEnough prompts?at least 300 onthis kind of shiftSay nothingcarried intonext monthnoA shared login?two stores at once,or no gap all dayFix the logina rate for a shiftis not a personyesFar below the shift?refusals and checksagainst the groupFlaga conversationthis weekyesReviewednumbers kept, andnothing to sayMost months most logins reach the last box. A flag means volume and shift cannot explain the numbers, and a manager should find out what can.
Fig 2. Three questions before anybody’s name goes anywhere. The first two exist to stop the third from being wrong.
  • Machine learning
  • Security & identity
  • Management
  • Analytics
  • People

The person behind the zero

One colleague worked most of Station Road’s Friday and Saturday late shifts. In the four weeks they answered 1,206 prompts: 1,171 as clearly over 25, 35 as ID seen, and none as refused. Across the whole group, day and late together, 7.9 per cent of prompts became an ID check or a refusal. For this person, on late shifts alone, where that share should be higher, it was 2.9 per cent. At the other stores’ late-shift rate their 1,206 prompts would have produced about 26 refusals.

None of that proves anybody sold to a child. It says a person was answering the prompt from the customer’s appearance nearly every time, on the shifts where appearance is least reliable, and the numbers would have said so weeks before the test purchase. Flagged, it reaches the area manager as a login, four numbers and the shifts they came from.

What happens next is not this system’s business. It might be training, which is owned elsewhere; a rota that stops leaving one person alone on the busiest nights of the week; or a conversation that finds nobody ever explained the third answer. The system records that the flag was raised, who received it and when it was closed, and nothing about the conversation.

What not to do with the rate

Do not publish it to staff as a league table. A target for refusals produces refusals, including of customers who are obviously forty, and a book full of invented lines is worse evidence than a thin honest one. Do not set a number below which people are disciplined. And do not calculate it on a few dozen prompts, where one extra refusal moves the rate by more than the difference being looked for.

The next post is what happens when it goes wrong anyway: the evidence a failed test purchase or a licence review asks for, and the three months afterwards, when a second sale becomes a different offence with a different defendant.

All posts