Skip to content
allanninal.dev/build
Collaborate

Part 5 of 7 · Form spam filter series ~5 min read

How the filter stays tuned

A spam filter’s accuracy cannot be measured, and understanding why is what makes it possible to tune one honestly. You can count what you caught. You cannot count what you wrongly caught, because nobody tells you.

Key takeaways

  • False negatives are visible: spam in the inbox. False positives are not.
  • The review queue’s size is the best available proxy for whether thresholds are right.
  • A confirmed false positive is the most valuable signal available and is rare.
  • Tune towards a review queue of about ten items a day, not towards accuracy.
  • Never tighten a threshold without checking what it would have caught historically.

Only one error is visible

Why only one of a spam filter's two error types is measurableA vertical chain of five steps entered by a box labelled Two kinds of error, not symmetric. Step one is spam in the inbox, which somebody sees; it exits to Countable, needing one tap to report. Step two is a real enquiry in quarantine, which nobody sees; it exits to Invisible, unless somebody rings. Step three concludes: do not tune on accuracy, because you cannot measure it. Step four tunes on queue size, a measurable proxy. Step five also tunes on confirmed misses, which are rare and decisive. A note says any claim about this filter's accuracy is a claim about the half you can see.AWS ACCOUNTTwo kinds of errornot symmetricSpam in the inboxsomebody sees itCountableone tap to reportvisibleReal in quarantinenobody sees itInvisibleunless somebody ringshiddenSo do not tune on accuracyyou cannot measure itTune on queue sizea measurable proxyAnd on confirmed missesrare and decisiveAny claim about this filter's accuracy is a claim about the half you can see.
Fig 1. The asymmetry that governs tuning. One error type announces itself and the other does not, so the tuning target has to be something other than accuracy.
  • App integration
  • Security & identity
  • Management
  • Analytics
  • Front-end & mobile

Tuning on the queue

The review queue is the tuning instrument. Too long and somebody stops reading it, which means the four-hour timeout starts delivering everything and the filter has effectively stopped. Too short and the thresholds are confident about things they should not be, which is where invisible false positives live.

About ten items a day is the target. That is ninety seconds of somebody’s attention, twice a day, and it is a band wide enough that genuinely ambiguous submissions land in it rather than being decided by a threshold.

The monthly numbers

One month of form submissions summarised in five numbersA horizontal row of five boxes. Submissions: four hundred and forty-one. To the inbox: thirty-eight. Reviewed: sixty-one, of which nine were real. Quarantined: three hundred and forty-two. Reported as spam: two, from the inbox. A note says the third number is the one to steer by, and about two a day reviewed is right.ONE MONTH OF A CONTACT FORMSubmissions441To the inbox38Reviewed61, of which 9 realQuarantined342Reported as spam2, from the inboxThe third number is the one to steer by. Two per day reviewed is about right.
Fig 2. A month of submissions in five numbers. The reviewed count and the proportion of it that turns out to be real are what indicate whether the middle band is set correctly.
  • Machine learning
  • Security & identity
  • Management
  • Front-end & mobile
  • People

Nine real submissions out of sixty-one reviewed is a healthy band: it means the middle genuinely contains a mix, which is what a middle should contain. If it were sixty out of sixty-one, the band is catching things the cheap signals should have passed; if it were one, the band is catching things they should have quarantined.

Two reported from the inbox

That is the visible error rate and it is deliberately not zero. A filter tuned to let nothing through is tuned tightly enough to be quarantining real enquiries, and two obvious spam messages a month reaching an inbox is a much cheaper cost than whatever tightening would be required to remove them.

Changing a threshold

Before tightening anything

  • Replay it. Every submission and its signals are stored, so a proposed threshold can be run over the last three months before it goes live.
  • Look at what it would have caught. Not the count — the actual messages. A tightening that would have quarantined two real enquiries is not worth the eleven spam it also catches.
  • Change one thing. Two threshold changes at once means the next month’s numbers cannot attribute the difference.
  • Write down why. Six months later, “why is the fill-time floor three seconds” is a question somebody will ask, and the answer should not be lost.

The replay is what makes tuning safe, and it is only possible because every submission is retained with its signals. That is the same structural decision as keeping raw rows in the search reporter and storing every sample in the speed watcher: keep the input, and changing the processing becomes cheap and reversible.

Next: what all of this costs to run.

All posts