Skip to content
allanninal.dev/build
Collaborate

Part 4 of 7 · Form spam filter series ~5 min read

How a borderline reaches a person

A review queue is the part of this design most likely to be abandoned, and it is abandoned for a completely predictable reason: it takes too long. Everything here is about the ninety seconds.

Key takeaways

  • The queue is a batched message twice a day, not a place somebody has to visit.
  • Each item shows the message and the reason it is held, not a score.
  • Two buttons: it is real, or it is spam. Nothing else.
  • An unreviewed item is delivered after four hours rather than held indefinitely.
  • Quarantine is searchable by name and address, which is what resolves the awkward call.

Twice a day, ninety seconds

How a held form submission reaches a person for reviewA vertical chain of five steps entered by a box labelled A held submission, from the model band. Step one waits for the batch at eleven in the morning and four in the afternoon; if four hours pass first it exits to deliver it anyway. Step two sends one message containing all held items. Step three shows the message and the reason it is held, rather than a score. Step four gives two buttons on each item: real, or spam. Step five routes and records, and the answer tunes the filter. A note says the timeout is the safety net, because an unreviewed enquiry is delivered rather than lost.AWS ACCOUNTA held submissionfrom the model bandWait for the batch11am and 4pmFour hours passed?deliver it anywaytimeoutOne messageall held itemsMessage and reasonnot a scoreTwo buttons eachreal, or spamRouted and recordedthe answer tunes the filterThe timeout is the safety net: an unreviewed enquiry is delivered, not lost.
Fig 1. How held submissions reach a person. The four-hour timeout means the review queue failing is a source of spam rather than a source of lost leads.
  • App integration
  • Management
  • Analytics

Why batched

A held submission that generates its own notification produces eight interruptions a day and gets muted. One message at eleven and one at four contains everything, takes ninety seconds, and is a thing somebody actually does.

The four-hour timeout underneath it is what makes the batching safe. If nobody opens the eleven o’clock message, its items are delivered to the inbox at three regardless. The failure mode of the review queue is therefore some spam getting through, which is the cheap error.

The reason, not the score

“Held because: no links, but submitted in 11 seconds and the message is two lines” is something a person can weigh in a second. “Spam score 0.62” is not, and it also invites somebody to start reasoning about the number rather than reading the message.

Every business running a contact form eventually gets the phone call: I filled in your form last week and never heard back. Without a searchable quarantine that conversation has no resolution and ends with an apology and a suspicion.

How a quarantined genuine enquiry is found and recoveredA horizontal row of five boxes. Somebody says they filled in your form last Tuesday. Search quarantine: by name or address. Found it: with the reason it was held. Deliver it now: and mark it real. The filter learns: that pattern loosens. A note says false positives are invisible by construction and this is the only way you find them.THE CONVERSATION THAT REVEALS A FALSE POSITIVE'I filled in your form'last TuesdaySearch quarantineby name or addressFound itwith why it was heldDeliver it nowand mark it realThe filter learnsthat pattern loosensFalse positives are invisible by construction. This is the only way you find them.
Fig 2. The one path by which a false positive becomes visible. Without a searchable quarantine there is no way to discover that the filter is wrong.
  • Machine learning
  • Management
  • Analytics
  • Front-end & mobile

That last box is the reason to record it rather than just fix the individual case. A quarantined message marked real afterwards is the single most valuable training signal the system produces, because it is a confirmed false positive with the signals that caused it attached.

How long quarantine is kept

Ninety days is a reasonable default and it should be a deliberate decision rather than a forever. The messages are small, but a quarantine that is never emptied is a growing store of unsolicited content that somebody eventually has to think about under a retention policy.

Anything marked real is moved out of quarantine and kept with the genuine enquiries, so the expiry only ever removes things a person confirmed were spam or never looked at.

Next: how the thresholds get tuned.

All posts