Skip to content
allanninal.dev/build
Collaborate

Part 5 of 7 · Lead source attributor series ~5 min read

How the report shows its uncertainty

The report is where all of this either becomes useful or becomes another confident chart. The difference is almost entirely presentational, which is frustrating and also means it is cheap to get right.

Key takeaways

  • Put the unattributed share in the headline, not in a footnote or a slice.
  • Never draw a pie chart. It forces the total to a hundred per cent by construction.
  • Show both rules as a range where they disagree.
  • Report volumes alongside percentages; small denominators make dramatic swings.
  • Carry the same standing caveat every time so it cannot quietly disappear.

What the headline says

The top of the report, every quarter

  • 256 enquiries this quarter.
  • 97 (38%) could not be attributed to any source. No referrer, no campaign tag, no earlier visit.
  • 159 have at least one known touch. The percentages below are shares of those 159, not of 256.
  • Rule: last click, with first click shown alongside.
  • Where the two rules disagree by more than 10 points, both numbers are shown rather than one.

Five lines, and they change how every number underneath is read. The third is the one that does the most work: making the denominator explicit stops a reader from mentally converting “paid search: 34%” into “a third of our business comes from paid search”, which is not what it says.

Why not a pie chart

Enquiries by source over three quarters with unattributed shown as a bandA stacked bar chart with three bars, one per quarter, counted in enquiries. Five series: Unattributed with no source at all in grey, Paid search in orange, Organic search in green, Referral and social in purple, and Everything else known in teal. Q1 totals two hundred and thirty-four, with unattributed at eighty-eight, paid search at seventy-one, organic search at thirty-four, referral and social at twenty-two, and everything else at nineteen. Q2 totals two hundred and forty-eight, with unattributed at ninety-four, paid search at sixty-six, organic search at forty-one, referral and social at twenty-six, and everything else at twenty-one. Q3 totals two hundred and fifty-six, with unattributed at ninety-seven, paid search at seventy-five, organic search at thirty-one, referral and social at twenty-nine, and everything else at twenty-four. A note says the grey band is the biggest one, and on a pie chart it is a slice like any other.0100200300400~234Q1 -- 234~248Q2 -- 248~256Q3 -- 256Unattributed -- no source at allPaid searchOrganic searchReferral and socialEverything else knownThe grey band is the biggest one. On a pie chart it is a slice like any other.
Fig 1. Three quarters of enquiries with the unattributed share drawn as a first-class band rather than left out of the picture. Its size relative to everything else is the single most important thing on the chart.

A pie chart is structurally unable to represent this honestly. It divides a whole, so the unattributed share becomes just another slice competing for attention, and if somebody removes it — which somebody always does, because it is not a source — the remaining slices silently inflate to fill the space.

Bars keep the magnitudes visible and keep the counts on the axis, which matters because a channel going from four enquiries to eight is a hundred per cent increase and also four enquiries.

Showing disagreement as a range

Attribution results shown as ranges where two rules disagreeA horizontal row of five boxes. Organic: fourteen per cent or thirty-one per cent. Paid search: thirty-four per cent or eleven per cent. Referral: nine per cent or nineteen per cent. Read as a range, not a point. The width is the finding. A note says a wide range means the channel's role depends on where in the path you look.WHERE THE TWO RULES DISAGREEOrganic14% or 31%Paid search34% or 11%Referral9% or 19%Read as a rangenot a pointThe widthis the findingA wide range means the channel's role depends on where in the path you look.
Fig 2. The disagreement presented as a range rather than resolved by picking a winner. Channels with wide ranges behave differently at different points in the path.
  • Compute
  • Networking
  • Analytics

The instinct on seeing two numbers is to ask which is right, and the answer is neither, which is unsatisfying and true. The productive version of the question is what the channel is doing: a channel that is high on first click and low on last click is starting conversations, and cutting it will show up somewhere else two quarters later.

Volumes next to percentages

Always both. A percentage on a base of eleven enquiries swings wildly for reasons that have nothing to do with marketing, and a quarterly report full of percentages invites people to read noise as trend.

The practical rule is to grey out or annotate any figure computed on fewer than about twenty enquiries. It stops the smallest, noisiest categories from producing the most dramatic-looking movements on the page.

The standing caveat

Printed at the bottom of every report

  • This report shows where enquiries came from as far as we can see it.
  • It cannot see: recommendations, conversations, podcasts and print, phone enquiries that never touched the site, and anyone whose browser withheld the referrer.
  • Those are real sources. Their absence from this report is a limit of the measurement, not evidence they do not work.
  • The ’how did you hear about us’ answers are reported separately and are the only view we have of them.
  • Do not cut a channel on the strength of this report alone.

It reads as boilerplate and that is exactly why it works. It stays on the page every quarter, so the day somebody proposes cutting the thing that generates all the word-of-mouth, the objection is already printed underneath the chart they are pointing at.

Next: what all of this costs to run.

All posts