Skip to content
allanninal.dev/build
Collaborate

Part 5 of 7 · Page speed watcher series ~5 min read

How the speed report reads

The report has one job: turn a number that moved into a thing somebody can change. That means the resource diff matters far more than the metric, and the deploy correlation matters more than either.

Key takeaways

  • Lead with the resource that changed, not the metric that moved.
  • Correlate to the deploy feed, and say plainly when nothing correlates.
  • Separate third-party regressions, because they are somebody else’s change.
  • Two numbers for a non-engineer: the slowest important page, and whether it is moving.
  • A monthly summary; alarms only for confirmed regressions.

What a regression message says

Four lines

  • /pricing is 1.7s slower. LCP median 3.9s, was 2.0–2.2s across the last 21 runs.
  • What changed: hero-q3.jpg is 1.4 MB. The image it replaced was 180 KB. Total page weight is up 1.2 MB.
  • When: first slow run 09:40 Wednesday. A release went out at 09:12.
  • Also: the page is now over its total-bytes budget for the first time.
  • Nine samples per run; the full resource diff is attached.

The second line is the one that gets acted on. Somebody reading “LCP median 3.9s” has to go and investigate; somebody reading “hero-q3.jpg is 1.4MB” already knows what to do and roughly how long it will take.

The resource diff

How a page speed regression is attributed to a causeA vertical chain of five steps entered by a box labelled A confirmed regression, on one page. Step one diffs the resource lists, today against the baseline, using every request stored per run. Step two asks whether anything is much bigger by absolute bytes; if so it exits to Name it, usually one thing. Step three asks whether anything is new, a request that was not there before; if so it exits to Name it, often a third party. Step four covers the case of the same bytes but slower, indicating a server or CDN change. Step five says which and correlates the deploy. A note says the fourth branch is the awkward one, where nothing got bigger and it still got slower.AWS ACCOUNTA confirmed regressionone pageDiff the resource liststoday vs baselineStored per runevery requestAnything much bigger?by absolute bytesName itusually one thingyesAnything new?a request that was not thereName itoften a third partyyesSame bytes, slower?a server or CDN changeSay whichand correlate the deployThe fourth branch is the awkward one: nothing got bigger and it still got slower.
Fig 1. How a regression gets its explanation. Three of the four outcomes name something concrete; the fourth is honest that the cause is not in the page.
  • Database
  • App integration
  • Management
  • Analytics
  • Outside AWS

When nothing got bigger

Same resources, same sizes, slower page. That is a server-side or network change: an origin that is responding more slowly, a CDN cache hit rate that dropped, a database query behind the page that got worse. The resource list cannot see any of it.

The honest report says exactly that: “page weight and request count are unchanged; the server’s response time for the document is up 800ms.” That is a different team’s problem and pointing at it precisely is more useful than a vague slowdown.

When nothing correlates

A regression with no deploy near it is worth reporting as such. It is usually a content change through a CMS, which does not appear in a deploy feed, and “no release correlates; the change is in an image, which suggests a content edit” points somebody at the right system on the first try.

Two numbers for everybody else

The monthly page speed summary in five numbersA horizontal row of five boxes. Slowest key page: the pricing page at two point one seconds. Three months ago: one point nine seconds. Direction: slightly worse. Regressions: one this month, fixed. Over budget: no pages. A note says two of these are for engineers and the first three are for everybody else.THE MONTHLY SUMMARYSlowest key page/pricing, 2.1sThree months ago1.9sDirectionslightly worseRegressions1 this month, fixedOver budget0 pagesTwo of these are for engineers. The first three are for everybody else.
Fig 2. The monthly summary. The first three boxes are the version that means something to somebody who does not work on the site.
  • Machine learning
  • Management
  • Analytics

The slowest important page and its direction over a quarter is the whole story for anybody who is not going to open a resource waterfall. It is two numbers, it is comparable across months because the profile never changes, and it answers the only question a non-engineer actually has, which is whether the site is getting better or worse.

Next: what all of this costs to run.

All posts