Skip to content
allanninal.dev/build
Collaborate

Part 2 of 7 · Page speed watcher series ~5 min read

How a page gets measured

Everything in this post is in service of one property: that a measurement taken today means the same thing as one taken in March. That sounds obvious and is surprisingly easy to lose, usually by accident and usually invisibly.

Key takeaways

  • One profile: a fixed CPU throttle, a fixed network shape, a fixed viewport.
  • Nine samples per page per run is a good default; the median is what is kept.
  • Cold and warm cache are different measurements. Pick one and say which.
  • Four things break comparability: the profile, the region, the browser version, and third parties.
  • Record the browser version with every run so a step change can be explained.

The profile

device      mobile, 412x915, dpr 2.6
cpu         4x slowdown        a mid-range phone, roughly
network     1.6 Mbps down, 750 Kbps up, 150ms RTT
cache       cold               every sample, a fresh profile
region      eu-west-2          fixed; latency is part of the number
samples     9                  median kept, all nine stored

None of those values is special and arguing about them is not worth much. What matters is writing them down once and never changing them, because the moment the profile changes, every historical measurement stops being comparable and the baseline has to start again.

If the profile genuinely has to change — a decision that the mid-range phone of 2026 is faster than the one you configured in 2024 — the honest handling is to start a new series rather than continuing the old one. The charts should show a break, not a smooth line through a methodology change.

How many samples

How a single page is measured across nine samplesA vertical chain of five steps entered by a box labelled One page, on the list. Step one starts a fresh browser profile with a cold cache, every time. Step two loads and measures, producing one sample. Step three asks whether nine samples have been taken, not three and not one; if not it exits to Repeat until nine. Step four takes the median and keeps all nine. Step five stores the result together with every request the page made. A note says keeping all nine is what lets you say later whether a run itself was unusual.AWS ACCOUNTOne pageon the listFresh browser profilecold cache, every timeLoad and measureone sampleNine times?not three, not oneRepeatuntil ninenoTake the medianand keep all nineStore with the resourcesevery request the page madeKeeping all nine is what lets you say later whether a run itself was unusual.
Fig 1. How one page is measured. Storing every sample rather than just the median is what makes it possible to distinguish a slow site from a bad run afterwards.
  • App integration
  • Management
  • Analytics
  • Front-end & mobile

Why nine

Three samples gives a median that still moves around a lot. Five is noticeably better. Nine is where the median becomes stable enough that day-to-day movement is mostly the site rather than the measurement, and beyond about eleven the improvement is small relative to the extra minutes.

Keeping all nine rather than only the median matters more than it looks. When a run looks odd, the spread within that run answers whether the site was slow or one sample was pathological — and a run where the spread itself is unusually wide is frequently a more interesting signal than the median.

Four ways comparability breaks

Four things that change a page speed number without the site changingA horizontal row of five boxes. Profile changed: obvious, and rare. Region changed: latency moved. Browser updated: producing a step in the chart. A third party: their change becomes your number. Record all four: with every run. A note says the third and fourth are the ones that produce unexplainable step changes.FOUR THINGS THAT MOVE THE NUMBER WITHOUT YOUR SITE CHANGINGProfile changedobvious, and rareRegion changedlatency movedBrowser updateda step in the chartA third partytheir change, your numberRecord all fourwith every runThe third and fourth are the ones that produce unexplainable step changes.
Fig 2. The four ways a measurement stops being comparable. Recording each of them with every run is what turns an unexplainable step in a chart into an obvious one.
  • Management
  • Front-end & mobile
  • Outside AWS

The browser version

A headless browser that auto-updates will, once every few weeks, change how it measures or how fast it renders. The result is a step in the chart on a day when nobody deployed anything, which is exactly the kind of unexplainable event that erodes trust in a monitoring system.

So the browser version is recorded with every run and pinned in the build. When it is deliberately upgraded, that is a recorded event and the report says so: “browser updated on the 14th; a step change on that date is expected.”

Third-party scripts

A page with an analytics tag, a chat widget and a font from somewhere else is measuring three other companies as well as itself. When one of them has a slow week, your number moves and your deploy log explains nothing.

The resource list is what saves you here: the diff will show that the regression is entirely in a third-party request, and the report says so rather than implying somebody shipped something. It is still worth knowing — a chat widget that adds a second to your pricing page is a business decision — but it is a different conversation from a regression you caused.

Next: how a regression is told apart from a bad day.

All posts