Skip to content
allanninal.dev/build
Collaborate

Part 4 of 7 · Cost anomaly alerter series ~5 min read

How a cost spike gets attributed

This is the step that decides whether the message gets acted on this morning or added to a list. A service name is a place to start looking. A resource name with a usage figure next to it is frequently the whole diagnosis.

Key takeaways

  • Three levels: service, resource, and the usage number that explains it.
  • Four causes account for most spikes: volume, retention, a new resource, and recursion.
  • Tags turn a resource name into an owner, which is who should be told.
  • Where attribution is impossible, list the candidates by size rather than guessing.
  • Correlate against deploys and infrastructure changes, and state it as correlation.

Service to resource to cause

How a cost spike is attributed to a specific resourceA vertical chain of five steps entered by a box labelled A service moved, Lambda up thirty-one pounds. Step one asks whether resource-level data exists for this service, using detailed daily costs per resource; if not it exits to List candidates by size and say why. Step two identifies which resource, usually one and sometimes two. Step three asks which usage metric moved, such as invocations, gigabytes or requests, read from CloudWatch for the same day. Step four asks who owns it, taken from the tags; an untagged resource exits to Untagged, which is itself a finding. Step five produces one sentence naming the resource, the number and the owner. A note says the usage metric is what turns this is expensive into this is running two hundred times normal.AWS ACCOUNTA service movedLambda, +£31Resource-level data?for this serviceDetailed costsper resource, dailyList candidatesby size, and say whynoWhich resource?usually one, sometimes twoWhich usage metric moved?invocations, GB, requestsCloudWatchthe same dayWho owns it?from the tagsUntaggeditself a findingno tagOne sentenceresource, number, ownerThe usage metric is what turns 'this is expensive' into 'this is running 200x'.
Fig 1. How a service-level movement becomes a specific diagnosis. The usage metric in step three is what makes the message a bug report rather than an observation.
  • Database
  • App integration
  • Management
  • Analytics
  • People

The four usual causes

CauseLooks likeTypical fix
VolumeUsage metric up sharply, unit cost unchangedFind what is calling it; often a retry loop or a recursion
RetentionStorage growing steadily; a ramp, not a stepA lifecycle rule or a log retention setting that was never set
New resourceA resource that did not exist last weekUsually deliberate; the question is whether it was expected to cost this
RecursionA function or queue triggering itselfAlmost always a trigger scoped too broadly; the most expensive of the four

Recursion deserves its own row because it is the one that produces genuinely alarming numbers in hours rather than days. The signature is unmistakable once you know it: invocations and writes both far outside normal, on a resource whose trigger points at somewhere the resource itself writes.

The system does not claim that diagnosis — it does not know the trigger configuration — but it can state both facts side by side, and anybody who has seen one before recognises it immediately from that.

Tags, and untagged resources

A resource name tells an engineer what to look at. A tag tells the system who to tell, which matters because the person who pays the bill and the person who can fix the resource are usually different people.

How resource tags determine who is told about a cost anomalyA horizontal row of five boxes. Resource named: image-resize. Owner tag: a team or a person. Both told: the payer and the builder. No owner tag: there is nobody to tell. Untagged spend: reported monthly as a share. A note says untagged spend as a percentage is the one FinOps metric worth tracking here.TAGS DECIDE WHO HEARS ABOUT ITResource namedimage-resizeOwner tagteam or personBoth toldpayer and builderNo owner tagnobody to tellUntagged spendreported monthly as a shareUntagged spend as a percentage is the one FinOps metric worth tracking here.
Fig 2. Why tags matter for a cost alerter specifically: they are the difference between telling somebody and telling the right somebody.
  • App integration
  • Management
  • Analytics
  • People

Reporting untagged spend as a monthly percentage is a small addition with a large effect. A business where sixty per cent of spend is untagged cannot route any cost finding to anybody, and watching that number fall is a more useful project than any individual anomaly.

When attribution is impossible

Data transfer, support charges, and a handful of services report at a level that makes resource attribution genuinely unavailable. The useful output there is a ranked list of candidates with the reason: “data transfer up £18; no resource attribution available for this line. The three largest egress sources by volume yesterday were these buckets.”

That is honest about the uncertainty and still points somebody at three things to check rather than at the whole account, which is most of the value attribution provides.

Next: why the system cannot act.

All posts