Diagnostic LLM APIs
Prompt sits under the model's cache minimum, so nothing caches
The cache_control breakpoint has been in that request builder since March, and the Haiku route has never reported a cache read. Not a small number: zero, in every daily bucket, on a key whose Opus traffic caches beautifully off the same code path. Nothing errored, because nothing was wrong. The prefix is about fifteen hundred tokens. Opus 5 starts caching at five hundred and twelve. Haiku 4.5 does not start until four thousand and ninety six. The parameter was accepted and thrown away.
With an Admin API key: GET /v1/organizations/usage_report/messages?starting_at={T-30d}&bucket_width=1d&limit=31&group_by[]=model&group_by[]=api_key_id.
For each key, split its models into the ones with any cache activity and the ones where cache_creation.* and cache_read_input_tokens are both exactly zero with real uncached_input_tokens behind them. Attach each model's documented minimum cacheable length. If every silent model sits above every caching one, the prefix is bracketed: at least the highest floor that still works, and under the lowest floor that does not.
That bracket is a size, and the report it came from has no request count and no prompt in it. It is the only way to measure a prefix here without sending anything.
The minimums are 512 (Opus 5, Fable 5, Mythos 5), 1,024 (Sonnet 5, Sonnet 4.6, Sonnet 4.5, Sonnet 4, Opus 4.8, Opus 4.1, Opus 4), 2,048 (Opus 4.7, Mythos Preview, Haiku 3.5) and 4,096 (Opus 4.6, Opus 4.5, Haiku 4.5). A migration from Opus 5 to Haiku 4.5 raises the bar eightfold on a prompt nobody edited.
The problem in plain words
Prompt caching on Claude is opt-in and, below a certain prefix length, inert. You set cache_control: {"type": "ephemeral"}, the request validates, the response comes back, and the API silently declines to cache anything because the prefix did not clear the model's minimum. There is no error, no warning header and no field on the response saying the breakpoint was ignored. The only trace is an absence: two counters that stay at zero while the code that should be filling them runs thousands of times a day.
What makes it survive review is that the absence looks exactly like three other absences. A team that never switched caching on sees the same two zeros. A team whose prefix changes on every call sees zero reads. A team whose traffic is slower than the TTL sees zero reads too. Four different problems, one dashboard tile, and the tile is the same shade of nothing in all four cases.
Why it happens
The prefix cannot be measured from this report, so it has to be bracketed instead. Anthropic's messages usage report returns token sums per bucket and carries no request count at all, which means there is no honest way to divide input tokens by calls and get a prefix size. What there is instead is a natural experiment already running in your account: one API key, one request builder, several model ids. The prompt is the same for all of them and only the floor moves. Caching that works below one floor and stops above another puts the prefix between the two, and that is a number rather than a shrug.
Zero is not a small number here, it is a different state. The check requires cache_creation.ephemeral_5m_input_tokens, cache_creation.ephemeral_1h_input_tokens and cache_read_input_tokens to be exactly zero on the silent side. A model that writes a little and reads nothing is not ineligible, it is failing to match, and that is a different note with a different repair. Ineligibility means the API never even tried.
A silent model beneath a caching one disproves the whole story. If Opus 5 with a floor of 512 is silent while Haiku 4.5 with a floor of 4,096 is caching, the same prompt already cleared the higher bar and size cannot be the explanation. The bracket is only computed when the split is clean, and when it is not, the script says so and hands the reader elsewhere rather than reporting a range it cannot support.
A model with no known floor is left out, never assumed to be zero. The minimums come from a table, and a table goes stale the moment a model ships. An unrecognised id is skipped in both directions: it cannot be the low end of a bracket, where a floor of zero would invent one out of nothing, and it cannot be the high end. Being unable to judge a model is a better outcome than judging it wrongly and printing a token range with false precision.
One prefix per key is the assumption, and it is stated rather than hidden. A key that deliberately sends a short prompt to Haiku and a long one to Opus brackets nothing, because the premise of the comparison is broken. The aggregate cannot see inside a key, so the finding is strongest on a key serving one workload and the output says exactly that. The cross-check for the ambiguous case is a peer key: if another key in the organization caches on the same model, the model's floor is not what is stopping you.
The fix, as a flow
Nothing in this chain raises and nothing in it is misconfigured. The breakpoint is set, the request validates, and the API declines to cache a prefix that never cleared the model's floor. The only trace is two counters that stay at exactly zero, which is the same trace three other problems leave. What separates them is a contrast the account is already running: one key, one prompt, several floors.
How to fix it
Pull thirty days grouped by model and by key
bucket_width=1d, group_by[]=model and group_by[]=api_key_id together. Grouping by model alone folds a caching route and a silent one into a single row and destroys the contrast the whole check runs on. Thirty days is long enough that a quiet weekend cannot fake a zero.
Split each key's models into caching and silent
Caching means cache_creation.ephemeral_5m_input_tokens + cache_creation.ephemeral_1h_input_tokens + cache_read_input_tokens greater than zero. Silent means all three at exactly zero with a material uncached_input_tokens behind them. A model that barely ran is evidence of nothing and is dropped.
Attach the documented minimum to each model id
Longest-prefix match against the published table, because the ids in a usage report are dated snapshots: claude-haiku-4-5-20251001 has to resolve through claude-haiku-4-5, and claude-opus-4-5 must not be swallowed by claude-opus-4, which has a different floor. Anything unrecognised gets no floor and no vote.
Check the split is clean before believing the bracket
Every silent model must sit above every caching one. If a low-floor model is silent while a high-floor one caches, the prompt has already proved it can clear the higher bar and the finding belongs to the prefix-instability note instead.
Read the bracket, then decide which side to move
The output is a range in tokens. Either lift the cached prefix past the upper floor with genuinely stable material — full tool schemas, few-shot examples, retrieval instructions — or drop cache_control on the routes above the boundary so the code stops claiming a discount it will never get. Do not pad with filler: padding is billed at the full input rate on the write.
How to check it worked
Re-read the same window the day after the breakpoint moves. What should appear first is a non-zero cache_creation on the model that was silent; reads follow on the second day, once there is an entry to match.
python3 anthropic_cache_floor_bracket.py --days 30
# below-cache-minimum apikey_01Ab caching works up to a floor of 1024 (claude-sonnet-5) and stops at 2048 (claude-haiku-3-5), so the cached prefix is at least 1024 tokens and under 2048. cache_control is being accepted and ignored above the boundary.
# repair: move more genuinely stable material in front of the last cache_control breakpoint until the prefix clears 2048 tokens
# repair: or drop cache_control on the routes above the boundary so the code is honest about not caching there
# note: the bracket assumes one prefix per key.
# 6 key(s) checked, 1 finding(s)The full code
One GET and no second call, because the contrast the check needs is already inside the one response. Eight pure functions: the floor lookup, which is longest-prefix so a dated snapshot resolves and claude-opus-4-5 is not swallowed by claude-opus-4; the accumulator, which reaches into the nested cache_creation object; the regrouping into one row per model per key; the org-wide set of models that cache for anyone, which is the control for the single-model case; the three-way split into caching, silent and unjudgeable; the bracket itself, which returns nothing unless the split is clean; the repair lines, sized to the bracket; and the classifier, whose first four branches exist to give the finding away.
"""Bracket a cached prefix against the cache minimums of the models it runs on.
Read only. One GET against the Admin API, which needs an Admin API key
(sk-ant-admin...); a workspace key is rejected by every /v1/organizations/
path, and an Admin key can be provisioned read-only.
A prefix shorter than a model's minimum cacheable token count is not cached and
no error is raised for it: cache_control is accepted and ignored. The messages
usage report carries no request count, so the prefix cannot be measured
directly from it. It can be bracketed. One API key that runs several models
sends the same prefix to all of them, so if caching works on the models with a
low minimum and stops dead on the models with a high one, the prefix sits
between the two floors. That bracket is the finding, and it is a size the
report was never asked for.
The repair is printed, never performed.
"""
import argparse
import datetime as dt
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("anthropic_cache_floor_bracket")
API = "https://api.anthropic.com/v1"
VERSION = "2023-06-01"
# Published minimum cacheable prompt length per model family, in tokens. Longest
# prefix wins, so a dated id such as claude-haiku-4-5-20251001 resolves through
# claude-haiku-4-5. A model that is not in this table gets no floor and is left
# out of the verdict rather than guessed at: a wrong floor here would invent a
# bracket that does not exist.
CACHE_MINIMUMS = {
"claude-opus-5": 512,
"claude-fable-5": 512,
"claude-mythos-5": 512,
"claude-mythos-preview": 2048,
"claude-opus-4-8": 1024,
"claude-opus-4-7": 2048,
"claude-opus-4-6": 4096,
"claude-opus-4-5": 4096,
"claude-opus-4-1": 1024,
"claude-opus-4": 1024,
"claude-sonnet-5": 1024,
"claude-sonnet-4-6": 1024,
"claude-sonnet-4-5": 1024,
"claude-sonnet-4": 1024,
"claude-haiku-4-5": 4096,
"claude-haiku-3-5": 2048,
}
FINDINGS = ("below-cache-minimum",)
def _int(value):
"""Read a usage field as an int. Pure. Missing and unreadable both mean 0."""
try:
return int(value or 0)
except (TypeError, ValueError):
return 0
def cache_minimum(model):
"""The model's minimum cacheable prompt length, in tokens. Pure.
Longest prefix match, because the ids that actually appear in a usage report
are dated snapshots. None for anything unrecognised, and None has to mean
"no opinion" everywhere downstream rather than "no floor": a model quietly
treated as floor zero would land on the caching side of every bracket.
"""
name = str(model or "").strip().lower()
if not name:
return None
best = None
for family, floor in CACHE_MINIMUMS.items():
if name == family or name.startswith(family + "-"):
if best is None or len(family) > len(best[0]):
best = (family, floor)
return best[1] if best else None
def series(buckets):
"""Per (api_key_id, model), the window's token totals. Pure."""
out = {}
for bucket in buckets or []:
for result in bucket.get("results") or []:
if not isinstance(result, dict):
continue
ident = (str(result.get("api_key_id") or "unknown"),
str(result.get("model") or "unknown"))
creation = result.get("cache_creation") or {}
row = out.setdefault(ident, {"uncached": 0, "writes": 0, "reads": 0})
row["uncached"] += _int(result.get("uncached_input_tokens"))
row["writes"] += (_int(creation.get("ephemeral_5m_input_tokens"))
+ _int(creation.get("ephemeral_1h_input_tokens")))
row["reads"] += _int(result.get("cache_read_input_tokens"))
return out
def by_key(totals):
"""Regroup the series into one list of model rows per api_key_id. Pure."""
out = {}
for (key, model), row in (totals or {}).items():
out.setdefault(key, []).append({
"model": model, "floor": cache_minimum(model),
"uncached": _int(row.get("uncached")), "writes": _int(row.get("writes")),
"reads": _int(row.get("reads")),
})
for rows in out.values():
rows.sort(key=lambda r: (r["floor"] if r["floor"] is not None else 10 ** 9,
r["model"]))
return out
def models_caching_anywhere(totals):
"""Models that cache for at least one key in the org. Pure.
The cross-key control. If another key caches on the same model, that model
is not the obstacle and this key's silence is about its own prompt.
"""
return {model for (_key, model), row in (totals or {}).items()
if _int(row.get("writes")) + _int(row.get("reads")) > 0}
def split_rows(rows, min_input=100_000):
"""Sort a key's models into caching, silent and unusable. Pure.
Silent means both cache counters are exactly zero with real input behind
them. Zero is not a small number here; it is the state cache_control
produces when the API declines to honour it.
"""
caching, silent, skipped = [], [], []
for row in rows or []:
if row.get("floor") is None:
skipped.append(row)
elif _int(row.get("writes")) + _int(row.get("reads")) > 0:
caching.append(row)
elif _int(row.get("uncached")) >= min_input:
silent.append(row)
else:
skipped.append(row)
return caching, silent, skipped
def floor_bracket(caching, silent):
"""Bracket the cached prefix between two floors. Pure. None if it does not.
lo is the highest floor the key still caches at, hi the lowest floor at
which it stops. The bracket exists only when the split is clean: every
silent model sits above every caching one. A silent model beneath a caching
one is not an eligibility story at all, because the same prompt cleared the
higher bar.
"""
if not caching or not silent:
return None
lo = max(_int(r.get("floor")) for r in caching)
hi = min(_int(r.get("floor")) for r in silent)
if hi <= lo:
return None
return (lo, hi)
def handoff(state):
"""Which note owns this shape, when it is not this one. Pure."""
if state == "no-caching-anywhere":
return ("this key writes and reads nothing on any model, so there is no "
"contrast to bracket against. Read the prompt-caching-never-used "
"note: with no cache_control anywhere the floors are irrelevant.")
if state == "silent-model-under-a-caching-floor":
return ("a model with a lower floor is silent while a model with a "
"higher floor caches, so the prefix cleared the higher bar and "
"size cannot be the reason. Read the "
"cache-invalidated-by-changing-prefix note.")
if state == "peer-caches-same-model":
return ("another key in this organization caches on this same model, so "
"the model's floor is not the obstacle. Read the "
"cache-invalidated-by-changing-prefix note.")
if state == "single-silent-model":
return ("one model and no contrast, so this check cannot separate a "
"prefix under the floor from caching that was never switched "
"on. Both prompt-caching-never-used and this note remain open. "
"Route a sample of the traffic through a model with a lower "
"floor and the ambiguity resolves itself.")
return ""
def classify(rows, caching_models=(), min_input=100_000):
"""Classify one api_key_id. Pure. Returns (state, detail).
Only a key that caches under one floor and goes silent above it belongs to
this note. Everything else is handed away, most of it by name.
"""
caching, silent, skipped = split_rows(rows, min_input)
if not caching and not silent:
return ("too-little-traffic",
"%d model(s) seen, none with a known floor and enough input to "
"judge" % len(skipped or []))
if not caching:
if len(silent) == 1:
model = silent[0]["model"]
if model in set(caching_models or ()):
return ("peer-caches-same-model",
"silent on %s (floor %d) while another key caches on the "
"same model" % (model, _int(silent[0]["floor"])))
return ("single-silent-model",
"silent on %s (floor %d) and running nothing else, so there "
"is no second floor to bracket against"
% (model, _int(silent[0]["floor"])))
return ("no-caching-anywhere",
"silent on all %d model(s) with known floors: %s"
% (len(silent), ", ".join(r["model"] for r in silent)))
if not silent:
return ("caches-on-every-model",
"cache activity on all %d model(s) with known floors" % len(caching))
bracket = floor_bracket(caching, silent)
if bracket is None:
low = min(silent, key=lambda r: _int(r.get("floor")))
high = max(caching, key=lambda r: _int(r.get("floor")))
return ("silent-model-under-a-caching-floor",
"%s (floor %d) is silent while %s (floor %d) caches"
% (low["model"], _int(low["floor"]), high["model"],
_int(high["floor"])))
lo, hi = bracket
return ("below-cache-minimum",
"caching works up to a floor of %d (%s) and stops at %d (%s), so the "
"cached prefix is at least %d tokens and under %d. cache_control is "
"being accepted and ignored above the boundary."
% (lo, ", ".join(r["model"] for r in caching if _int(r["floor"]) == lo),
hi, ", ".join(r["model"] for r in silent if _int(r["floor"]) == hi),
lo, hi))
def repair_lines(bracket):
"""The two honest repairs, sized to the bracket. Pure."""
if not bracket:
return []
lo, hi = bracket
return [
"move more genuinely stable material in front of the last cache_control "
"breakpoint until the prefix clears %d tokens: full tool schemas, "
"few-shot examples, retrieval instructions." % hi,
"or drop cache_control on the routes above the boundary so the code is "
"honest about not caching there, and stop budgeting for a discount that "
"cannot arrive.",
"do not pad with filler to cross %d. Padding is billed at the full input "
"rate on the write and only pays back at high repeat volume." % hi,
"the bracket is %d to %d tokens. If that straddles a route you thought "
"was much longer, the prefix is being truncated or rebuilt somewhere "
"before the breakpoint." % (lo, hi),
]
def window_start(days):
"""Floor to the day: starting_at has to sit on a bucket boundary."""
now = dt.datetime.now(dt.timezone.utc).replace(hour=0, minute=0, second=0,
microsecond=0)
return (now - dt.timedelta(days=days)).strftime("%Y-%m-%dT%H:%M:%SZ")
def get(session, path, params=None):
r = session.get(API + path, params=params or {}, timeout=60)
if r.status_code in (401, 403):
raise SystemExit("%d from Anthropic: /v1/organizations/ needs an Admin "
"API key (sk-ant-admin...), not a workspace key"
% r.status_code)
r.raise_for_status()
return r.json()
def read_buckets(session, path, params):
"""Walk the paginated usage report."""
params = dict(params)
while True:
page = get(session, path, params)
for bucket in page.get("data") or []:
yield bucket
if not page.get("has_more") or not page.get("next_page"):
return
params["page"] = page["next_page"]
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--days", type=int, default=30,
help="days of daily buckets to read (max 90)")
ap.add_argument("--min-input", type=int, default=100_000,
help="uncached input tokens a silent model needs before its "
"silence counts as evidence")
ap.add_argument("--show-all", action="store_true",
help="also print keys that are behaving")
args = ap.parse_args()
admin = os.environ.get("ANTHROPIC_ADMIN_KEY")
if not admin:
log.error("set ANTHROPIC_ADMIN_KEY to an Admin API key "
"(sk-ant-admin...); a workspace key cannot read "
"/v1/organizations/")
return 2
days = max(2, min(int(args.days), 90))
session = requests.Session()
session.headers.update({"x-api-key": admin, "anthropic-version": VERSION})
buckets = read_buckets(session, "/organizations/usage_report/messages", {
"starting_at": window_start(days),
"bucket_width": "1d",
"limit": days + 1,
"group_by[]": ["model", "api_key_id"],
})
totals = series(buckets)
if not totals:
log.info("no messages usage in the last %d day(s)", days)
return 0
caching_models = models_caching_anywhere(totals)
keyed = by_key(totals)
checked = 0
bad = 0
for key in sorted(keyed):
rows = keyed[key]
state, detail = classify(rows, caching_models, args.min_input)
checked += 1
line = "%-32s %s %s" % (state, key, detail)
if state in FINDINGS:
bad += 1
log.warning(line)
caching, silent, _ = split_rows(rows, args.min_input)
for repair in repair_lines(floor_bracket(caching, silent)):
log.warning(" repair: %s", repair)
log.warning(" note: the bracket assumes one prefix per key. A key "
"that sends a different prompt per model brackets "
"nothing, and the report cannot see inside a key.")
else:
note = handoff(state)
if note:
log.info(line)
log.info(" %s", note)
elif args.show_all:
log.info(line)
log.info("%d key(s) checked, %d finding(s)", checked, bad)
return 1 if bad else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Bracket a cached prefix against the cache minimums of the models it runs on.
*
* Read only. One GET against the Admin API, which needs an Admin API key
* (sk-ant-admin...). A workspace key is rejected by /v1/organizations/.
*
* A prefix under a model's minimum cacheable length is silently not cached.
* The messages usage report has no request count, so the prefix cannot be
* measured from it. It can be bracketed: one key that runs several models
* sends the same prefix to all of them, so caching that works below one floor
* and stops above another puts the prefix between the two.
*/
const API = 'https://api.anthropic.com/v1';
const VERSION = '2023-06-01';
const FINDINGS = new Set(['below-cache-minimum']);
/**
* Published minimum cacheable prompt length per model family, in tokens.
* A model absent from this table gets no floor and is left out of the verdict
* rather than guessed at.
*/
export const CACHE_MINIMUMS = {
'claude-opus-5': 512,
'claude-fable-5': 512,
'claude-mythos-5': 512,
'claude-mythos-preview': 2048,
'claude-opus-4-8': 1024,
'claude-opus-4-7': 2048,
'claude-opus-4-6': 4096,
'claude-opus-4-5': 4096,
'claude-opus-4-1': 1024,
'claude-opus-4': 1024,
'claude-sonnet-5': 1024,
'claude-sonnet-4-6': 1024,
'claude-sonnet-4-5': 1024,
'claude-sonnet-4': 1024,
'claude-haiku-4-5': 4096,
'claude-haiku-3-5': 2048,
};
/** Read a usage field as an integer. Pure. Missing and unreadable both mean 0. */
export function readInt(value) {
const n = Number(value ?? 0);
return Number.isFinite(n) ? Math.trunc(n) : 0;
}
/**
* The model's minimum cacheable prompt length. Pure. Null if unrecognised.
* Longest prefix match, because usage reports carry dated snapshot ids, and
* null has to mean "no opinion" rather than "floor zero" downstream.
*/
export function cacheMinimum(model) {
const name = String(model ?? '').trim().toLowerCase();
if (!name) return null;
let best = null;
for (const [family, floor] of Object.entries(CACHE_MINIMUMS)) {
if (name === family || name.startsWith(`${family}-`)) {
if (best === null || family.length > best[0].length) best = [family, floor];
}
}
return best ? best[1] : null;
}
/** Per api_key_id and model, the window's token totals. Pure. */
export function series(buckets) {
const out = new Map();
for (const bucket of buckets ?? []) {
for (const result of bucket?.results ?? []) {
if (!result || typeof result !== 'object') continue;
const ident = `${result.api_key_id ?? 'unknown'}\t${result.model ?? 'unknown'}`;
if (!out.has(ident)) out.set(ident, { uncached: 0, writes: 0, reads: 0 });
const row = out.get(ident);
const creation = result.cache_creation ?? {};
row.uncached += readInt(result.uncached_input_tokens);
row.writes += readInt(creation.ephemeral_5m_input_tokens)
+ readInt(creation.ephemeral_1h_input_tokens);
row.reads += readInt(result.cache_read_input_tokens);
}
}
return out;
}
/** Regroup the series into one list of model rows per api_key_id. Pure. */
export function byKey(totals) {
const out = new Map();
for (const [ident, row] of totals ?? new Map()) {
const [key, model] = String(ident).split('\t');
if (!out.has(key)) out.set(key, []);
out.get(key).push({
model,
floor: cacheMinimum(model),
uncached: readInt(row?.uncached),
writes: readInt(row?.writes),
reads: readInt(row?.reads),
});
}
for (const rows of out.values()) {
rows.sort((a, b) => (a.floor ?? 1e9) - (b.floor ?? 1e9)
|| a.model.localeCompare(b.model));
}
return out;
}
/** Models that cache for at least one key in the org. Pure. The cross-key control. */
export function modelsCachingAnywhere(totals) {
const out = new Set();
for (const [ident, row] of totals ?? new Map()) {
if (readInt(row?.writes) + readInt(row?.reads) > 0) {
out.add(String(ident).split('\t')[1]);
}
}
return out;
}
/** Sort a key's models into caching, silent and unusable. Pure. */
export function splitRows(rows, minInput = 100000) {
const caching = [];
const silent = [];
const skipped = [];
for (const row of rows ?? []) {
if (row?.floor === null || row?.floor === undefined) skipped.push(row);
else if (readInt(row?.writes) + readInt(row?.reads) > 0) caching.push(row);
else if (readInt(row?.uncached) >= minInput) silent.push(row);
else skipped.push(row);
}
return { caching, silent, skipped };
}
/**
* Bracket the cached prefix between two floors. Pure. Null if it does not.
* The bracket exists only when the split is clean: every silent model above
* every caching one. A silent model beneath a caching one means the prompt
* already cleared the higher bar, so size is not the story.
*/
export function floorBracket(caching, silent) {
if (!caching?.length || !silent?.length) return null;
const lo = Math.max(...caching.map((r) => readInt(r?.floor)));
const hi = Math.min(...silent.map((r) => readInt(r?.floor)));
if (hi <= lo) return null;
return [lo, hi];
}
/** Which note owns this shape, when it is not this one. Pure. */
export function handoff(state) {
if (state === 'no-caching-anywhere') {
return 'this key writes and reads nothing on any model, so there is no '
+ 'contrast to bracket against. Read the prompt-caching-never-used note: '
+ 'with no cache_control anywhere the floors are irrelevant.';
}
if (state === 'silent-model-under-a-caching-floor') {
return 'a model with a lower floor is silent while a model with a higher '
+ 'floor caches, so the prefix cleared the higher bar and size cannot be '
+ 'the reason. Read the cache-invalidated-by-changing-prefix note.';
}
if (state === 'peer-caches-same-model') {
return 'another key in this organization caches on this same model, so the '
+ "model's floor is not the obstacle. Read the "
+ 'cache-invalidated-by-changing-prefix note.';
}
if (state === 'single-silent-model') {
return 'one model and no contrast, so this check cannot separate a prefix '
+ 'under the floor from caching that was never switched on. Both '
+ 'prompt-caching-never-used and this note remain open. Route a sample of '
+ 'the traffic through a model with a lower floor and the ambiguity '
+ 'resolves itself.';
}
return '';
}
/** Classify one api_key_id. Pure. Returns [state, detail]. */
export function classify(rows, cachingModels = new Set(), minInput = 100000) {
const { caching, silent, skipped } = splitRows(rows, minInput);
if (!caching.length && !silent.length) {
return ['too-little-traffic',
`${skipped.length} model(s) seen, none with a known floor and enough input to judge`];
}
if (!caching.length) {
if (silent.length === 1) {
const row = silent[0];
if (cachingModels?.has?.(row.model)) {
return ['peer-caches-same-model',
`silent on ${row.model} (floor ${readInt(row.floor)}) while another `
+ 'key caches on the same model'];
}
return ['single-silent-model',
`silent on ${row.model} (floor ${readInt(row.floor)}) and running `
+ 'nothing else, so there is no second floor to bracket against'];
}
return ['no-caching-anywhere',
`silent on all ${silent.length} model(s) with known floors: `
+ silent.map((r) => r.model).join(', ')];
}
if (!silent.length) {
return ['caches-on-every-model',
`cache activity on all ${caching.length} model(s) with known floors`];
}
const bracket = floorBracket(caching, silent);
if (bracket === null) {
const low = silent.reduce((a, b) => (readInt(a.floor) <= readInt(b.floor) ? a : b));
const high = caching.reduce((a, b) => (readInt(a.floor) >= readInt(b.floor) ? a : b));
return ['silent-model-under-a-caching-floor',
`${low.model} (floor ${readInt(low.floor)}) is silent while ${high.model} `
+ `(floor ${readInt(high.floor)}) caches`];
}
const [lo, hi] = bracket;
const loNames = caching.filter((r) => readInt(r.floor) === lo).map((r) => r.model).join(', ');
const hiNames = silent.filter((r) => readInt(r.floor) === hi).map((r) => r.model).join(', ');
return ['below-cache-minimum',
`caching works up to a floor of ${lo} (${loNames}) and stops at ${hi} `
+ `(${hiNames}), so the cached prefix is at least ${lo} tokens and under `
+ `${hi}. cache_control is being accepted and ignored above the boundary.`];
}
/** The two honest repairs, sized to the bracket. Pure. */
export function repairLines(bracket) {
if (!bracket) return [];
const [lo, hi] = bracket;
return [
'move more genuinely stable material in front of the last cache_control '
+ `breakpoint until the prefix clears ${hi} tokens: full tool schemas, `
+ 'few-shot examples, retrieval instructions.',
'or drop cache_control on the routes above the boundary so the code is '
+ 'honest about not caching there, and stop budgeting for a discount that '
+ 'cannot arrive.',
`do not pad with filler to cross ${hi}. Padding is billed at the full input `
+ 'rate on the write and only pays back at high repeat volume.',
`the bracket is ${lo} to ${hi} tokens. If that straddles a route you thought `
+ 'was much longer, the prefix is being truncated or rebuilt somewhere '
+ 'before the breakpoint.',
];
}
function windowStart(days) {
const now = new Date();
now.setUTCHours(0, 0, 0, 0);
return `${new Date(now.getTime() - days * 86400000).toISOString().slice(0, 19)}Z`;
}
async function get(key, path, params) {
const url = new URL(API + path);
for (const [k, v] of Object.entries(params ?? {})) {
if (Array.isArray(v)) v.forEach((item) => url.searchParams.append(k, item));
else url.searchParams.set(k, String(v));
}
const res = await fetch(url, {
headers: { 'x-api-key': key, 'anthropic-version': VERSION },
});
if (res.status === 401 || res.status === 403) {
throw new Error(`${res.status} from Anthropic: /v1/organizations/ needs an `
+ 'Admin API key (sk-ant-admin...), not a workspace key');
}
if (!res.ok) throw new Error(`${res.status} from ${path}`);
return res.json();
}
async function* readBuckets(key, path, params) {
let query = { ...params };
for (;;) {
const page = await get(key, path, query);
for (const bucket of page?.data ?? []) yield bucket;
if (!page?.has_more || !page?.next_page) return;
query = { ...params, page: page.next_page };
}
}
async function main() {
const admin = process.env.ANTHROPIC_ADMIN_KEY;
if (!admin) {
console.error('set ANTHROPIC_ADMIN_KEY to an Admin API key (sk-ant-admin...); '
+ 'a workspace key cannot read /v1/organizations/');
process.exitCode = 2;
return;
}
const days = Math.max(2, Math.min(Number(process.env.DAYS ?? 30), 90));
const minInput = Number(process.env.MIN_INPUT ?? 100000);
const showAll = process.env.SHOW_ALL === '1';
const buckets = [];
for await (const bucket of readBuckets(admin, '/organizations/usage_report/messages', {
starting_at: windowStart(days),
bucket_width: '1d',
limit: days + 1,
'group_by[]': ['model', 'api_key_id'],
})) buckets.push(bucket);
const totals = series(buckets);
if (totals.size === 0) {
console.log(`no messages usage in the last ${days} day(s)`);
return;
}
const cachingModels = modelsCachingAnywhere(totals);
const keyed = byKey(totals);
let checked = 0;
let bad = 0;
for (const key of [...keyed.keys()].sort()) {
const rows = keyed.get(key);
const [state, detail] = classify(rows, cachingModels, minInput);
checked += 1;
const line = `${state.padEnd(32)} ${key} ${detail}`;
if (FINDINGS.has(state)) {
bad += 1;
console.warn(line);
const { caching, silent } = splitRows(rows, minInput);
for (const repair of repairLines(floorBracket(caching, silent))) {
console.warn(` repair: ${repair}`);
}
console.warn(' note: the bracket assumes one prefix per key. A key that '
+ 'sends a different prompt per model brackets nothing, and '
+ 'the report cannot see inside a key.');
} else {
const note = handoff(state);
if (note) {
console.log(line);
console.log(` ${note}`);
} else if (showAll) {
console.log(line);
}
}
}
console.log(`${checked} key(s) checked, ${bad} finding(s)`);
process.exitCode = bad ? 1 : 0;
}
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The fixture is one key running four models on one prompt: Opus 5 at 512 and Sonnet 5 at 1,024 both caching, Haiku 3.5 at 2,048 and Haiku 4.5 at 4,096 both silent. The assertion is a range, 1,024 to 2,048, which is a measurement of something the API never returns. Around it sit the cases that must not produce a range: a low-floor model silent while a high-floor one caches, which disproves the premise outright; an unrecognised model id, which would drag the low end to zero and invent a bracket of nothing to 4,096; a single silent model, where the honest answer names two open notes rather than picking one; and a dated snapshot id, because a floor lookup that misses claude-haiku-4-5-20251001 reports every real organization as unrecognisable.
from anthropic_cache_floor_bracket import (by_key, cache_minimum, classify,
floor_bracket, handoff,
models_caching_anywhere, repair_lines,
series, split_rows)
def model(name, uncached=5_000_000, writes=0, reads=0):
return {"model": name, "floor": cache_minimum(name), "uncached": uncached,
"writes": writes, "reads": reads}
# One key, one prompt, four models. Caching works at 512 and 1,024 and stops
# dead at 2,048 and 4,096. The prefix is therefore between 1,024 and 2,048.
BRACKETED = [
model("claude-opus-5", writes=2_000_000, reads=9_000_000),
model("claude-sonnet-5-20260115", writes=1_500_000, reads=7_000_000),
model("claude-haiku-3-5"),
model("claude-haiku-4-5-20251001"),
]
def test_the_bracket_is_the_finding():
# The note in one assertion: a size derived from a report with no request
# count and no prompt in it, purely from where caching stops working.
state, detail = classify(BRACKETED)
assert state == "below-cache-minimum"
assert "caching works up to a floor of 1024" in detail
assert "stops at 2048" in detail
assert "at least 1024 tokens and under 2048" in detail
caching, silent, _ = split_rows(BRACKETED)
assert floor_bracket(caching, silent) == (1024, 2048)
assert handoff(state) == ""
def test_a_dated_snapshot_resolves_to_its_family_floor():
# The ids in a usage report are dated. A floor lookup that misses them
# reports every real organization as unrecognised.
assert cache_minimum("claude-haiku-4-5-20251001") == 4096
assert cache_minimum("claude-sonnet-4-5-20250929") == 1024
assert cache_minimum("claude-opus-5") == 512
assert cache_minimum("claude-fable-5") == 512
# Longest prefix wins: opus-4 and opus-4-5 have different floors and the
# shorter family must not swallow the longer one.
assert cache_minimum("claude-opus-4-5-20251101") == 4096
assert cache_minimum("claude-opus-4-20250514") == 1024
assert cache_minimum("gpt-5.6") is None
assert cache_minimum("") is None
def test_an_unknown_floor_is_never_treated_as_zero():
# A model with no floor on the caching side would drag lo down to 0 and
# invent a bracket of 0 to 4096, which is not a finding, it is a shrug.
rows = [model("claude-future-9", writes=1_000_000, reads=5_000_000),
model("claude-haiku-4-5")]
caching, silent, skipped = split_rows(rows)
assert [r["model"] for r in skipped] == ["claude-future-9"]
assert caching == []
state, _ = classify(rows)
assert state == "single-silent-model"
def test_a_silent_model_under_a_caching_floor_is_someone_elses_note():
# opus-5 needs 512 and is silent while haiku-4-5 needs 4,096 and caches.
# The prompt cleared the higher bar, so size cannot be the explanation.
rows = [model("claude-opus-5"),
model("claude-haiku-4-5", writes=2_000_000, reads=8_000_000)]
state, detail = classify(rows)
assert state == "silent-model-under-a-caching-floor"
assert "claude-opus-5 (floor 512) is silent" in detail
assert "cache-invalidated-by-changing-prefix" in handoff(state)
caching, silent, _ = split_rows(rows)
assert floor_bracket(caching, silent) is None
def test_no_caching_at_all_is_the_never_switched_on_note():
rows = [model("claude-opus-5"), model("claude-haiku-4-5")]
state, detail = classify(rows)
assert state == "no-caching-anywhere"
assert "silent on all 2 model(s)" in detail
assert "prompt-caching-never-used" in handoff(state)
def test_one_silent_model_is_ambiguous_and_says_so():
# No contrast, no bracket. This is the case the note refuses to claim.
state, detail = classify([model("claude-haiku-4-5")])
assert state == "single-silent-model"
assert "no second floor to bracket against" in detail
note = handoff(state)
assert "prompt-caching-never-used" in note and "remain open" in note
def test_a_peer_key_caching_the_same_model_clears_the_model():
peers = {"claude-haiku-4-5"}
state, detail = classify([model("claude-haiku-4-5")], peers)
assert state == "peer-caches-same-model"
assert "another key caches on the same model" in detail
assert "cache-invalidated-by-changing-prefix" in handoff(state)
def test_a_thin_silent_model_is_not_evidence():
# Silence on a model that barely ran proves nothing, and must not be the
# hi end of a bracket.
rows = [model("claude-opus-5", writes=1_000_000, reads=4_000_000),
model("claude-haiku-4-5", uncached=900)]
caching, silent, skipped = split_rows(rows)
assert silent == [] and len(skipped) == 1
assert classify(rows)[0] == "caches-on-every-model"
def test_the_report_is_folded_into_keys_and_models():
buckets = [{"starting_at": "2026-08-%02dT00:00:00Z" % day,
"results": [
{"api_key_id": "apikey_01Ab", "model": "claude-opus-5",
"uncached_input_tokens": 1_000_000,
"cache_read_input_tokens": 4_000_000,
"cache_creation": {"ephemeral_5m_input_tokens": 500_000,
"ephemeral_1h_input_tokens": 0}},
{"api_key_id": "apikey_01Ab", "model": "claude-haiku-4-5",
"uncached_input_tokens": 3_000_000,
"cache_read_input_tokens": 0,
"cache_creation": {}},
]} for day in range(1, 6)]
totals = series(buckets)
assert totals[("apikey_01Ab", "claude-opus-5")]["reads"] == 20_000_000
assert totals[("apikey_01Ab", "claude-haiku-4-5")]["writes"] == 0
assert models_caching_anywhere(totals) == {"claude-opus-5"}
keyed = by_key(totals)
rows = keyed["apikey_01Ab"]
assert [r["floor"] for r in rows] == [512, 4096]
state, _ = classify(rows)
assert state == "below-cache-minimum"
assert any("4096 tokens" in line for line in repair_lines((512, 4096)))
def test_empty_and_unreadable_input_produce_no_verdict():
assert classify([])[0] == "too-little-traffic"
assert classify(None)[0] == "too-little-traffic"
assert series([]) == {}
assert series([{"results": [None, "nonsense"]}]) == {}
assert by_key({}) == {}
assert floor_bracket([], []) is None
assert repair_lines(None) == []
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { byKey, cacheMinimum, classify, floorBracket, handoff,
modelsCachingAnywhere, repairLines, series, splitRows }
from './anthropic-cache-floor-bracket.mjs';
const model = (name, { uncached = 5000000, writes = 0, reads = 0 } = {}) => ({
model: name, floor: cacheMinimum(name), uncached, writes, reads,
});
const BRACKETED = [
model('claude-opus-5', { writes: 2000000, reads: 9000000 }),
model('claude-sonnet-5-20260115', { writes: 1500000, reads: 7000000 }),
model('claude-haiku-3-5'),
model('claude-haiku-4-5-20251001'),
];
test('the bracket is the finding', () => {
const [state, detail] = classify(BRACKETED);
assert.equal(state, 'below-cache-minimum');
assert.match(detail, /caching works up to a floor of 1024/);
assert.match(detail, /stops at 2048/);
assert.match(detail, /at least 1024 tokens and under 2048/);
const { caching, silent } = splitRows(BRACKETED);
assert.deepEqual(floorBracket(caching, silent), [1024, 2048]);
assert.equal(handoff(state), '');
});
test('a dated snapshot resolves to its family floor', () => {
assert.equal(cacheMinimum('claude-haiku-4-5-20251001'), 4096);
assert.equal(cacheMinimum('claude-sonnet-4-5-20250929'), 1024);
assert.equal(cacheMinimum('claude-opus-5'), 512);
assert.equal(cacheMinimum('claude-fable-5'), 512);
assert.equal(cacheMinimum('claude-opus-4-5-20251101'), 4096);
assert.equal(cacheMinimum('claude-opus-4-20250514'), 1024);
assert.equal(cacheMinimum('gpt-5.6'), null);
assert.equal(cacheMinimum(''), null);
});
test('an unknown floor is never treated as zero', () => {
const rows = [model('claude-future-9', { writes: 1000000, reads: 5000000 }),
model('claude-haiku-4-5')];
const { caching, skipped } = splitRows(rows);
assert.deepEqual(skipped.map((r) => r.model), ['claude-future-9']);
assert.equal(caching.length, 0);
assert.equal(classify(rows)[0], 'single-silent-model');
});
test('a silent model under a caching floor is someone elses note', () => {
const rows = [model('claude-opus-5'),
model('claude-haiku-4-5', { writes: 2000000, reads: 8000000 })];
const [state, detail] = classify(rows);
assert.equal(state, 'silent-model-under-a-caching-floor');
assert.match(detail, /claude-opus-5 \(floor 512\) is silent/);
assert.match(handoff(state), /cache-invalidated-by-changing-prefix/);
const { caching, silent } = splitRows(rows);
assert.equal(floorBracket(caching, silent), null);
});
test('no caching at all is the never switched on note', () => {
const rows = [model('claude-opus-5'), model('claude-haiku-4-5')];
const [state, detail] = classify(rows);
assert.equal(state, 'no-caching-anywhere');
assert.match(detail, /silent on all 2 model\(s\)/);
assert.match(handoff(state), /prompt-caching-never-used/);
});
test('one silent model is ambiguous and says so', () => {
const [state, detail] = classify([model('claude-haiku-4-5')]);
assert.equal(state, 'single-silent-model');
assert.match(detail, /no second floor to bracket against/);
assert.match(handoff(state), /prompt-caching-never-used/);
assert.match(handoff(state), /remain open/);
});
test('a peer key caching the same model clears the model', () => {
const peers = new Set(['claude-haiku-4-5']);
const [state, detail] = classify([model('claude-haiku-4-5')], peers);
assert.equal(state, 'peer-caches-same-model');
assert.match(detail, /another key caches on the same model/);
assert.match(handoff(state), /cache-invalidated-by-changing-prefix/);
});
test('a thin silent model is not evidence', () => {
const rows = [model('claude-opus-5', { writes: 1000000, reads: 4000000 }),
model('claude-haiku-4-5', { uncached: 900 })];
const { silent, skipped } = splitRows(rows);
assert.equal(silent.length, 0);
assert.equal(skipped.length, 1);
assert.equal(classify(rows)[0], 'caches-on-every-model');
});
test('the report is folded into keys and models', () => {
const buckets = Array.from({ length: 5 }, (_, i) => ({
starting_at: `2026-08-0${i + 1}T00:00:00Z`,
results: [
{ api_key_id: 'apikey_01Ab', model: 'claude-opus-5',
uncached_input_tokens: 1000000, cache_read_input_tokens: 4000000,
cache_creation: { ephemeral_5m_input_tokens: 500000,
ephemeral_1h_input_tokens: 0 } },
{ api_key_id: 'apikey_01Ab', model: 'claude-haiku-4-5',
uncached_input_tokens: 3000000, cache_read_input_tokens: 0,
cache_creation: {} },
],
}));
const totals = series(buckets);
assert.equal(totals.get('apikey_01Ab\tclaude-opus-5').reads, 20000000);
assert.equal(totals.get('apikey_01Ab\tclaude-haiku-4-5').writes, 0);
assert.deepEqual([...modelsCachingAnywhere(totals)], ['claude-opus-5']);
const rows = byKey(totals).get('apikey_01Ab');
assert.deepEqual(rows.map((r) => r.floor), [512, 4096]);
assert.equal(classify(rows)[0], 'below-cache-minimum');
assert.ok(repairLines([512, 4096]).some((l) => l.includes('4096 tokens')));
});
test('empty and unreadable input produce no verdict', () => {
assert.equal(classify([])[0], 'too-little-traffic');
assert.equal(classify(null)[0], 'too-little-traffic');
assert.equal(series([]).size, 0);
assert.equal(series([{ results: [null, 'nonsense'] }]).size, 0);
assert.equal(byKey(new Map()).size, 0);
assert.equal(floorBracket([], []), null);
assert.deepEqual(repairLines(null), []);
});
FAQ
How is this different from prompt caching never being used?
That note reads organization-wide totals and answers a yes-or-no question: is there any cache activity anywhere. When the answer is no, the repair is to add a cache_control breakpoint. This note starts where there is cache activity and it is uneven, and asks why it stops on some models and not others. The two produce the same pair of zeros on the silent model, which is exactly why the contrast between models on one key is the only thing that separates them. With one model and no contrast, this script refuses the verdict and names both notes as still open.
Why not just count the prefix with the token counter?
Because that is a POST, and every script in this section is read-only. Anthropic's count_tokens endpoint generates nothing and bills nothing, so it is a perfectly reasonable thing for you to run by hand against a copy of your prompt, and it will give you the exact number this check can only bracket. What it will not tell you is which of your production routes are affected, which key they run on, or how much uncached input they have already burned. The bracket comes from traffic that actually happened.
The minimums are in a table. What happens when they change?
The table goes stale, so an unrecognised model id is skipped rather than guessed at. It cannot be the low end of a bracket, where an assumed floor of zero would invent a range out of nothing, and it cannot be the high end either. The failure mode of a stale table is therefore a check that declines to judge a new model, not one that prints a confident token range that happens to be wrong. When a model you use is missing, add it and the contrast comes back.
Could a multi-tenant key produce a false bracket?
Yes, and the output says so. The bracket assumes one prefix per key: the whole argument is that the same prompt met two different floors. A key that deliberately sends a short prompt to a cheap model and a long one to an expensive one satisfies the arithmetic and means nothing by it. Group by key and model, treat the finding as strong on a key with one workload, and check the request builder before acting.
Should I pad the prompt to clear the floor?
Only with material you would want cached anyway. Padding is billed at the full input rate on the write and at 1.25x on top of that for a five-minute entry, so filler pays back only at very high repeat volume and often not at all. The honest choices are to move real stable content forward — tool schemas, few-shot examples, retrieval instructions — or to accept that this route cannot be cached on this model and stop budgeting for a discount that will never arrive.
Related field notes
- Prompt caching that was never switched on at all
- A prefix that changes every call, so nothing is ever read back
- A step down in cache reads aligned with a new model id
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Prompt caching — Claude Docs
- Get messages usage report — Claude Admin API
- Models overview — Claude Docs
- Pricing — Claude Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.