Diagnostic LLM APIs
most of your input tokens sit in the 200k-1M band
The agent keeps everything. That was the design: give it the whole ticket history, the whole document, every tool result it has ever produced in this session, and let it decide what matters. It worked beautifully in testing, where a session was four turns. In production a session is forty, each turn carries everything the previous thirty-nine produced, and the prefix has quietly grown to four hundred thousand tokens that get sent again from scratch every single time somebody types.
GET /v1/organizations/usage_report/messages?starting_at={T-30d}&bucket_width=1d&limit=31&group_by[]=context_window&group_by[]=model with an Admin API key. context_window comes back as "0-200k", "200k-1M" or null. Compute the 200k-1M share of uncached_input_tokens, and read cache_read_input_tokens in the same results.
Now the part that has to be said before anything else: this is not a price band. On current models the 1M window is the default, no beta header is involved, and long-context requests bill at standard rates. The widespread belief that crossing 200k triggers premium pricing is out of date — it was true of a retired 1M-context beta and is not true now.
What the band actually measures is size. A large and growing 200k-1M share means a prefix that grows every turn, and at $5 per million input tokens a 400k-token prefix is about $2 on every uncached call, before the model has said a word. The same growth degrades the answer as the window fills. Cache reads in that band are the difference between paying full rate for it and paying a tenth.
The problem in plain words
The shape of this is unusual: the money is real, the alarm people expect is imaginary, and the correction usually goes the wrong way. Someone sees the 200k-1M bucket, remembers hearing about long-context pricing, and either panics about a premium that no longer exists or — having checked and found there is no premium — concludes there is nothing to look at. Both readings miss it.
What is actually happening is that a conversation or an agent loop is resending a very large prefix on every turn. Each turn appends: the tool result, the retrieved document, the model's own last answer. Nothing removes anything, because nothing was ever written to remove anything. The prefix is a monotonic function of session length, and sessions in production are longer than sessions in testing, always.
The second cost is not on the invoice at all. Accuracy degrades as the window fills — the failure people have started calling context rot — so the same growth that is doubling the input bill is also making the answers worse. That is the version of this finding that gets a fix prioritised, and it is invisible to every cost dashboard by construction.
Why it happens
The band is a size alarm, not a price alarm. Every model with a 1M-token context window defaults to it, no beta header is required, and tokens in the 200k-1M band bill at standard rates. Reporting this finding as a pricing tier would be wrong, and it would also be the version a reader dismisses as soon as they check the pricing page.
Standard rates on an enormous number is still an enormous number. At $5 per million input tokens, 400k tokens of prefix is $2 per uncached call. A thousand calls a day is $2,000 a day of input, all of it re-sending text the model has already seen. The size is the whole finding; the rate never had to be unusual.
Cache reads change the severity, not the diagnosis. A long prefix that is cached is read back at 0.1x, which is a tenfold improvement and worth having. It does not shrink the context, so it does not touch the accuracy half of the problem at all. That is why this script grades a cached long-context workload as a note rather than a finding, and still says it out loud.
A null context_window is not the short band. Some results come back unbanded. Counting them as 0-200k deflates the long share and turns a real finding into a comfortable number, so the script keeps them separate and reports the share of banded traffic only.
The repair is to remove context, not to buy a cheaper token. Server-side compaction and context editing shrink what gets resent; a cache_control breakpoint on the stable part makes what remains cheap. In that order, because caching a prefix that should not exist is optimising the wrong thing.
The fix, as a flow
The band is a size alarm and everybody reads it as a price alarm, so the fix has to say the quiet part first: standard rates, extraordinary volume. Cache reads inside the band grade how bad it is without changing what it is, and traffic the report never banded is kept out of the share rather than counted as short.
How to fix it
Group by context_window and model
group_by[]=context_window and group_by[]=model, bucket_width=1d, limit=31, starting_at floored to midnight UTC. Grouping by model matters because one agent on a 1M-window model will otherwise be averaged against every ordinary chat request in the organization.
Keep unbanded traffic out of the denominator
The field takes "0-200k", "200k-1M" or null. The script maps null to unbanded and computes the long share against banded tokens only, then reports the unbanded volume separately. Folding nulls into the short band is the quiet way to make this finding disappear.
Read cache reads inside the same band
cache_read_input_tokens on the 200k-1M results tells you whether the big prefix is being read back from cache or reprocessed from scratch. Large volume with near-zero reads is the expensive combination and the one worth waking someone for.
Price it at a rate you pass in
The script takes the input rate per million tokens as an argument rather than shipping a price table that will be wrong in a quarter. Default it to the model you actually run. The output is a dollar figure for the uncached long-context input in the window, which is the number that makes compaction a scheduled piece of work rather than an idea.
Print compaction first, caching second
Recommend server-side compaction or context editing for the routes generating 200k-plus prefixes, then a cache_control breakpoint on whatever stable portion remains. Both are application changes, both are printed, and the order matters: caching a prefix that should have been compacted away locks in the accuracy problem at a tenth of the price.
How to check it worked
Add compaction, redeploy, and re-read the same grouping a week later. The 200k-1M share of uncached input should fall; if it holds steady while total tokens fall, the sessions got shorter rather than the context getting smaller, which is not the same fix.
python3 anthropic_long_context_audit.py --input-rate 5.0
# long-context-uncached claude-opus-5 71% of banded uncached input is 200k-1M, with 2% of that band read from cache
# 408.0M uncached token(s) in the band, about $2040.00 at $5.00 per million
# repair: compact or edit the context on the routes generating 200k+ prefixes, then cache what stays stable
# 3 model(s) checked, 1 finding(s)
The full code
One usage read, GET, against /v1/organizations/usage_report/messages, so ANTHROPIC_ADMIN_KEY has to be an Admin key. Five pure functions, and the interesting one is the smallest: band() maps a null context_window to unbanded rather than to the short band, which is the difference between this check finding something and reassuring you. The rest fold the buckets per model and band, compute the long share against banded traffic only, compute the cached share inside the band, and price the uncached remainder at a rate you supply.
"""Report Claude workloads whose input has grown into the 200k-1M band.
Read only. GET requests and nothing else: ANTHROPIC_ADMIN_KEY must be an Admin
API key (sk-ant-admin...), which can be provisioned read-only. A workspace key
is rejected by every /v1/organizations/* path.
This is a SIZE alarm and not a price alarm. On current models the 1M context
window is the default, no beta header is involved, and long-context requests
bill at standard rates. The old belief that crossing 200k triggers premium
pricing came from a retired beta and is not true now.
What the band measures is a prefix that grows every turn: expensive because it
is enormous at an ordinary rate, and inaccurate because the window fills. The
repair is compaction first and caching second, and it is printed, because both
are application changes.
"""
import argparse
import datetime as dt
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("anthropic_long_context_audit")
API = "https://api.anthropic.com/v1"
VERSION = "2023-06-01"
SHORT_BAND = "0-200k"
LONG_BAND = "200k-1M"
UNBANDED = "unbanded"
FINDINGS = ("long-context-uncached",)
def band(result):
"""Normalise the context_window value. Pure.
A null becomes "unbanded", never "0-200k". Folding unbanded traffic into the
short band deflates the long share and turns a real finding into a
comfortable number, which is the one outcome this whole check exists to
prevent.
"""
raw = str((result or {}).get("context_window") or "").strip().lower()
if raw == LONG_BAND.lower():
return LONG_BAND
if raw == SHORT_BAND.lower():
return SHORT_BAND
return UNBANDED
def fold(pages):
"""Sum input tokens into {model: {band: {uncached, cache_read}}}. Pure."""
out = {}
for page in pages:
for bucket in page.get("data") or []:
for result in bucket.get("results") or []:
model = str(result.get("model") or "all models")
where = band(result)
row = out.setdefault(model, {}).setdefault(
where, {"uncached": 0, "cache_read": 0})
for field, key in (("uncached_input_tokens", "uncached"),
("cache_read_input_tokens", "cache_read")):
try:
row[key] += int(result.get(field) or 0)
except (TypeError, ValueError):
pass
return out
def long_share(model_rows):
"""Share of BANDED uncached input sitting in the 200k-1M band. Pure.
Banded only. Unbanded traffic cannot be placed on either side, and putting
it in the denominator would make a workload look shorter than it is purely
because the report declined to classify some of it.
"""
rows = model_rows or {}
short = int((rows.get(SHORT_BAND) or {}).get("uncached") or 0)
long_ = int((rows.get(LONG_BAND) or {}).get("uncached") or 0)
banded = short + long_
if banded <= 0:
return 0.0
return long_ / float(banded)
def cached_share(row):
"""Share of a band's input that was read back from cache. Pure.
Grades severity, not diagnosis: a cached long prefix costs a tenth as much
and is exactly as long, so it fixes the money and none of the accuracy.
"""
data = row or {}
reads = int(data.get("cache_read") or 0)
uncached = int(data.get("uncached") or 0)
total = reads + uncached
if total <= 0:
return 0.0
return reads / float(total)
def uncached_cost(tokens, rate_per_mtok):
"""Dollars for a number of uncached input tokens. Pure.
The rate is passed in rather than baked into a table. A price table in an
audit script is a fact with an expiry date on it, and nothing warns you the
day it passes.
"""
if rate_per_mtok < 0:
raise ValueError("rate_per_mtok must not be negative")
return max(0, int(tokens or 0)) / 1e6 * float(rate_per_mtok)
def verdict(model_rows, min_tokens=10_000_000, long_threshold=0.25,
cache_floor=0.30):
"""Classify one model's context profile. Pure. Returns (state, detail)."""
rows = model_rows or {}
banded = sum(int((rows.get(b) or {}).get("uncached") or 0)
for b in (SHORT_BAND, LONG_BAND))
unbanded = int((rows.get(UNBANDED) or {}).get("uncached") or 0)
total = banded + unbanded
if total < min_tokens:
return ("low-volume",
"%d uncached input token(s) in the window, too few to conclude "
"anything" % total)
if banded <= 0:
return ("unbanded-only",
"%.1fM uncached input token(s) with no context_window on any "
"result, so this traffic cannot be placed in a band at all"
% (unbanded / 1e6))
share = long_share(rows)
long_row = rows.get(LONG_BAND) or {}
cached = cached_share(long_row)
shape = ("%.0f%% of banded uncached input is %s, with %.0f%% of that band "
"read from cache" % (share * 100, LONG_BAND, cached * 100))
if share < long_threshold:
return ("short-context",
"%s. The prefix is not where the money is going here." % shape)
if cached >= cache_floor:
return ("long-context-cached",
"%s. The big prefix is being read back rather than reprocessed, "
"so it costs a tenth of full rate. It is still just as long, "
"and length is what degrades the answer." % shape)
return ("long-context-uncached",
"%s. A very large prefix reprocessed from scratch on every call. "
"Standard rates, extraordinary volume." % shape)
def get(session, path, params):
r = session.get(API + path, params=params, timeout=60)
if r.status_code in (401, 403):
raise SystemExit("%d from Anthropic: /v1/organizations/* needs an Admin "
"API key (sk-ant-admin...), not a workspace key"
% r.status_code)
r.raise_for_status()
return r.json()
def pages(session, path, params):
"""Walk the paginated usage report."""
params = dict(params)
while True:
page = get(session, path, params)
yield page
if not page.get("has_more") or not page.get("next_page"):
return
params["page"] = page["next_page"]
def window_start(days):
"""Floor to midnight UTC: starting_at must sit on a bucket boundary."""
now = dt.datetime.now(dt.timezone.utc)
midnight = now.replace(hour=0, minute=0, second=0, microsecond=0)
return (midnight - dt.timedelta(days=days)).strftime("%Y-%m-%dT%H:%M:%SZ")
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--days", type=int, default=30,
help="days of daily buckets to read (default 30)")
ap.add_argument("--input-rate", type=float, default=5.0,
help="dollars per million uncached input tokens, for the "
"printed estimate only (default 5.0)")
ap.add_argument("--min-tokens", type=int, default=10_000_000,
help="uncached input tokens below which no claim is made")
args = ap.parse_args()
admin = os.environ.get("ANTHROPIC_ADMIN_KEY")
if not admin:
log.error("set ANTHROPIC_ADMIN_KEY to an Admin API key (sk-ant-admin...); "
"a workspace key cannot read /v1/organizations/*")
return 2
s = requests.Session()
s.headers.update({"x-api-key": admin, "anthropic-version": VERSION})
rows = fold(pages(s, "/organizations/usage_report/messages",
{"starting_at": window_start(args.days),
"bucket_width": "1d", "limit": min(args.days + 1, 31),
"group_by[]": ["context_window", "model"]}))
checked = 0
bad = 0
for model in sorted(rows, key=lambda m: -((rows[m].get(LONG_BAND) or {})
.get("uncached") or 0)):
state, detail = verdict(rows[model], args.min_tokens)
checked += 1
line = "%-22s %-22s %s" % (state, model, detail)
if state == "long-context-cached":
log.warning(line)
log.warning(" note: caching fixed the price and not the length. "
"Compaction is still the lever for answer quality.")
continue
if state not in FINDINGS:
log.info(line)
continue
bad += 1
log.warning(line)
tokens = (rows[model].get(LONG_BAND) or {}).get("uncached") or 0
log.warning(" %.1fM uncached token(s) in the band, about $%.2f at "
"$%.2f per million", tokens / 1e6,
uncached_cost(tokens, args.input_rate), args.input_rate)
log.warning(" repair: compact or edit the context on the routes "
"generating 200k+ prefixes, then put a cache_control "
"breakpoint on whatever stays stable. In that order.")
log.warning(" note: this band is not a premium price tier. It is "
"standard rates on a very large number of tokens.")
unbanded = sum((rows[m].get(UNBANDED) or {}).get("uncached") or 0 for m in rows)
if unbanded:
log.info("%.1fM uncached token(s) carried no context_window and were "
"excluded from every share above", unbanded / 1e6)
log.info("%d model(s) checked, %d finding(s)", checked, bad)
return 1 if bad else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Report Claude workloads whose input has grown into the 200k-1M band.
*
* Read only. GET requests and nothing else against the Admin API, which needs
* an Admin API key (sk-ant-admin...); a workspace key is rejected by every
* /v1/organizations/* path.
*
* A SIZE alarm and not a price alarm. On current models the 1M window is the
* default, no beta header is involved, and long-context requests bill at
* standard rates. What the band measures is a prefix that grows every turn.
* The repair is compaction first, caching second, and it is printed.
*/
const API = 'https://api.anthropic.com/v1';
const VERSION = '2023-06-01';
const SHORT_BAND = '0-200k';
const LONG_BAND = '200k-1M';
const UNBANDED = 'unbanded';
const FINDINGS = ['long-context-uncached'];
/**
* Normalise the context_window value. Pure.
* A null becomes "unbanded", never "0-200k": folding unbanded traffic into the
* short band deflates the long share and turns a real finding into a
* comfortable number.
*/
export function band(result) {
const raw = String(result?.context_window ?? '').trim().toLowerCase();
if (raw === LONG_BAND.toLowerCase()) return LONG_BAND;
if (raw === SHORT_BAND.toLowerCase()) return SHORT_BAND;
return UNBANDED;
}
/** Sum input tokens into {model: {band: {uncached, cache_read}}}. Pure. */
export function fold(pages) {
const out = {};
for (const page of pages ?? []) {
for (const bucket of page.data ?? []) {
for (const result of bucket.results ?? []) {
const model = String(result.model ?? 'all models');
const where = band(result);
if (!out[model]) out[model] = {};
if (!out[model][where]) out[model][where] = { uncached: 0, cache_read: 0 };
const row = out[model][where];
for (const [field, key] of [['uncached_input_tokens', 'uncached'],
['cache_read_input_tokens', 'cache_read']]) {
const n = Number(result[field] ?? 0);
if (Number.isFinite(n)) row[key] += Math.trunc(n);
}
}
}
}
return out;
}
/**
* Share of BANDED uncached input sitting in the 200k-1M band. Pure.
* Banded only: unbanded traffic cannot be placed on either side, and putting it
* in the denominator makes a workload look shorter than it is.
*/
export function longShare(modelRows) {
const rows = modelRows ?? {};
const short = Number(rows[SHORT_BAND]?.uncached ?? 0) || 0;
const long = Number(rows[LONG_BAND]?.uncached ?? 0) || 0;
const banded = short + long;
if (banded <= 0) return 0;
return long / banded;
}
/**
* Share of a band's input read back from cache. Pure. Grades severity, not
* diagnosis: a cached long prefix costs a tenth and is exactly as long.
*/
export function cachedShare(row) {
const reads = Number(row?.cache_read ?? 0) || 0;
const uncached = Number(row?.uncached ?? 0) || 0;
const total = reads + uncached;
if (total <= 0) return 0;
return reads / total;
}
/**
* Dollars for a number of uncached input tokens. Pure. The rate is passed in
* rather than baked into a table: a price table in an audit script is a fact
* with an expiry date and nothing warns you the day it passes.
*/
export function uncachedCost(tokens, ratePerMtok) {
if (ratePerMtok < 0) throw new Error('ratePerMtok must not be negative');
return Math.max(0, Math.trunc(Number(tokens ?? 0))) / 1e6 * Number(ratePerMtok);
}
/** Classify one model's context profile. Pure. Returns [state, detail]. */
export function verdict(modelRows, minTokens = 10000000, longThreshold = 0.25,
cacheFloor = 0.30) {
const rows = modelRows ?? {};
const banded = [SHORT_BAND, LONG_BAND]
.reduce((a, b) => a + (Number(rows[b]?.uncached ?? 0) || 0), 0);
const unbanded = Number(rows[UNBANDED]?.uncached ?? 0) || 0;
const total = banded + unbanded;
if (total < minTokens) {
return ['low-volume',
`${total} uncached input token(s) in the window, too few to conclude anything`];
}
if (banded <= 0) {
return ['unbanded-only',
`${(unbanded / 1e6).toFixed(1)}M uncached input token(s) with no ` +
'context_window on any result, so this traffic cannot be placed in a band at all'];
}
const share = longShare(rows);
const cached = cachedShare(rows[LONG_BAND]);
const shape = `${(share * 100).toFixed(0)}% of banded uncached input is ` +
`${LONG_BAND}, with ${(cached * 100).toFixed(0)}% of that band ` +
'read from cache';
if (share < longThreshold) {
return ['short-context',
`${shape}. The prefix is not where the money is going here.`];
}
if (cached >= cacheFloor) {
return ['long-context-cached',
`${shape}. The big prefix is being read back rather than reprocessed, so ` +
'it costs a tenth of full rate. It is still just as long, and length is ' +
'what degrades the answer.'];
}
return ['long-context-uncached',
`${shape}. A very large prefix reprocessed from scratch on every call. ` +
'Standard rates, extraordinary volume.'];
}
async function get(key, path, params) {
const url = new URL(API + path);
for (const [k, v] of Object.entries(params)) {
if (Array.isArray(v)) for (const item of v) url.searchParams.append(k, String(item));
else if (v !== undefined && v !== null) url.searchParams.set(k, String(v));
}
const res = await fetch(url, {
headers: { 'x-api-key': key, 'anthropic-version': VERSION },
});
if (res.status === 401 || res.status === 403) {
throw new Error(`${res.status} from Anthropic: /v1/organizations/* needs an ` +
'Admin API key (sk-ant-admin...), not a workspace key');
}
if (!res.ok) throw new Error(`${res.status} from ${path}`);
return res.json();
}
async function readPages(key, path, params) {
const out = [];
let next = { ...params };
for (;;) {
const page = await get(key, path, next);
out.push(page);
if (!page.has_more || !page.next_page) return out;
next = { ...next, page: page.next_page };
}
}
/** Floor to midnight UTC: starting_at must sit on a bucket boundary. */
function windowStart(days) {
const midnight = new Date();
midnight.setUTCHours(0, 0, 0, 0);
midnight.setUTCDate(midnight.getUTCDate() - days);
return midnight.toISOString().replace(/\.\d{3}Z$/, 'Z');
}
async function main() {
const key = process.env.ANTHROPIC_ADMIN_KEY;
if (!key) {
console.error('set ANTHROPIC_ADMIN_KEY to an Admin API key (sk-ant-admin...); ' +
'a workspace key cannot read /v1/organizations/*');
process.exitCode = 2;
return;
}
const days = Number(process.env.DAYS ?? 30);
const inputRate = Number(process.env.INPUT_RATE ?? 5.0);
const minTokens = Number(process.env.MIN_TOKENS ?? 10000000);
const rows = fold(await readPages(key, '/organizations/usage_report/messages', {
starting_at: windowStart(days), bucket_width: '1d',
limit: Math.min(days + 1, 31),
'group_by[]': ['context_window', 'model'],
}));
let checked = 0;
let bad = 0;
const models = Object.keys(rows).sort(
(a, b) => (rows[b][LONG_BAND]?.uncached ?? 0) - (rows[a][LONG_BAND]?.uncached ?? 0));
for (const model of models) {
const [state, detail] = verdict(rows[model], minTokens);
checked += 1;
const line = `${state.padEnd(22)} ${model.padEnd(22)} ${detail}`;
if (state === 'long-context-cached') {
console.warn(line);
console.warn(' note: caching fixed the price and not the length. ' +
'Compaction is still the lever for answer quality.');
continue;
}
if (!FINDINGS.includes(state)) {
console.log(line);
continue;
}
bad += 1;
console.warn(line);
const tokens = rows[model][LONG_BAND]?.uncached ?? 0;
console.warn(` ${(tokens / 1e6).toFixed(1)}M uncached token(s) in the band, ` +
`about $${uncachedCost(tokens, inputRate).toFixed(2)} at ` +
`$${inputRate.toFixed(2)} per million`);
console.warn(' repair: compact or edit the context on the routes generating ' +
'200k+ prefixes, then put a cache_control breakpoint on whatever ' +
'stays stable. In that order.');
console.warn(' note: this band is not a premium price tier. It is standard ' +
'rates on a very large number of tokens.');
}
const unbanded = Object.values(rows)
.reduce((a, r) => a + (r[UNBANDED]?.uncached ?? 0), 0);
if (unbanded) {
console.log(`${(unbanded / 1e6).toFixed(1)}M uncached token(s) carried no ` +
'context_window and were excluded from every share above');
}
console.log(`${checked} model(s) checked, ${bad} finding(s)`);
process.exitCode = bad ? 1 : 0;
}
// Only run when invoked directly, so importing this module from the test file
// does not fire main() and fail on the missing key.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
Two tests carry the note. The first is that a null context_window becomes unbanded and stays out of the denominator — treat it as short traffic and a 71% long share reads as 41%, which is the difference between an investigation and a shrug. The second is that a heavily cached long-context workload is a different state from an uncached one, with a different sentence attached, because caching fixes the price and leaves the length exactly where it was.
from anthropic_long_context_audit import (band, cached_share, fold,
long_share, uncached_cost, verdict)
def result(window="200k-1M", model="claude-opus-5", uncached=400_000_000,
cache_read=0):
"""One result from the messages usage report."""
return {"context_window": window, "model": model,
"uncached_input_tokens": uncached,
"cache_read_input_tokens": cache_read}
def page(*results):
return {"data": [{"starting_at": "2026-08-01T00:00:00Z",
"results": list(results)}], "has_more": False}
def rows(long_uncached=400_000_000, long_read=0, short_uncached=160_000_000,
unbanded=0):
"""A folded model row shaped like fold() returns them."""
out = {"200k-1M": {"uncached": long_uncached, "cache_read": long_read},
"0-200k": {"uncached": short_uncached, "cache_read": 0}}
if unbanded:
out["unbanded"] = {"uncached": unbanded, "cache_read": 0}
return out
def test_a_null_context_window_is_unbanded_and_not_the_short_band():
# The load-bearing one. 400M long against 160M short is 71%. Counting a
# further 400M of unbanded traffic as short would report 41% and nothing
# would ever be looked at.
assert band({"context_window": None}) == "unbanded"
assert band({}) == "unbanded"
assert band({"context_window": "200k-1M"}) == "200k-1M"
assert band({"context_window": "0-200k"}) == "0-200k"
with_nulls = rows(unbanded=400_000_000)
assert abs(long_share(with_nulls) - 400 / 560) < 1e-9
state, detail = verdict(with_nulls)
assert state == "long-context-uncached"
assert "71% of banded uncached input" in detail
def test_a_cached_long_prefix_is_a_different_state_with_a_different_sentence():
state, detail = verdict(rows(long_uncached=40_000_000,
long_read=360_000_000,
short_uncached=10_000_000))
assert state == "long-context-cached"
assert "It is still just as long" in detail
def test_a_short_context_workload_is_not_a_finding():
assert verdict(rows(long_uncached=10_000_000,
short_uncached=400_000_000))[0] == "short-context"
assert verdict(rows(long_uncached=100, short_uncached=100))[0] == "low-volume"
def test_traffic_the_report_never_banded_is_reported_as_such():
state, detail = verdict({"unbanded": {"uncached": 400_000_000, "cache_read": 0}})
assert state == "unbanded-only"
assert "cannot be placed in a band" in detail
def test_the_cached_share_is_read_inside_the_band():
assert cached_share({"uncached": 0, "cache_read": 100}) == 1.0
assert cached_share({"uncached": 100, "cache_read": 0}) == 0.0
assert cached_share({"uncached": 50, "cache_read": 50}) == 0.5
assert cached_share({}) == 0.0
def test_the_rate_is_supplied_rather_than_baked_in():
# 408M uncached input tokens at $5 per million.
assert uncached_cost(408_000_000, 5.0) == 2040.0
assert uncached_cost(0, 5.0) == 0.0
assert uncached_cost(1_000_000, 0.0) == 0.0
def test_folding_keeps_models_and_bands_apart():
folded = fold([page(result(window="200k-1M", uncached=200_000_000),
result(window="200k-1M", uncached=200_000_000,
cache_read=5_000_000),
result(window="0-200k", uncached=160_000_000),
result(window=None, model="claude-haiku-4-5-20251001",
uncached=9_000_000))])
assert folded["claude-opus-5"]["200k-1M"]["uncached"] == 400_000_000
assert folded["claude-opus-5"]["200k-1M"]["cache_read"] == 5_000_000
assert folded["claude-opus-5"]["0-200k"]["uncached"] == 160_000_000
assert folded["claude-haiku-4-5-20251001"]["unbanded"]["uncached"] == 9_000_000
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { band, cachedShare, fold, longShare, uncachedCost, verdict }
from './anthropic-long-context-audit.mjs';
/** One result from the messages usage report. */
function result({ window = '200k-1M', model = 'claude-opus-5',
uncached = 400000000, cacheRead = 0 } = {}) {
return {
context_window: window, model,
uncached_input_tokens: uncached, cache_read_input_tokens: cacheRead,
};
}
function page(...results) {
return { data: [{ starting_at: '2026-08-01T00:00:00Z', results }], has_more: false };
}
/** A folded model row shaped like fold() returns them. */
function rows({ longUncached = 400000000, longRead = 0,
shortUncached = 160000000, unbanded = 0 } = {}) {
const out = {
'200k-1M': { uncached: longUncached, cache_read: longRead },
'0-200k': { uncached: shortUncached, cache_read: 0 },
};
if (unbanded) out.unbanded = { uncached: unbanded, cache_read: 0 };
return out;
}
test('a null context_window is unbanded and not the short band', () => {
assert.equal(band({ context_window: null }), 'unbanded');
assert.equal(band({}), 'unbanded');
assert.equal(band({ context_window: '200k-1M' }), '200k-1M');
assert.equal(band({ context_window: '0-200k' }), '0-200k');
const withNulls = rows({ unbanded: 400000000 });
assert.ok(Math.abs(longShare(withNulls) - 400 / 560) < 1e-9);
const [state, detail] = verdict(withNulls);
assert.equal(state, 'long-context-uncached');
assert.match(detail, /71% of banded uncached input/);
});
test('a cached long prefix is a different state with a different sentence', () => {
const [state, detail] = verdict(rows({ longUncached: 40000000,
longRead: 360000000,
shortUncached: 10000000 }));
assert.equal(state, 'long-context-cached');
assert.match(detail, /It is still just as long/);
});
test('a short context workload is not a finding', () => {
assert.equal(verdict(rows({ longUncached: 10000000, shortUncached: 400000000 }))[0],
'short-context');
assert.equal(verdict(rows({ longUncached: 100, shortUncached: 100 }))[0], 'low-volume');
});
test('traffic the report never banded is reported as such', () => {
const [state, detail] = verdict({ unbanded: { uncached: 400000000, cache_read: 0 } });
assert.equal(state, 'unbanded-only');
assert.match(detail, /cannot be placed in a band/);
});
test('the cached share is read inside the band', () => {
assert.equal(cachedShare({ uncached: 0, cache_read: 100 }), 1);
assert.equal(cachedShare({ uncached: 100, cache_read: 0 }), 0);
assert.equal(cachedShare({ uncached: 50, cache_read: 50 }), 0.5);
assert.equal(cachedShare({}), 0);
});
test('the rate is supplied rather than baked in', () => {
assert.equal(uncachedCost(408000000, 5.0), 2040);
assert.equal(uncachedCost(0, 5.0), 0);
assert.equal(uncachedCost(1000000, 0), 0);
});
test('folding keeps models and bands apart', () => {
const folded = fold([page(
result({ window: '200k-1M', uncached: 200000000 }),
result({ window: '200k-1M', uncached: 200000000, cacheRead: 5000000 }),
result({ window: '0-200k', uncached: 160000000 }),
result({ window: null, model: 'claude-haiku-4-5-20251001', uncached: 9000000 }),
)]);
assert.equal(folded['claude-opus-5']['200k-1M'].uncached, 400000000);
assert.equal(folded['claude-opus-5']['200k-1M'].cache_read, 5000000);
assert.equal(folded['claude-opus-5']['0-200k'].uncached, 160000000);
assert.equal(folded['claude-haiku-4-5-20251001'].unbanded.uncached, 9000000);
});
FAQ
Doesn't crossing 200k tokens trigger premium pricing?
Not on current models. Every model with a 1M-token context window defaults to it, no beta header is required, and tokens in the 200k-1M band bill at standard rates. The belief comes from a retired 1M-context beta that did carry a premium, and it is worth correcting explicitly, because both wrong readings of it are harmful: panic about a price tier that no longer exists, or relief that leads to ignoring a real and growing bill.
Then why should I care about the band at all?
Because standard rates on an enormous number is still an enormous number. At $5 per million input tokens, a 400k-token prefix is about $2 on every uncached call. A thousand calls a day is $2,000 a day, spent re-sending text the model already saw. And the second cost is not financial: accuracy degrades as the window fills, so the same growth that doubles the bill is also making the answers worse.
Is this just the prompt caching note again?
No, and the script keeps them apart deliberately. The caching notes ask whether caching is switched on and whether it earns back what it costs. This one asks how big the prefix is. A workload with excellent caching can still be growing its context every turn — it pays a tenth of the rate for it, which is a real saving, and it keeps the entire accuracy problem. Cache reads grade the severity here; they do not resolve the finding.
What does compaction actually mean in practice?
Summarising or dropping the parts of the conversation that are no longer load-bearing before the next turn is sent, rather than appending forever. Context editing does the same thing to tool results, which are usually the largest and most disposable part of an agent's history. Both are application changes, both are dull to build, and both attack the size rather than the price of it — which is why the script prints them ahead of the cache_control suggestion.
Some of my results have no context_window. What does that mean?
That the report did not band that traffic, which is a third answer rather than a small one. The script maps it to unbanded, keeps it out of the share denominator, and prints the volume separately. Counting it as 0-200k would be the easy choice and the wrong one: it deflates the long share, and a finding that quietly disappears is worse than one that never ran.
Related field notes
- A stable prefix reprocessed at full price on every call
- Cache entries written every call and never read back
- A workspace setting that multiplies every token price
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Context windows — Claude Docs
- Get messages usage report — Claude Docs
- Pricing — Claude Docs
- Prompt caching — Claude Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.