Diagnostic LLM APIs
429s are retried blindly without reading which limit hit
The handler is four lines long and it has been in production for a year. Catch the 429, sleep, retry, give up after three. It works, in the sense that the process does not crash. It has also never once told anybody which of three separate limiters emptied, because the answer was in the response headers and the handler caught an exception class instead of reading a response.
Anthropic does not have a TPM limit. Every model group carries three limiters that empty independently — requests per minute, input tokens per minute, output tokens per minute — and each has its own header triple: anthropic-ratelimit-requests-limit / -remaining / -reset, and the same for -input-tokens- and -output-tokens-.
There is a fourth triple, anthropic-ratelimit-tokens-*, and it is not a fourth limiter. It reports whichever token limit is currently most restrictive. So its -limit value equals either the input ceiling or the output ceiling, and matching it back to the named triples is the platform telling you which bucket it thinks is binding.
Then read the configured numbers rather than guessing them: GET /v1/organizations/rate_limits with an Admin key returns each model_group with its {type, value} pairs for all three limiters.
The problem in plain words
A 429 from Anthropic carries a message that names the limit that was exceeded, a retry-after header saying how long to wait, and three -remaining counters showing exactly how much of each bucket was left. A handler written as except APIStatusError: sleep(1); retry() receives all of it and keeps none of it. What reaches your logs is the string "429", and from a string that says 429 there is no way back to whether you sent too many requests, too much prompt, or asked for too much generation — three problems with three unrelated fixes.
So the incident review reaches for the only lever anybody can name. Concurrency comes down, because concurrency is the knob everyone has. Sometimes that helps, when requests per minute really was the binding limiter. Often it does nothing at all, because the bucket that emptied was output tokens and fewer workers generating the same volume of text saturate it just as fast. Nobody can tell which happened, so the next incident is debugged the same way.
Why it happens
Three limiters, not one, and they are per model group. RPM, ITPM and OTPM are enforced by separate token buckets that refill continuously, and the docs are explicit that you hit whichever empties first. A model group can be saturated on output while its input bucket is barely touched. There is no single number that summarises that, which is why there is no single number in the headers.
The aggregate triple is a pointer, not a limit. anthropic-ratelimit-tokens-* reports the most restrictive token limit in effect. It is the one header that already contains the answer to "which one", and it gives it up only if you compare its -limit value against the input and output ceilings. Logging it on its own is logging a number with no referent.
The probe's headers describe the endpoint you probed. GET /v1/models counts against a limiter group, and it is not the group your inference traffic uses. The headers prove the plumbing works and show you the shape; the per-model-group ceilings have to come from GET /v1/organizations/rate_limits, which is why this script wants both credentials and says which number came from where.
Token counters are rounded. The -remaining values on the token buckets are rounded to the nearest thousand. On a large ceiling that is noise; on a small one it means a bucket reading zero may not be quite empty and one reading a thousand may be. Treat them as a shape, not as an accounting record.
Separate groups exist that nobody thinks to check. Message Batches, the Token Counting API, the Files API, agent skills and web search each carry their own limiters. A 429 from a batch submission is not the same bucket as a 429 from a message, and a handler that treats every 429 as "the model is busy" will back off inference because a file upload was throttled.
A spend cap wears the same status. An Anthropic 429 whose error.details.error_code is enforced_spend_limit_reached arrives with no retry-after at all and will not clear until the month rolls over. Identifying the limiter and identifying a billing wall are two different reads of the same status code, and both have to happen before anything sleeps.
The fix, as a flow
Four header triples arrive and one of them already contains the answer: the aggregate reports whichever token ceiling is most restrictive, so matching it back to the named ones names the bucket. The script then reads the configured limits per model group, because the probe's own headers describe the endpoint it probed.
How to fix it
Probe once, read all four triples
GET /v1/models with a workspace key and the anthropic-version header. It generates nothing and bills nothing, and it comes back with the full anthropic-ratelimit-* set. Do not manufacture a 429 to inspect one: driving a production bucket to zero to see the error is an outage you scheduled.
Match the aggregate ceiling back to a named one
Compare anthropic-ratelimit-tokens-limit against -input-tokens-limit and -output-tokens-limit. Equal to one of them means that is the tighter ceiling and the platform is telling you so. Equal to neither means a third, lower limit is in play — a workspace override, or a different limiter group than the one you probed — and that is worth knowing before anything else.
Find the emptiest bucket separately
The tightest ceiling and the emptiest bucket are different questions. A generous request limit that you have burned through will 429 you long before a tight token limit you are nowhere near. Compute remaining / limit for each named triple and report the minimum next to the aggregate's answer; when they disagree, both belong in the report.
Read the configured limits per model group
GET /v1/organizations/rate_limits with an Admin key returns every model_group and its limits[] array of {type, value}. A limiter type missing from that array is not unlimited — it inherits — so record it as unpublished rather than as absent. Reading a missing number as "no ceiling" is how a team decides it has room it does not have.
Print the header names the handler should have been logging
The output of this script is a list: the exact headers that arrived on this response and that a 429 handler should record every time. That is the repair. Changing your retry code is a deploy with an owner, so the script names the headers and stops, and it never touches max_retries or the client's backoff.
How to check it worked
Re-run after the handler starts logging the triples. Then the next 429 arrives already labelled, and this script becomes something you stop needing.
python3 anthropic_limiter_identify.py
# identified output-tokens is the emptiest named bucket at 3% remaining,
# and the aggregate ceiling mirrors output-tokens.
# claude-sonnet-5 rpm 4000 itpm 5000000 otpm 1000000
# log these on every 429: anthropic-ratelimit-output-tokens-remaining, retry-after, ...
The full code
Two GETs: one probe with ANTHROPIC_API_KEY, a workspace key, and one Admin call with ANTHROPIC_ADMIN_KEY for the configured ceilings. Anthropic has no read-only tier on the data plane, so the workspace key could send a message; this script is written not to and makes exactly one request with it. Seven pure functions: the header reader, an RFC 3339 clock, the ratio, the match that names the bucket, the minimum that finds the emptiest one, the verdict that reports when those two disagree, and the fold of the Admin response that keeps an unpublished limiter distinct from an absent one.
"""Name which Anthropic rate limiter is binding, instead of catching 429.
Read only. Two GET requests and nothing else. ANTHROPIC_API_KEY is a workspace
key used for a single probe against /v1/models, which generates no tokens and
bills nothing; ANTHROPIC_ADMIN_KEY is an Admin API key (sk-ant-admin...) used
for the configured limits, because /v1/organizations/* rejects a workspace key.
Anthropic has no read-only tier on the data plane: the same workspace key that
reads /v1/models could send a message. This script is trusted not to rather
than prevented from it, so it makes exactly one call with that credential.
Nothing here provokes a 429. Draining a production bucket to inspect the error
is an outage you scheduled on purpose.
"""
import argparse
import datetime as dt
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("anthropic_limiter_identify")
API = "https://api.anthropic.com/v1"
VERSION = "2023-06-01"
# The three limiters that empty independently. "tokens" is deliberately not in
# this tuple: it is not a fourth bucket, it is a report on whichever of the two
# token buckets is currently most restrictive.
NAMED = ("requests", "input-tokens", "output-tokens")
AGGREGATE = "tokens"
LIMITER_TYPES = ("requests_per_minute", "input_tokens_per_minute",
"output_tokens_per_minute")
FINDINGS = ("disagreement", "aggregate-unmatched", "headers-missing")
def parse_count(value):
"""Read a limit or remaining header as an integer. Pure, None if unreadable.
None and zero must stay distinct. Zero means the bucket is empty; None means
nothing told us, and reporting a stripped header as an empty limiter sends
somebody chasing a throttle that is not there.
"""
if value is None:
return None
text = str(value).strip().replace(",", "").replace("_", "")
if not text:
return None
try:
return int(text)
except ValueError:
return None
def read_triples(headers):
"""Parse the anthropic-ratelimit-* triples off one response. Pure.
Case-insensitive, because a proxy that rewrites header casing should not be
able to turn a working probe into a report of missing headers.
"""
lower = {}
for name, value in dict(headers or {}).items():
lower[str(name).strip().lower()] = value
out = {}
for name in NAMED + (AGGREGATE,):
limit_h = "anthropic-ratelimit-%s-limit" % name
remaining_h = "anthropic-ratelimit-%s-remaining" % name
reset_h = "anthropic-ratelimit-%s-reset" % name
if limit_h not in lower and remaining_h not in lower:
continue
reset = lower.get(reset_h)
out[name] = {"limit": parse_count(lower.get(limit_h)),
"remaining": parse_count(lower.get(remaining_h)),
"reset": str(reset).strip() if reset is not None else None}
return out
def seconds_until(value, now):
"""Seconds until an RFC 3339 reset stamp. Pure; the caller supplies now.
Returns None when the stamp cannot be read rather than guessing. A reset
window is a number a reader will act on, and half-parsing one is worse than
printing that it was unreadable.
"""
if value is None:
return None
text = str(value).strip()
if not text:
return None
for suffix in ("Z", "z"):
if text.endswith(suffix):
text = text[:-1] + "+00:00"
break
try:
when = dt.datetime.fromisoformat(text)
except ValueError:
return None
if when.tzinfo is None:
when = when.replace(tzinfo=dt.timezone.utc)
return (when - now).total_seconds()
def share_left(triple):
"""remaining / limit for one triple, or None. Pure."""
if not isinstance(triple, dict):
return None
limit = triple.get("limit")
remaining = triple.get("remaining")
if limit is None or remaining is None or limit <= 0:
return None
return max(0.0, min(1.0, remaining / float(limit)))
def mirrors(parsed):
"""Which named token limiter the aggregate triple is reporting. Pure.
anthropic-ratelimit-tokens-* is documented as the most restrictive token
limit currently in effect, so its ceiling equals the input ceiling or the
output ceiling. Matching it back is the platform naming the binding bucket
for you; nothing else in the response does that.
"""
aggregate = (parsed or {}).get(AGGREGATE) or {}
limit = aggregate.get("limit")
if limit is None:
return "no-aggregate"
matched = []
for name in ("input-tokens", "output-tokens"):
other = (parsed or {}).get(name) or {}
if other.get("limit") is not None and other.get("limit") == limit:
matched.append(name)
if len(matched) == 2:
return "both"
if len(matched) == 1:
return matched[0]
return "unmatched"
def emptiest(parsed):
"""The named bucket with the least left. Pure. Returns (name, share).
The aggregate is excluded on purpose: it duplicates one of the named
buckets, and letting it compete would report the same limiter twice under
two names and hide whichever one it is not mirroring.
"""
best = None
for name in NAMED:
share = share_left((parsed or {}).get(name) or {})
if share is None:
continue
if best is None or share < best[1]:
best = (name, share)
return best
def verdict(parsed):
"""Say which limiter is binding, and when the two answers disagree. Pure."""
if not parsed:
return ("headers-missing",
"no anthropic-ratelimit-* headers reached this process, so a "
"429 here would arrive with nothing to classify it by and "
"retry-after would be missing too")
scarce = emptiest(parsed)
if scarce is None:
return ("unreadable",
"the named triples arrived without a usable limit and "
"remaining pair, so there is no ratio to compare")
name, share = scarce
shape = "%s is the emptiest named bucket at %.0f%% remaining" % (name, share * 100)
mirror = mirrors(parsed)
if mirror == "no-aggregate":
return ("no-aggregate",
"%s, and the aggregate anthropic-ratelimit-tokens triple is "
"absent, so the platform's own view of the most restrictive "
"token limit is not available on this response." % shape)
if mirror == "unmatched":
return ("aggregate-unmatched",
"%s, but the aggregate token ceiling matches neither the input "
"nor the output ceiling. A third and lower limit is in effect: "
"a workspace override, or a different limiter group than the "
"one this probe touched." % shape)
if mirror == "both":
return ("identified",
"%s, and the aggregate ceiling equals both token ceilings, so "
"input and output share a number here and only the remaining "
"counters tell them apart." % shape)
if mirror == name:
return ("identified",
"%s, and the aggregate ceiling mirrors %s. The tightest ceiling "
"and the emptiest bucket are the same limiter." % (shape, mirror))
return ("disagreement",
"%s, while the aggregate ceiling mirrors %s. The tightest ceiling "
"and the emptiest bucket are different limiters, so a handler that "
"records only one of them will name the wrong cause."
% (shape, mirror))
def configured(payload):
"""Fold GET /v1/organizations/rate_limits into {model_group: {type: value}}. Pure.
A limiter type missing from a group's limits[] is not unlimited: it
inherits. It is recorded as None and printed as unpublished, never as
absent, because "no number" read as "no ceiling" is how a team convinces
itself it has headroom it was never granted.
"""
out = {}
for entry in (payload or {}).get("data") or []:
group = str(entry.get("model_group") or "").strip()
if not group:
continue
row = out.setdefault(group, dict.fromkeys(LIMITER_TYPES))
for limit in entry.get("limits") or []:
kind = str(limit.get("type") or "").strip()
if kind not in row:
continue
try:
row[kind] = int(limit.get("value"))
except (TypeError, ValueError):
row[kind] = None
return out
def log_headers(headers):
"""The header names a 429 handler should be recording. Pure.
Built from what actually arrived rather than from a hardcoded list, so the
printed repair does not tell a reader to log a header their gateway is
stripping. This list is the whole output of the script that matters.
"""
lower = set()
for name in dict(headers or {}):
lower.add(str(name).strip().lower())
wanted = set()
for name in NAMED + (AGGREGATE,):
for suffix in ("limit", "remaining", "reset"):
candidate = "anthropic-ratelimit-%s-%s" % (name, suffix)
if candidate in lower:
wanted.add(candidate)
for extra in ("retry-after", "request-id", "anthropic-organization-id"):
if extra in lower:
wanted.add(extra)
return sorted(wanted)
def probe(session):
"""One cheap real call with the workspace key. Generates nothing."""
r = session.get(API + "/models", timeout=30)
if r.status_code in (401, 403):
raise SystemExit("%d from Anthropic: ANTHROPIC_API_KEY must be a "
"workspace or project key" % r.status_code)
if r.status_code == 429:
log.warning("the probe itself was rate limited; the headers below "
"describe the bucket that rejected it")
return r.headers
r.raise_for_status()
return r.headers
def admin_limits(admin_key):
"""GET the configured per-model-group limits. Returns {} if no admin key."""
if not admin_key:
return {}
s = requests.Session()
s.headers.update({"x-api-key": admin_key, "anthropic-version": VERSION})
r = s.get(API + "/organizations/rate_limits", timeout=60)
if r.status_code in (401, 403):
log.warning("%d from the Admin API: /v1/organizations/* needs an Admin "
"key (sk-ant-admin...). Continuing on headers alone.",
r.status_code)
return {}
r.raise_for_status()
return configured(r.json())
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--show-all", action="store_true",
help="also print every triple, not only the verdict")
args = ap.parse_args()
key = os.environ.get("ANTHROPIC_API_KEY")
if not key:
log.error("set ANTHROPIC_API_KEY (a workspace key) for the probe")
return 2
session = requests.Session()
session.headers.update({"x-api-key": key, "anthropic-version": VERSION})
headers = probe(session)
parsed = read_triples(headers)
state, detail = verdict(parsed)
line = "%-20s %s" % (state, detail)
if state in FINDINGS:
log.warning(line)
else:
log.info(line)
now = dt.datetime.now(dt.timezone.utc)
if args.show_all:
for name in sorted(parsed):
triple = parsed[name]
until = seconds_until(triple.get("reset"), now)
log.info(" %-14s limit %s, remaining %s, resets %s", name,
triple.get("limit"), triple.get("remaining"),
"in %.0fs" % until if until is not None else "unreadable")
groups = admin_limits(os.environ.get("ANTHROPIC_ADMIN_KEY"))
for group in sorted(groups):
row = groups[group]
log.info(" %-24s rpm %s itpm %s otpm %s", group,
row["requests_per_minute"] if row["requests_per_minute"] is not None
else "unpublished",
row["input_tokens_per_minute"] if row["input_tokens_per_minute"] is not None
else "unpublished",
row["output_tokens_per_minute"] if row["output_tokens_per_minute"] is not None
else "unpublished")
if not groups:
log.info(" no configured limits read; set ANTHROPIC_ADMIN_KEY to name "
"the ceilings per model group rather than only the probe's")
names = log_headers(headers)
if names:
log.warning(" repair: record these on every 429 instead of catching a "
"broad status error: %s", ", ".join(names))
log.warning(" repair: branch before sleeping. No retry-after plus "
"error.details.error_code of enforced_spend_limit_reached "
"is a billing stop, not a throttle, and will not clear.")
else:
log.warning(" repair: no rate-limit headers arrived at all. Check the "
"proxy or gateway in front of api.anthropic.com and let "
"the anthropic-ratelimit-* and retry-after headers through.")
return 1 if state in FINDINGS else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Name which Anthropic rate limiter is binding, instead of catching 429.
*
* Read only. Two GET requests and nothing else. ANTHROPIC_API_KEY is a
* workspace key used for a single probe against /v1/models, which generates no
* tokens and bills nothing; ANTHROPIC_ADMIN_KEY is an Admin API key used for
* the configured limits, because /v1/organizations/* rejects a workspace key.
*
* Nothing here provokes a 429.
*/
const API = 'https://api.anthropic.com/v1';
const VERSION = '2023-06-01';
// The three limiters that empty independently. "tokens" is not a fourth bucket:
// it reports whichever of the two token buckets is most restrictive.
const NAMED = ['requests', 'input-tokens', 'output-tokens'];
const AGGREGATE = 'tokens';
const LIMITER_TYPES = ['requests_per_minute', 'input_tokens_per_minute',
'output_tokens_per_minute'];
const FINDINGS = new Set(['disagreement', 'aggregate-unmatched', 'headers-missing']);
/**
* Read a limit or remaining header as an integer. Pure, null if unreadable.
* null and 0 stay distinct: 0 is an empty bucket, null is a stripped header.
*/
export function parseCount(value) {
if (value === null || value === undefined) return null;
const text = String(value).trim().replace(/[,_]/g, '');
if (!text) return null;
const n = Number(text);
return Number.isInteger(n) ? n : null;
}
/** Parse the anthropic-ratelimit-* triples off one response. Pure. */
export function readTriples(headers) {
const lower = new Map();
const entries = typeof headers?.entries === 'function'
? [...headers.entries()] : Object.entries(headers ?? {});
for (const [name, value] of entries) lower.set(String(name).trim().toLowerCase(), value);
const out = {};
for (const name of [...NAMED, AGGREGATE]) {
const limitH = `anthropic-ratelimit-${name}-limit`;
const remainingH = `anthropic-ratelimit-${name}-remaining`;
const resetH = `anthropic-ratelimit-${name}-reset`;
if (!lower.has(limitH) && !lower.has(remainingH)) continue;
const reset = lower.get(resetH);
out[name] = {
limit: parseCount(lower.get(limitH)),
remaining: parseCount(lower.get(remainingH)),
reset: reset === null || reset === undefined ? null : String(reset).trim(),
};
}
return out;
}
/**
* Seconds until an RFC 3339 reset stamp. Pure; the caller supplies now.
* Returns null when the stamp cannot be read rather than guessing.
*/
export function secondsUntil(value, now) {
if (value === null || value === undefined) return null;
const text = String(value).trim();
if (!text) return null;
const when = Date.parse(text);
if (Number.isNaN(when)) return null;
return (when - now.getTime()) / 1000;
}
/** remaining / limit for one triple, or null. Pure. */
export function shareLeft(triple) {
if (!triple || typeof triple !== 'object') return null;
const { limit, remaining } = triple;
if (limit === null || limit === undefined) return null;
if (remaining === null || remaining === undefined) return null;
if (limit <= 0) return null;
return Math.max(0, Math.min(1, remaining / limit));
}
/**
* Which named token limiter the aggregate triple is reporting. Pure.
* The aggregate ceiling equals the input ceiling or the output ceiling, so
* matching it back is the platform naming the binding bucket for you.
*/
export function mirrors(parsed) {
const limit = (parsed ?? {})[AGGREGATE]?.limit;
if (limit === null || limit === undefined) return 'no-aggregate';
const matched = [];
for (const name of ['input-tokens', 'output-tokens']) {
const other = (parsed ?? {})[name]?.limit;
if (other !== null && other !== undefined && other === limit) matched.push(name);
}
if (matched.length === 2) return 'both';
if (matched.length === 1) return matched[0];
return 'unmatched';
}
/**
* The named bucket with the least left. Pure. Returns [name, share].
* The aggregate is excluded: it duplicates one of the named buckets.
*/
export function emptiest(parsed) {
let best = null;
for (const name of NAMED) {
const share = shareLeft((parsed ?? {})[name]);
if (share === null) continue;
if (best === null || share < best[1]) best = [name, share];
}
return best;
}
/** Say which limiter is binding, and when the two answers disagree. Pure. */
export function verdict(parsed) {
if (!parsed || Object.keys(parsed).length === 0) {
return ['headers-missing',
'no anthropic-ratelimit-* headers reached this process, so a 429 here ' +
'would arrive with nothing to classify it by and retry-after would be ' +
'missing too'];
}
const scarce = emptiest(parsed);
if (scarce === null) {
return ['unreadable',
'the named triples arrived without a usable limit and remaining pair, ' +
'so there is no ratio to compare'];
}
const [name, share] = scarce;
const shape = `${name} is the emptiest named bucket at ` +
`${(share * 100).toFixed(0)}% remaining`;
const mirror = mirrors(parsed);
if (mirror === 'no-aggregate') {
return ['no-aggregate',
`${shape}, and the aggregate anthropic-ratelimit-tokens triple is absent, ` +
"so the platform's own view of the most restrictive token limit is not " +
'available on this response.'];
}
if (mirror === 'unmatched') {
return ['aggregate-unmatched',
`${shape}, but the aggregate token ceiling matches neither the input nor ` +
'the output ceiling. A third and lower limit is in effect: a workspace ' +
'override, or a different limiter group than the one this probe touched.'];
}
if (mirror === 'both') {
return ['identified',
`${shape}, and the aggregate ceiling equals both token ceilings, so input ` +
'and output share a number here and only the remaining counters tell ' +
'them apart.'];
}
if (mirror === name) {
return ['identified',
`${shape}, and the aggregate ceiling mirrors ${mirror}. The tightest ` +
'ceiling and the emptiest bucket are the same limiter.'];
}
return ['disagreement',
`${shape}, while the aggregate ceiling mirrors ${mirror}. The tightest ` +
'ceiling and the emptiest bucket are different limiters, so a handler that ' +
'records only one of them will name the wrong cause.'];
}
/**
* Fold GET /v1/organizations/rate_limits into {model_group: {type: value}}. Pure.
* A limiter type missing from limits[] inherits rather than being unlimited, so
* it is recorded as null and printed as unpublished.
*/
export function configured(payload) {
const out = {};
for (const entry of (payload ?? {}).data ?? []) {
const group = String(entry.model_group ?? '').trim();
if (!group) continue;
if (!out[group]) {
out[group] = {};
for (const t of LIMITER_TYPES) out[group][t] = null;
}
for (const limit of entry.limits ?? []) {
const kind = String(limit.type ?? '').trim();
if (!(kind in out[group])) continue;
const value = Number(limit.value);
out[group][kind] = Number.isInteger(value) ? value : null;
}
}
return out;
}
/**
* The header names a 429 handler should be recording. Pure.
* Built from what actually arrived, so the printed repair never tells a reader
* to log a header their gateway is stripping.
*/
export function logHeaders(headers) {
const lower = new Set();
const entries = typeof headers?.entries === 'function'
? [...headers.entries()] : Object.entries(headers ?? {});
for (const [name] of entries) lower.add(String(name).trim().toLowerCase());
const wanted = new Set();
for (const name of [...NAMED, AGGREGATE]) {
for (const suffix of ['limit', 'remaining', 'reset']) {
const candidate = `anthropic-ratelimit-${name}-${suffix}`;
if (lower.has(candidate)) wanted.add(candidate);
}
}
for (const extra of ['retry-after', 'request-id', 'anthropic-organization-id']) {
if (lower.has(extra)) wanted.add(extra);
}
return [...wanted].sort();
}
/** One cheap real call with the workspace key. Generates nothing. */
async function probe(key) {
const res = await fetch(`${API}/models`, {
headers: { 'x-api-key': key, 'anthropic-version': VERSION },
});
if (res.status === 401 || res.status === 403) {
throw new Error(`${res.status} from Anthropic: ANTHROPIC_API_KEY must be a ` +
'workspace or project key');
}
if (res.status === 429) {
console.warn('the probe itself was rate limited; the headers below describe ' +
'the bucket that rejected it');
return res.headers;
}
if (!res.ok) throw new Error(`${res.status} from /v1/models`);
return res.headers;
}
async function adminLimits(adminKey) {
if (!adminKey) return {};
const res = await fetch(`${API}/organizations/rate_limits`, {
headers: { 'x-api-key': adminKey, 'anthropic-version': VERSION },
});
if (res.status === 401 || res.status === 403) {
console.warn(`${res.status} from the Admin API: /v1/organizations/* needs an ` +
'Admin key (sk-ant-admin...). Continuing on headers alone.');
return {};
}
if (!res.ok) throw new Error(`${res.status} from /v1/organizations/rate_limits`);
return configured(await res.json());
}
async function main() {
const key = process.env.ANTHROPIC_API_KEY;
if (!key) {
console.error('set ANTHROPIC_API_KEY (a workspace key) for the probe');
process.exitCode = 2;
return;
}
const showAll = process.env.SHOW_ALL === '1';
const headers = await probe(key);
const parsed = readTriples(headers);
const [state, detail] = verdict(parsed);
const line = `${state.padEnd(20)} ${detail}`;
if (FINDINGS.has(state)) console.warn(line); else console.log(line);
const now = new Date();
if (showAll) {
for (const name of Object.keys(parsed).sort()) {
const triple = parsed[name];
const until = secondsUntil(triple.reset, now);
console.log(` ${name.padEnd(14)} limit ${triple.limit}, remaining ` +
`${triple.remaining}, resets ` +
`${until === null ? 'unreadable' : `in ${until.toFixed(0)}s`}`);
}
}
const groups = await adminLimits(process.env.ANTHROPIC_ADMIN_KEY);
const names = Object.keys(groups).sort();
for (const group of names) {
const row = groups[group];
const show = (v) => (v === null ? 'unpublished' : String(v));
console.log(` ${group.padEnd(24)} rpm ${show(row.requests_per_minute)} ` +
`itpm ${show(row.input_tokens_per_minute)} ` +
`otpm ${show(row.output_tokens_per_minute)}`);
}
if (names.length === 0) {
console.log(' no configured limits read; set ANTHROPIC_ADMIN_KEY to name the ' +
"ceilings per model group rather than only the probe's");
}
const toLog = logHeaders(headers);
if (toLog.length > 0) {
console.warn(' repair: record these on every 429 instead of catching a broad ' +
`status error: ${toLog.join(', ')}`);
console.warn(' repair: branch before sleeping. No retry-after plus ' +
'error.details.error_code of enforced_spend_limit_reached is a ' +
'billing stop, not a throttle, and will not clear.');
} else {
console.warn(' repair: no rate-limit headers arrived at all. Check the proxy ' +
'or gateway in front of api.anthropic.com and let the ' +
'anthropic-ratelimit-* and retry-after headers through.');
}
process.exitCode = FINDINGS.has(state) ? 1 : 0;
}
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The first test is the whole mechanism: an aggregate ceiling of four hundred thousand against an input ceiling of five million and an output ceiling of four hundred thousand, which identifies output as the limiter without any inference at all. The second is the case that makes the note worth writing — the tightest ceiling and the emptiest bucket are different limiters, and a report that prints one of them is confidently wrong. The rest hold the honest gaps open: an aggregate matching neither ceiling is a third limit rather than a bug to smooth over, an unpublished limiter is not an unlimited one, and a response with no headers at all is a finding rather than a pass.
import datetime as dt
from anthropic_limiter_identify import (configured, emptiest, log_headers,
mirrors, parse_count, read_triples,
seconds_until, share_left, verdict)
NOW = dt.datetime(2026, 8, 30, 12, 0, 0, tzinfo=dt.timezone.utc)
def test_the_aggregate_ceiling_names_the_binding_limiter():
# Sonnet-shaped numbers: ITPM five million, OTPM four hundred thousand. The
# aggregate equals the output ceiling, which is Anthropic telling you which
# bucket is tightest without anything having to infer it.
parsed = read_triples({
"anthropic-ratelimit-requests-limit": "4000",
"anthropic-ratelimit-requests-remaining": "3600",
"anthropic-ratelimit-input-tokens-limit": "5000000",
"anthropic-ratelimit-input-tokens-remaining": "4000000",
"anthropic-ratelimit-output-tokens-limit": "400000",
"anthropic-ratelimit-output-tokens-remaining": "12000",
"anthropic-ratelimit-tokens-limit": "400000",
"anthropic-ratelimit-tokens-remaining": "12000",
})
assert mirrors(parsed) == "output-tokens"
assert emptiest(parsed) == ("output-tokens", 0.03)
state, detail = verdict(parsed)
assert state == "identified"
assert "output-tokens is the emptiest named bucket at 3% remaining" in detail
def test_the_tightest_ceiling_and_the_emptiest_bucket_can_disagree():
# The request bucket is nearly gone while the output ceiling is the lower
# number. Reporting either alone names the wrong cause.
parsed = read_triples({
"anthropic-ratelimit-requests-limit": "4000",
"anthropic-ratelimit-requests-remaining": "40",
"anthropic-ratelimit-input-tokens-limit": "5000000",
"anthropic-ratelimit-input-tokens-remaining": "4900000",
"anthropic-ratelimit-output-tokens-limit": "400000",
"anthropic-ratelimit-output-tokens-remaining": "380000",
"anthropic-ratelimit-tokens-limit": "400000",
"anthropic-ratelimit-tokens-remaining": "380000",
})
state, detail = verdict(parsed)
assert state == "disagreement"
assert "requests is the emptiest named bucket at 1% remaining" in detail
assert "mirrors output-tokens" in detail
def test_an_aggregate_matching_neither_ceiling_is_a_third_limit():
parsed = read_triples({
"anthropic-ratelimit-requests-limit": "4000",
"anthropic-ratelimit-requests-remaining": "3900",
"anthropic-ratelimit-input-tokens-limit": "5000000",
"anthropic-ratelimit-input-tokens-remaining": "4900000",
"anthropic-ratelimit-output-tokens-limit": "400000",
"anthropic-ratelimit-output-tokens-remaining": "390000",
"anthropic-ratelimit-tokens-limit": "150000",
"anthropic-ratelimit-tokens-remaining": "150000",
})
assert mirrors(parsed) == "unmatched"
assert verdict(parsed)[0] == "aggregate-unmatched"
def test_no_headers_is_a_finding_and_not_a_pass():
assert read_triples({"content-type": "application/json"}) == {}
assert read_triples(None) == {}
state, detail = verdict({})
assert state == "headers-missing"
assert "retry-after would be missing too" in detail
assert log_headers({"content-type": "application/json"}) == []
def test_the_aggregate_never_competes_to_be_the_emptiest_bucket():
# Otherwise the same limiter is reported twice under two names, and the
# bucket the aggregate is not mirroring disappears from the report.
parsed = {"requests": {"limit": 100, "remaining": 90},
"output-tokens": {"limit": 1000, "remaining": 500},
"tokens": {"limit": 1000, "remaining": 1}}
assert emptiest(parsed) == ("output-tokens", 0.5)
def test_absent_and_empty_are_different_readings():
assert parse_count(None) is None
assert parse_count("") is None
assert parse_count("0") == 0
assert parse_count("2,000,000") == 2000000
assert parse_count("lots") is None
assert share_left({"limit": 100, "remaining": None}) is None
assert share_left({"limit": 0, "remaining": 0}) is None
assert verdict({"requests": {"limit": None, "remaining": None}})[0] == "unreadable"
def test_rfc3339_resets_parse_and_unreadable_ones_stay_unreadable():
assert seconds_until("2026-08-30T12:00:30Z", NOW) == 30.0
assert seconds_until("2026-08-30T12:00:30+00:00", NOW) == 30.0
assert seconds_until("in a bit", NOW) is None
assert seconds_until("", NOW) is None
assert seconds_until(None, NOW) is None
def test_an_unpublished_limiter_is_not_an_unlimited_one():
payload = {"data": [
{"model_group": "claude-sonnet-5", "limits": [
{"type": "requests_per_minute", "value": 4000},
{"type": "input_tokens_per_minute", "value": 5000000},
{"type": "output_tokens_per_minute", "value": 1000000}]},
{"model_group": "message-batches", "limits": [
{"type": "requests_per_minute", "value": 100}]},
]}
folded = configured(payload)
assert folded["claude-sonnet-5"]["output_tokens_per_minute"] == 1000000
# Absent from limits[] means it inherits, so it must read as None and get
# printed as unpublished rather than silently becoming zero or infinity.
assert folded["message-batches"]["input_tokens_per_minute"] is None
assert folded["message-batches"]["requests_per_minute"] == 100
assert configured({}) == {}
assert configured(None) == {}
def test_the_repair_lists_only_headers_that_actually_arrived():
names = log_headers({
"Anthropic-RateLimit-Output-Tokens-Remaining": "12000",
"anthropic-ratelimit-tokens-limit": "400000",
"Retry-After": "12",
"request-id": "req_fake123",
"content-type": "application/json",
})
assert names == ["anthropic-ratelimit-output-tokens-remaining",
"anthropic-ratelimit-tokens-limit",
"request-id", "retry-after"]
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { configured, emptiest, logHeaders, mirrors, parseCount, readTriples,
secondsUntil, shareLeft, verdict }
from './anthropic-limiter-identify.mjs';
const NOW = new Date('2026-08-30T12:00:00Z');
test('the aggregate ceiling names the binding limiter', () => {
const parsed = readTriples({
'anthropic-ratelimit-requests-limit': '4000',
'anthropic-ratelimit-requests-remaining': '3600',
'anthropic-ratelimit-input-tokens-limit': '5000000',
'anthropic-ratelimit-input-tokens-remaining': '4000000',
'anthropic-ratelimit-output-tokens-limit': '400000',
'anthropic-ratelimit-output-tokens-remaining': '12000',
'anthropic-ratelimit-tokens-limit': '400000',
'anthropic-ratelimit-tokens-remaining': '12000',
});
assert.equal(mirrors(parsed), 'output-tokens');
assert.deepEqual(emptiest(parsed), ['output-tokens', 0.03]);
const [state, detail] = verdict(parsed);
assert.equal(state, 'identified');
assert.match(detail, /output-tokens is the emptiest named bucket at 3% remaining/);
});
test('the tightest ceiling and the emptiest bucket can disagree', () => {
const parsed = readTriples({
'anthropic-ratelimit-requests-limit': '4000',
'anthropic-ratelimit-requests-remaining': '40',
'anthropic-ratelimit-input-tokens-limit': '5000000',
'anthropic-ratelimit-input-tokens-remaining': '4900000',
'anthropic-ratelimit-output-tokens-limit': '400000',
'anthropic-ratelimit-output-tokens-remaining': '380000',
'anthropic-ratelimit-tokens-limit': '400000',
'anthropic-ratelimit-tokens-remaining': '380000',
});
const [state, detail] = verdict(parsed);
assert.equal(state, 'disagreement');
assert.match(detail, /requests is the emptiest named bucket at 1% remaining/);
assert.match(detail, /mirrors output-tokens/);
});
test('an aggregate matching neither ceiling is a third limit', () => {
const parsed = readTriples({
'anthropic-ratelimit-requests-limit': '4000',
'anthropic-ratelimit-requests-remaining': '3900',
'anthropic-ratelimit-input-tokens-limit': '5000000',
'anthropic-ratelimit-input-tokens-remaining': '4900000',
'anthropic-ratelimit-output-tokens-limit': '400000',
'anthropic-ratelimit-output-tokens-remaining': '390000',
'anthropic-ratelimit-tokens-limit': '150000',
'anthropic-ratelimit-tokens-remaining': '150000',
});
assert.equal(mirrors(parsed), 'unmatched');
assert.equal(verdict(parsed)[0], 'aggregate-unmatched');
});
test('no headers is a finding and not a pass', () => {
assert.deepEqual(readTriples({ 'content-type': 'application/json' }), {});
assert.deepEqual(readTriples(null), {});
const [state, detail] = verdict({});
assert.equal(state, 'headers-missing');
assert.match(detail, /retry-after would be missing too/);
assert.deepEqual(logHeaders({ 'content-type': 'application/json' }), []);
});
test('the aggregate never competes to be the emptiest bucket', () => {
const parsed = {
requests: { limit: 100, remaining: 90 },
'output-tokens': { limit: 1000, remaining: 500 },
tokens: { limit: 1000, remaining: 1 },
};
assert.deepEqual(emptiest(parsed), ['output-tokens', 0.5]);
});
test('absent and empty are different readings', () => {
assert.equal(parseCount(null), null);
assert.equal(parseCount(''), null);
assert.equal(parseCount('0'), 0);
assert.equal(parseCount('2,000,000'), 2000000);
assert.equal(parseCount('lots'), null);
assert.equal(shareLeft({ limit: 100, remaining: null }), null);
assert.equal(shareLeft({ limit: 0, remaining: 0 }), null);
assert.equal(verdict({ requests: { limit: null, remaining: null } })[0], 'unreadable');
});
test('rfc3339 resets parse and unreadable ones stay unreadable', () => {
assert.equal(secondsUntil('2026-08-30T12:00:30Z', NOW), 30);
assert.equal(secondsUntil('2026-08-30T12:00:30+00:00', NOW), 30);
assert.equal(secondsUntil('in a bit', NOW), null);
assert.equal(secondsUntil('', NOW), null);
assert.equal(secondsUntil(null, NOW), null);
});
test('an unpublished limiter is not an unlimited one', () => {
const folded = configured({ data: [
{ model_group: 'claude-sonnet-5', limits: [
{ type: 'requests_per_minute', value: 4000 },
{ type: 'input_tokens_per_minute', value: 5000000 },
{ type: 'output_tokens_per_minute', value: 1000000 }] },
{ model_group: 'message-batches', limits: [
{ type: 'requests_per_minute', value: 100 }] },
] });
assert.equal(folded['claude-sonnet-5'].output_tokens_per_minute, 1000000);
assert.equal(folded['message-batches'].input_tokens_per_minute, null);
assert.equal(folded['message-batches'].requests_per_minute, 100);
assert.deepEqual(configured({}), {});
assert.deepEqual(configured(null), {});
});
test('the repair lists only headers that actually arrived', () => {
assert.deepEqual(logHeaders({
'Anthropic-RateLimit-Output-Tokens-Remaining': '12000',
'anthropic-ratelimit-tokens-limit': '400000',
'Retry-After': '12',
'request-id': 'req_fake123',
'content-type': 'application/json',
}), ['anthropic-ratelimit-output-tokens-remaining',
'anthropic-ratelimit-tokens-limit',
'request-id', 'retry-after']);
});
FAQ
Why is there no single TPM number on Anthropic?
Because input and output are metered separately and empty at different rates. Input tokens per minute is roughly five times output tokens per minute at every tier, so a summarisation workload and a generation workload run out of completely different things. One combined number would have to be the minimum of the two, which is exactly what the aggregate anthropic-ratelimit-tokens header already is.
The aggregate matches neither ceiling. What now?
Something lower than both is in effect. The usual cause is a workspace-level override, which can be set below the organization limit and which the aggregate faithfully reports. The other cause is that you probed a different limiter group than you think: batches, token counting, files and web search each carry their own. Read GET /v1/organizations/rate_limits and compare per model group before assuming a bug.
Can the script tell me which limiter caused yesterday's 429s?
No, and nothing can. Neither provider exposes a request log, so there is no endpoint that will return the status codes or error bodies of calls you already made. That is precisely why the repair is to start recording the headers now: the only version of this question that has an answer is the one asked about a 429 you captured.
Should the script just retry until it sees a 429 and read that?
No. Deliberately draining a production token bucket to inspect the error is an outage you scheduled, and it costs your colleagues the same headroom it costs you. Every header this note reads is present on a successful response, so one call to GET /v1/models answers the question without spending anything.
Does the retry-after header make the branching unnecessary?
It makes the sleeping correct, not the diagnosis. retry-after tells you when the bucket refills; it does not tell you which bucket, so it cannot tell you whether to reduce concurrency, cache the prefix or shorten the answers. And its absence is itself a signal: a 429 with no retry-after and error.details.error_code of enforced_spend_limit_reached is a monthly spend cap that will not clear until the month does.
Related field notes
- The input limiter is full and cache reads do not count toward it
- Output per minute is the ceiling, so concurrency will not help
- A 429 that is a billing wall rather than a throttle
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Rate limits — Claude Docs
- Rate Limits API — Claude Docs
- Errors — Claude API
- Usage and Cost API — Claude Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.