Skip to content

Diagnostic LLM APIs

429s are retried blindly without reading which limit hit

The handler is four lines long and it has been in production for a year. Catch the 429, sleep, retry, give up after three. It works, in the sense that the process does not crash. It has also never once told anybody which of three separate limiters emptied, because the answer was in the response headers and the handler caught an exception class instead of reading a response.

Read-only key Python and Node.js Tests included
A store aisle filled with lots of items
Photo by Oxana Melis on Unsplash
The short answer

Anthropic does not have a TPM limit. Every model group carries three limiters that empty independently — requests per minute, input tokens per minute, output tokens per minute — and each has its own header triple: anthropic-ratelimit-requests-limit / -remaining / -reset, and the same for -input-tokens- and -output-tokens-.

There is a fourth triple, anthropic-ratelimit-tokens-*, and it is not a fourth limiter. It reports whichever token limit is currently most restrictive. So its -limit value equals either the input ceiling or the output ceiling, and matching it back to the named triples is the platform telling you which bucket it thinks is binding.

Then read the configured numbers rather than guessing them: GET /v1/organizations/rate_limits with an Admin key returns each model_group with its {type, value} pairs for all three limiters.

The problem in plain words

A 429 from Anthropic carries a message that names the limit that was exceeded, a retry-after header saying how long to wait, and three -remaining counters showing exactly how much of each bucket was left. A handler written as except APIStatusError: sleep(1); retry() receives all of it and keeps none of it. What reaches your logs is the string "429", and from a string that says 429 there is no way back to whether you sent too many requests, too much prompt, or asked for too much generation — three problems with three unrelated fixes.

So the incident review reaches for the only lever anybody can name. Concurrency comes down, because concurrency is the knob everyone has. Sometimes that helps, when requests per minute really was the binding limiter. Often it does nothing at all, because the bucket that emptied was output tokens and fewer workers generating the same volume of text saturate it just as fast. Nobody can tell which happened, so the next incident is debugged the same way.

429 comes backmessage names thelimitHandler catchesa classresponse isdiscardedLog records 429and nothing elseReview picks aleverconcurrency,alwaysOutput bucketwas fullso nothing moved
The message named the limit, the headers counted three buckets, and retry-after said how long. All of it reached the process and none of it reached the log.

Why it happens

Three limiters, not one, and they are per model group. RPM, ITPM and OTPM are enforced by separate token buckets that refill continuously, and the docs are explicit that you hit whichever empties first. A model group can be saturated on output while its input bucket is barely touched. There is no single number that summarises that, which is why there is no single number in the headers.

The aggregate triple is a pointer, not a limit. anthropic-ratelimit-tokens-* reports the most restrictive token limit in effect. It is the one header that already contains the answer to "which one", and it gives it up only if you compare its -limit value against the input and output ceilings. Logging it on its own is logging a number with no referent.

The probe's headers describe the endpoint you probed. GET /v1/models counts against a limiter group, and it is not the group your inference traffic uses. The headers prove the plumbing works and show you the shape; the per-model-group ceilings have to come from GET /v1/organizations/rate_limits, which is why this script wants both credentials and says which number came from where.

Token counters are rounded. The -remaining values on the token buckets are rounded to the nearest thousand. On a large ceiling that is noise; on a small one it means a bucket reading zero may not be quite empty and one reading a thousand may be. Treat them as a shape, not as an accounting record.

Separate groups exist that nobody thinks to check. Message Batches, the Token Counting API, the Files API, agent skills and web search each carry their own limiters. A 429 from a batch submission is not the same bucket as a 429 from a message, and a handler that treats every 429 as "the model is busy" will back off inference because a file upload was throttled.

A spend cap wears the same status. An Anthropic 429 whose error.details.error_code is enforced_spend_limit_reached arrives with no retry-after at all and will not clear until the month rolls over. Identifying the limiter and identifying a billing wall are two different reads of the same status code, and both have to happen before anything sleeps.

The fix, as a flow

Four header triples arrive and one of them already contains the answer: the aggregate reports whichever token ceiling is most restrictive, so matching it back to the named ones names the bucket. The script then reads the configured limits per model group, because the probe's own headers describe the endpoint it probed.

Aggregate ceilingagainst the named onesAggregate mirrors onethat bucket is the tightestEmptiest bucket differsrecord both, not oneAggregate matches neithera third limit is in effectNo triples arriveda 429 here is unclassifiable
The tightest ceiling and the emptiest bucket are two questions. When they disagree, both belong in the report.

How to fix it

Probe once, read all four triples

GET /v1/models with a workspace key and the anthropic-version header. It generates nothing and bills nothing, and it comes back with the full anthropic-ratelimit-* set. Do not manufacture a 429 to inspect one: driving a production bucket to zero to see the error is an outage you scheduled.

Match the aggregate ceiling back to a named one

Compare anthropic-ratelimit-tokens-limit against -input-tokens-limit and -output-tokens-limit. Equal to one of them means that is the tighter ceiling and the platform is telling you so. Equal to neither means a third, lower limit is in play — a workspace override, or a different limiter group than the one you probed — and that is worth knowing before anything else.

Find the emptiest bucket separately

The tightest ceiling and the emptiest bucket are different questions. A generous request limit that you have burned through will 429 you long before a tight token limit you are nowhere near. Compute remaining / limit for each named triple and report the minimum next to the aggregate's answer; when they disagree, both belong in the report.

Read the configured limits per model group

GET /v1/organizations/rate_limits with an Admin key returns every model_group and its limits[] array of {type, value}. A limiter type missing from that array is not unlimited — it inherits — so record it as unpublished rather than as absent. Reading a missing number as "no ceiling" is how a team decides it has room it does not have.

Print the header names the handler should have been logging

The output of this script is a list: the exact headers that arrived on this response and that a 429 handler should record every time. That is the repair. Changing your retry code is a deploy with an owner, so the script names the headers and stops, and it never touches max_retries or the client's backoff.

How to check it worked

Re-run after the handler starts logging the triples. Then the next 429 arrives already labelled, and this script becomes something you stop needing.

python3 anthropic_limiter_identify.py
# identified       output-tokens is the emptiest named bucket at 3% remaining,
#                  and the aggregate ceiling mirrors output-tokens.
# claude-sonnet-5   rpm 4000  itpm 5000000  otpm 1000000
# log these on every 429: anthropic-ratelimit-output-tokens-remaining, retry-after, ...

The full code

Two GETs: one probe with ANTHROPIC_API_KEY, a workspace key, and one Admin call with ANTHROPIC_ADMIN_KEY for the configured ceilings. Anthropic has no read-only tier on the data plane, so the workspace key could send a message; this script is written not to and makes exactly one request with it. Seven pure functions: the header reader, an RFC 3339 clock, the ratio, the match that names the bucket, the minimum that finds the emptiest one, the verdict that reports when those two disagree, and the fold of the Admin response that keeps an unpublished limiter distinct from an absent one.

Get this script on GitHub Follow @allanninal Python and Node.js, with tests. Read only, it never writes. One of 97 LLM API fixes, free and open source.
anthropic_limiter_identify.py
"""Name which Anthropic rate limiter is binding, instead of catching 429.

Read only. Two GET requests and nothing else. ANTHROPIC_API_KEY is a workspace
key used for a single probe against /v1/models, which generates no tokens and
bills nothing; ANTHROPIC_ADMIN_KEY is an Admin API key (sk-ant-admin...) used
for the configured limits, because /v1/organizations/* rejects a workspace key.

Anthropic has no read-only tier on the data plane: the same workspace key that
reads /v1/models could send a message. This script is trusted not to rather
than prevented from it, so it makes exactly one call with that credential.

Nothing here provokes a 429. Draining a production bucket to inspect the error
is an outage you scheduled on purpose.
"""
import argparse
import datetime as dt
import logging
import os
import sys

import requests

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("anthropic_limiter_identify")

API = "https://api.anthropic.com/v1"
VERSION = "2023-06-01"

# The three limiters that empty independently. "tokens" is deliberately not in
# this tuple: it is not a fourth bucket, it is a report on whichever of the two
# token buckets is currently most restrictive.
NAMED = ("requests", "input-tokens", "output-tokens")
AGGREGATE = "tokens"

LIMITER_TYPES = ("requests_per_minute", "input_tokens_per_minute",
                 "output_tokens_per_minute")

FINDINGS = ("disagreement", "aggregate-unmatched", "headers-missing")


def parse_count(value):
    """Read a limit or remaining header as an integer. Pure, None if unreadable.

    None and zero must stay distinct. Zero means the bucket is empty; None means
    nothing told us, and reporting a stripped header as an empty limiter sends
    somebody chasing a throttle that is not there.
    """
    if value is None:
        return None
    text = str(value).strip().replace(",", "").replace("_", "")
    if not text:
        return None
    try:
        return int(text)
    except ValueError:
        return None


def read_triples(headers):
    """Parse the anthropic-ratelimit-* triples off one response. Pure.

    Case-insensitive, because a proxy that rewrites header casing should not be
    able to turn a working probe into a report of missing headers.
    """
    lower = {}
    for name, value in dict(headers or {}).items():
        lower[str(name).strip().lower()] = value

    out = {}
    for name in NAMED + (AGGREGATE,):
        limit_h = "anthropic-ratelimit-%s-limit" % name
        remaining_h = "anthropic-ratelimit-%s-remaining" % name
        reset_h = "anthropic-ratelimit-%s-reset" % name
        if limit_h not in lower and remaining_h not in lower:
            continue
        reset = lower.get(reset_h)
        out[name] = {"limit": parse_count(lower.get(limit_h)),
                     "remaining": parse_count(lower.get(remaining_h)),
                     "reset": str(reset).strip() if reset is not None else None}
    return out


def seconds_until(value, now):
    """Seconds until an RFC 3339 reset stamp. Pure; the caller supplies now.

    Returns None when the stamp cannot be read rather than guessing. A reset
    window is a number a reader will act on, and half-parsing one is worse than
    printing that it was unreadable.
    """
    if value is None:
        return None
    text = str(value).strip()
    if not text:
        return None
    for suffix in ("Z", "z"):
        if text.endswith(suffix):
            text = text[:-1] + "+00:00"
            break
    try:
        when = dt.datetime.fromisoformat(text)
    except ValueError:
        return None
    if when.tzinfo is None:
        when = when.replace(tzinfo=dt.timezone.utc)
    return (when - now).total_seconds()


def share_left(triple):
    """remaining / limit for one triple, or None. Pure."""
    if not isinstance(triple, dict):
        return None
    limit = triple.get("limit")
    remaining = triple.get("remaining")
    if limit is None or remaining is None or limit <= 0:
        return None
    return max(0.0, min(1.0, remaining / float(limit)))


def mirrors(parsed):
    """Which named token limiter the aggregate triple is reporting. Pure.

    anthropic-ratelimit-tokens-* is documented as the most restrictive token
    limit currently in effect, so its ceiling equals the input ceiling or the
    output ceiling. Matching it back is the platform naming the binding bucket
    for you; nothing else in the response does that.
    """
    aggregate = (parsed or {}).get(AGGREGATE) or {}
    limit = aggregate.get("limit")
    if limit is None:
        return "no-aggregate"
    matched = []
    for name in ("input-tokens", "output-tokens"):
        other = (parsed or {}).get(name) or {}
        if other.get("limit") is not None and other.get("limit") == limit:
            matched.append(name)
    if len(matched) == 2:
        return "both"
    if len(matched) == 1:
        return matched[0]
    return "unmatched"


def emptiest(parsed):
    """The named bucket with the least left. Pure. Returns (name, share).

    The aggregate is excluded on purpose: it duplicates one of the named
    buckets, and letting it compete would report the same limiter twice under
    two names and hide whichever one it is not mirroring.
    """
    best = None
    for name in NAMED:
        share = share_left((parsed or {}).get(name) or {})
        if share is None:
            continue
        if best is None or share < best[1]:
            best = (name, share)
    return best


def verdict(parsed):
    """Say which limiter is binding, and when the two answers disagree. Pure."""
    if not parsed:
        return ("headers-missing",
                "no anthropic-ratelimit-* headers reached this process, so a "
                "429 here would arrive with nothing to classify it by and "
                "retry-after would be missing too")
    scarce = emptiest(parsed)
    if scarce is None:
        return ("unreadable",
                "the named triples arrived without a usable limit and "
                "remaining pair, so there is no ratio to compare")

    name, share = scarce
    shape = "%s is the emptiest named bucket at %.0f%% remaining" % (name, share * 100)
    mirror = mirrors(parsed)

    if mirror == "no-aggregate":
        return ("no-aggregate",
                "%s, and the aggregate anthropic-ratelimit-tokens triple is "
                "absent, so the platform's own view of the most restrictive "
                "token limit is not available on this response." % shape)
    if mirror == "unmatched":
        return ("aggregate-unmatched",
                "%s, but the aggregate token ceiling matches neither the input "
                "nor the output ceiling. A third and lower limit is in effect: "
                "a workspace override, or a different limiter group than the "
                "one this probe touched." % shape)
    if mirror == "both":
        return ("identified",
                "%s, and the aggregate ceiling equals both token ceilings, so "
                "input and output share a number here and only the remaining "
                "counters tell them apart." % shape)
    if mirror == name:
        return ("identified",
                "%s, and the aggregate ceiling mirrors %s. The tightest ceiling "
                "and the emptiest bucket are the same limiter." % (shape, mirror))
    return ("disagreement",
            "%s, while the aggregate ceiling mirrors %s. The tightest ceiling "
            "and the emptiest bucket are different limiters, so a handler that "
            "records only one of them will name the wrong cause."
            % (shape, mirror))


def configured(payload):
    """Fold GET /v1/organizations/rate_limits into {model_group: {type: value}}. Pure.

    A limiter type missing from a group's limits[] is not unlimited: it
    inherits. It is recorded as None and printed as unpublished, never as
    absent, because "no number" read as "no ceiling" is how a team convinces
    itself it has headroom it was never granted.
    """
    out = {}
    for entry in (payload or {}).get("data") or []:
        group = str(entry.get("model_group") or "").strip()
        if not group:
            continue
        row = out.setdefault(group, dict.fromkeys(LIMITER_TYPES))
        for limit in entry.get("limits") or []:
            kind = str(limit.get("type") or "").strip()
            if kind not in row:
                continue
            try:
                row[kind] = int(limit.get("value"))
            except (TypeError, ValueError):
                row[kind] = None
    return out


def log_headers(headers):
    """The header names a 429 handler should be recording. Pure.

    Built from what actually arrived rather than from a hardcoded list, so the
    printed repair does not tell a reader to log a header their gateway is
    stripping. This list is the whole output of the script that matters.
    """
    lower = set()
    for name in dict(headers or {}):
        lower.add(str(name).strip().lower())
    wanted = set()
    for name in NAMED + (AGGREGATE,):
        for suffix in ("limit", "remaining", "reset"):
            candidate = "anthropic-ratelimit-%s-%s" % (name, suffix)
            if candidate in lower:
                wanted.add(candidate)
    for extra in ("retry-after", "request-id", "anthropic-organization-id"):
        if extra in lower:
            wanted.add(extra)
    return sorted(wanted)


def probe(session):
    """One cheap real call with the workspace key. Generates nothing."""
    r = session.get(API + "/models", timeout=30)
    if r.status_code in (401, 403):
        raise SystemExit("%d from Anthropic: ANTHROPIC_API_KEY must be a "
                         "workspace or project key" % r.status_code)
    if r.status_code == 429:
        log.warning("the probe itself was rate limited; the headers below "
                    "describe the bucket that rejected it")
        return r.headers
    r.raise_for_status()
    return r.headers


def admin_limits(admin_key):
    """GET the configured per-model-group limits. Returns {} if no admin key."""
    if not admin_key:
        return {}
    s = requests.Session()
    s.headers.update({"x-api-key": admin_key, "anthropic-version": VERSION})
    r = s.get(API + "/organizations/rate_limits", timeout=60)
    if r.status_code in (401, 403):
        log.warning("%d from the Admin API: /v1/organizations/* needs an Admin "
                    "key (sk-ant-admin...). Continuing on headers alone.",
                    r.status_code)
        return {}
    r.raise_for_status()
    return configured(r.json())


def main():
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("--show-all", action="store_true",
                    help="also print every triple, not only the verdict")
    args = ap.parse_args()

    key = os.environ.get("ANTHROPIC_API_KEY")
    if not key:
        log.error("set ANTHROPIC_API_KEY (a workspace key) for the probe")
        return 2

    session = requests.Session()
    session.headers.update({"x-api-key": key, "anthropic-version": VERSION})

    headers = probe(session)
    parsed = read_triples(headers)
    state, detail = verdict(parsed)
    line = "%-20s %s" % (state, detail)
    if state in FINDINGS:
        log.warning(line)
    else:
        log.info(line)

    now = dt.datetime.now(dt.timezone.utc)
    if args.show_all:
        for name in sorted(parsed):
            triple = parsed[name]
            until = seconds_until(triple.get("reset"), now)
            log.info("  %-14s limit %s, remaining %s, resets %s", name,
                     triple.get("limit"), triple.get("remaining"),
                     "in %.0fs" % until if until is not None else "unreadable")

    groups = admin_limits(os.environ.get("ANTHROPIC_ADMIN_KEY"))
    for group in sorted(groups):
        row = groups[group]
        log.info("  %-24s rpm %s  itpm %s  otpm %s", group,
                 row["requests_per_minute"] if row["requests_per_minute"] is not None
                 else "unpublished",
                 row["input_tokens_per_minute"] if row["input_tokens_per_minute"] is not None
                 else "unpublished",
                 row["output_tokens_per_minute"] if row["output_tokens_per_minute"] is not None
                 else "unpublished")
    if not groups:
        log.info("  no configured limits read; set ANTHROPIC_ADMIN_KEY to name "
                 "the ceilings per model group rather than only the probe's")

    names = log_headers(headers)
    if names:
        log.warning("  repair: record these on every 429 instead of catching a "
                    "broad status error: %s", ", ".join(names))
        log.warning("  repair: branch before sleeping. No retry-after plus "
                    "error.details.error_code of enforced_spend_limit_reached "
                    "is a billing stop, not a throttle, and will not clear.")
    else:
        log.warning("  repair: no rate-limit headers arrived at all. Check the "
                    "proxy or gateway in front of api.anthropic.com and let "
                    "the anthropic-ratelimit-* and retry-after headers through.")

    return 1 if state in FINDINGS else 0


if __name__ == "__main__":
    sys.exit(main())
anthropic-limiter-identify.mjs
/**
 * Name which Anthropic rate limiter is binding, instead of catching 429.
 *
 * Read only. Two GET requests and nothing else. ANTHROPIC_API_KEY is a
 * workspace key used for a single probe against /v1/models, which generates no
 * tokens and bills nothing; ANTHROPIC_ADMIN_KEY is an Admin API key used for
 * the configured limits, because /v1/organizations/* rejects a workspace key.
 *
 * Nothing here provokes a 429.
 */
const API = 'https://api.anthropic.com/v1';
const VERSION = '2023-06-01';

// The three limiters that empty independently. "tokens" is not a fourth bucket:
// it reports whichever of the two token buckets is most restrictive.
const NAMED = ['requests', 'input-tokens', 'output-tokens'];
const AGGREGATE = 'tokens';

const LIMITER_TYPES = ['requests_per_minute', 'input_tokens_per_minute',
                       'output_tokens_per_minute'];

const FINDINGS = new Set(['disagreement', 'aggregate-unmatched', 'headers-missing']);

/**
 * Read a limit or remaining header as an integer. Pure, null if unreadable.
 * null and 0 stay distinct: 0 is an empty bucket, null is a stripped header.
 */
export function parseCount(value) {
  if (value === null || value === undefined) return null;
  const text = String(value).trim().replace(/[,_]/g, '');
  if (!text) return null;
  const n = Number(text);
  return Number.isInteger(n) ? n : null;
}

/** Parse the anthropic-ratelimit-* triples off one response. Pure. */
export function readTriples(headers) {
  const lower = new Map();
  const entries = typeof headers?.entries === 'function'
    ? [...headers.entries()] : Object.entries(headers ?? {});
  for (const [name, value] of entries) lower.set(String(name).trim().toLowerCase(), value);

  const out = {};
  for (const name of [...NAMED, AGGREGATE]) {
    const limitH = `anthropic-ratelimit-${name}-limit`;
    const remainingH = `anthropic-ratelimit-${name}-remaining`;
    const resetH = `anthropic-ratelimit-${name}-reset`;
    if (!lower.has(limitH) && !lower.has(remainingH)) continue;
    const reset = lower.get(resetH);
    out[name] = {
      limit: parseCount(lower.get(limitH)),
      remaining: parseCount(lower.get(remainingH)),
      reset: reset === null || reset === undefined ? null : String(reset).trim(),
    };
  }
  return out;
}

/**
 * Seconds until an RFC 3339 reset stamp. Pure; the caller supplies now.
 * Returns null when the stamp cannot be read rather than guessing.
 */
export function secondsUntil(value, now) {
  if (value === null || value === undefined) return null;
  const text = String(value).trim();
  if (!text) return null;
  const when = Date.parse(text);
  if (Number.isNaN(when)) return null;
  return (when - now.getTime()) / 1000;
}

/** remaining / limit for one triple, or null. Pure. */
export function shareLeft(triple) {
  if (!triple || typeof triple !== 'object') return null;
  const { limit, remaining } = triple;
  if (limit === null || limit === undefined) return null;
  if (remaining === null || remaining === undefined) return null;
  if (limit <= 0) return null;
  return Math.max(0, Math.min(1, remaining / limit));
}

/**
 * Which named token limiter the aggregate triple is reporting. Pure.
 * The aggregate ceiling equals the input ceiling or the output ceiling, so
 * matching it back is the platform naming the binding bucket for you.
 */
export function mirrors(parsed) {
  const limit = (parsed ?? {})[AGGREGATE]?.limit;
  if (limit === null || limit === undefined) return 'no-aggregate';
  const matched = [];
  for (const name of ['input-tokens', 'output-tokens']) {
    const other = (parsed ?? {})[name]?.limit;
    if (other !== null && other !== undefined && other === limit) matched.push(name);
  }
  if (matched.length === 2) return 'both';
  if (matched.length === 1) return matched[0];
  return 'unmatched';
}

/**
 * The named bucket with the least left. Pure. Returns [name, share].
 * The aggregate is excluded: it duplicates one of the named buckets.
 */
export function emptiest(parsed) {
  let best = null;
  for (const name of NAMED) {
    const share = shareLeft((parsed ?? {})[name]);
    if (share === null) continue;
    if (best === null || share < best[1]) best = [name, share];
  }
  return best;
}

/** Say which limiter is binding, and when the two answers disagree. Pure. */
export function verdict(parsed) {
  if (!parsed || Object.keys(parsed).length === 0) {
    return ['headers-missing',
      'no anthropic-ratelimit-* headers reached this process, so a 429 here ' +
      'would arrive with nothing to classify it by and retry-after would be ' +
      'missing too'];
  }
  const scarce = emptiest(parsed);
  if (scarce === null) {
    return ['unreadable',
      'the named triples arrived without a usable limit and remaining pair, ' +
      'so there is no ratio to compare'];
  }
  const [name, share] = scarce;
  const shape = `${name} is the emptiest named bucket at ` +
                `${(share * 100).toFixed(0)}% remaining`;
  const mirror = mirrors(parsed);

  if (mirror === 'no-aggregate') {
    return ['no-aggregate',
      `${shape}, and the aggregate anthropic-ratelimit-tokens triple is absent, ` +
      "so the platform's own view of the most restrictive token limit is not " +
      'available on this response.'];
  }
  if (mirror === 'unmatched') {
    return ['aggregate-unmatched',
      `${shape}, but the aggregate token ceiling matches neither the input nor ` +
      'the output ceiling. A third and lower limit is in effect: a workspace ' +
      'override, or a different limiter group than the one this probe touched.'];
  }
  if (mirror === 'both') {
    return ['identified',
      `${shape}, and the aggregate ceiling equals both token ceilings, so input ` +
      'and output share a number here and only the remaining counters tell ' +
      'them apart.'];
  }
  if (mirror === name) {
    return ['identified',
      `${shape}, and the aggregate ceiling mirrors ${mirror}. The tightest ` +
      'ceiling and the emptiest bucket are the same limiter.'];
  }
  return ['disagreement',
    `${shape}, while the aggregate ceiling mirrors ${mirror}. The tightest ` +
    'ceiling and the emptiest bucket are different limiters, so a handler that ' +
    'records only one of them will name the wrong cause.'];
}

/**
 * Fold GET /v1/organizations/rate_limits into {model_group: {type: value}}. Pure.
 * A limiter type missing from limits[] inherits rather than being unlimited, so
 * it is recorded as null and printed as unpublished.
 */
export function configured(payload) {
  const out = {};
  for (const entry of (payload ?? {}).data ?? []) {
    const group = String(entry.model_group ?? '').trim();
    if (!group) continue;
    if (!out[group]) {
      out[group] = {};
      for (const t of LIMITER_TYPES) out[group][t] = null;
    }
    for (const limit of entry.limits ?? []) {
      const kind = String(limit.type ?? '').trim();
      if (!(kind in out[group])) continue;
      const value = Number(limit.value);
      out[group][kind] = Number.isInteger(value) ? value : null;
    }
  }
  return out;
}

/**
 * The header names a 429 handler should be recording. Pure.
 * Built from what actually arrived, so the printed repair never tells a reader
 * to log a header their gateway is stripping.
 */
export function logHeaders(headers) {
  const lower = new Set();
  const entries = typeof headers?.entries === 'function'
    ? [...headers.entries()] : Object.entries(headers ?? {});
  for (const [name] of entries) lower.add(String(name).trim().toLowerCase());

  const wanted = new Set();
  for (const name of [...NAMED, AGGREGATE]) {
    for (const suffix of ['limit', 'remaining', 'reset']) {
      const candidate = `anthropic-ratelimit-${name}-${suffix}`;
      if (lower.has(candidate)) wanted.add(candidate);
    }
  }
  for (const extra of ['retry-after', 'request-id', 'anthropic-organization-id']) {
    if (lower.has(extra)) wanted.add(extra);
  }
  return [...wanted].sort();
}

/** One cheap real call with the workspace key. Generates nothing. */
async function probe(key) {
  const res = await fetch(`${API}/models`, {
    headers: { 'x-api-key': key, 'anthropic-version': VERSION },
  });
  if (res.status === 401 || res.status === 403) {
    throw new Error(`${res.status} from Anthropic: ANTHROPIC_API_KEY must be a ` +
                    'workspace or project key');
  }
  if (res.status === 429) {
    console.warn('the probe itself was rate limited; the headers below describe ' +
                 'the bucket that rejected it');
    return res.headers;
  }
  if (!res.ok) throw new Error(`${res.status} from /v1/models`);
  return res.headers;
}

async function adminLimits(adminKey) {
  if (!adminKey) return {};
  const res = await fetch(`${API}/organizations/rate_limits`, {
    headers: { 'x-api-key': adminKey, 'anthropic-version': VERSION },
  });
  if (res.status === 401 || res.status === 403) {
    console.warn(`${res.status} from the Admin API: /v1/organizations/* needs an ` +
                 'Admin key (sk-ant-admin...). Continuing on headers alone.');
    return {};
  }
  if (!res.ok) throw new Error(`${res.status} from /v1/organizations/rate_limits`);
  return configured(await res.json());
}

async function main() {
  const key = process.env.ANTHROPIC_API_KEY;
  if (!key) {
    console.error('set ANTHROPIC_API_KEY (a workspace key) for the probe');
    process.exitCode = 2;
    return;
  }
  const showAll = process.env.SHOW_ALL === '1';

  const headers = await probe(key);
  const parsed = readTriples(headers);
  const [state, detail] = verdict(parsed);
  const line = `${state.padEnd(20)} ${detail}`;
  if (FINDINGS.has(state)) console.warn(line); else console.log(line);

  const now = new Date();
  if (showAll) {
    for (const name of Object.keys(parsed).sort()) {
      const triple = parsed[name];
      const until = secondsUntil(triple.reset, now);
      console.log(`  ${name.padEnd(14)} limit ${triple.limit}, remaining ` +
                  `${triple.remaining}, resets ` +
                  `${until === null ? 'unreadable' : `in ${until.toFixed(0)}s`}`);
    }
  }

  const groups = await adminLimits(process.env.ANTHROPIC_ADMIN_KEY);
  const names = Object.keys(groups).sort();
  for (const group of names) {
    const row = groups[group];
    const show = (v) => (v === null ? 'unpublished' : String(v));
    console.log(`  ${group.padEnd(24)} rpm ${show(row.requests_per_minute)}  ` +
                `itpm ${show(row.input_tokens_per_minute)}  ` +
                `otpm ${show(row.output_tokens_per_minute)}`);
  }
  if (names.length === 0) {
    console.log('  no configured limits read; set ANTHROPIC_ADMIN_KEY to name the ' +
                "ceilings per model group rather than only the probe's");
  }

  const toLog = logHeaders(headers);
  if (toLog.length > 0) {
    console.warn('  repair: record these on every 429 instead of catching a broad ' +
                 `status error: ${toLog.join(', ')}`);
    console.warn('  repair: branch before sleeping. No retry-after plus ' +
                 'error.details.error_code of enforced_spend_limit_reached is a ' +
                 'billing stop, not a throttle, and will not clear.');
  } else {
    console.warn('  repair: no rate-limit headers arrived at all. Check the proxy ' +
                 'or gateway in front of api.anthropic.com and let the ' +
                 'anthropic-ratelimit-* and retry-after headers through.');
  }

  process.exitCode = FINDINGS.has(state) ? 1 : 0;
}

if (import.meta.url === `file://${process.argv[1]}`) {
  main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}

Add a test

The first test is the whole mechanism: an aggregate ceiling of four hundred thousand against an input ceiling of five million and an output ceiling of four hundred thousand, which identifies output as the limiter without any inference at all. The second is the case that makes the note worth writing — the tightest ceiling and the emptiest bucket are different limiters, and a report that prints one of them is confidently wrong. The rest hold the honest gaps open: an aggregate matching neither ceiling is a third limit rather than a bug to smooth over, an unpublished limiter is not an unlimited one, and a response with no headers at all is a finding rather than a pass.

test_anthropic_limiter_identify.py
import datetime as dt

from anthropic_limiter_identify import (configured, emptiest, log_headers,
                                        mirrors, parse_count, read_triples,
                                        seconds_until, share_left, verdict)

NOW = dt.datetime(2026, 8, 30, 12, 0, 0, tzinfo=dt.timezone.utc)


def test_the_aggregate_ceiling_names_the_binding_limiter():
    # Sonnet-shaped numbers: ITPM five million, OTPM four hundred thousand. The
    # aggregate equals the output ceiling, which is Anthropic telling you which
    # bucket is tightest without anything having to infer it.
    parsed = read_triples({
        "anthropic-ratelimit-requests-limit": "4000",
        "anthropic-ratelimit-requests-remaining": "3600",
        "anthropic-ratelimit-input-tokens-limit": "5000000",
        "anthropic-ratelimit-input-tokens-remaining": "4000000",
        "anthropic-ratelimit-output-tokens-limit": "400000",
        "anthropic-ratelimit-output-tokens-remaining": "12000",
        "anthropic-ratelimit-tokens-limit": "400000",
        "anthropic-ratelimit-tokens-remaining": "12000",
    })
    assert mirrors(parsed) == "output-tokens"
    assert emptiest(parsed) == ("output-tokens", 0.03)
    state, detail = verdict(parsed)
    assert state == "identified"
    assert "output-tokens is the emptiest named bucket at 3% remaining" in detail


def test_the_tightest_ceiling_and_the_emptiest_bucket_can_disagree():
    # The request bucket is nearly gone while the output ceiling is the lower
    # number. Reporting either alone names the wrong cause.
    parsed = read_triples({
        "anthropic-ratelimit-requests-limit": "4000",
        "anthropic-ratelimit-requests-remaining": "40",
        "anthropic-ratelimit-input-tokens-limit": "5000000",
        "anthropic-ratelimit-input-tokens-remaining": "4900000",
        "anthropic-ratelimit-output-tokens-limit": "400000",
        "anthropic-ratelimit-output-tokens-remaining": "380000",
        "anthropic-ratelimit-tokens-limit": "400000",
        "anthropic-ratelimit-tokens-remaining": "380000",
    })
    state, detail = verdict(parsed)
    assert state == "disagreement"
    assert "requests is the emptiest named bucket at 1% remaining" in detail
    assert "mirrors output-tokens" in detail


def test_an_aggregate_matching_neither_ceiling_is_a_third_limit():
    parsed = read_triples({
        "anthropic-ratelimit-requests-limit": "4000",
        "anthropic-ratelimit-requests-remaining": "3900",
        "anthropic-ratelimit-input-tokens-limit": "5000000",
        "anthropic-ratelimit-input-tokens-remaining": "4900000",
        "anthropic-ratelimit-output-tokens-limit": "400000",
        "anthropic-ratelimit-output-tokens-remaining": "390000",
        "anthropic-ratelimit-tokens-limit": "150000",
        "anthropic-ratelimit-tokens-remaining": "150000",
    })
    assert mirrors(parsed) == "unmatched"
    assert verdict(parsed)[0] == "aggregate-unmatched"


def test_no_headers_is_a_finding_and_not_a_pass():
    assert read_triples({"content-type": "application/json"}) == {}
    assert read_triples(None) == {}
    state, detail = verdict({})
    assert state == "headers-missing"
    assert "retry-after would be missing too" in detail
    assert log_headers({"content-type": "application/json"}) == []


def test_the_aggregate_never_competes_to_be_the_emptiest_bucket():
    # Otherwise the same limiter is reported twice under two names, and the
    # bucket the aggregate is not mirroring disappears from the report.
    parsed = {"requests": {"limit": 100, "remaining": 90},
              "output-tokens": {"limit": 1000, "remaining": 500},
              "tokens": {"limit": 1000, "remaining": 1}}
    assert emptiest(parsed) == ("output-tokens", 0.5)


def test_absent_and_empty_are_different_readings():
    assert parse_count(None) is None
    assert parse_count("") is None
    assert parse_count("0") == 0
    assert parse_count("2,000,000") == 2000000
    assert parse_count("lots") is None
    assert share_left({"limit": 100, "remaining": None}) is None
    assert share_left({"limit": 0, "remaining": 0}) is None
    assert verdict({"requests": {"limit": None, "remaining": None}})[0] == "unreadable"


def test_rfc3339_resets_parse_and_unreadable_ones_stay_unreadable():
    assert seconds_until("2026-08-30T12:00:30Z", NOW) == 30.0
    assert seconds_until("2026-08-30T12:00:30+00:00", NOW) == 30.0
    assert seconds_until("in a bit", NOW) is None
    assert seconds_until("", NOW) is None
    assert seconds_until(None, NOW) is None


def test_an_unpublished_limiter_is_not_an_unlimited_one():
    payload = {"data": [
        {"model_group": "claude-sonnet-5", "limits": [
            {"type": "requests_per_minute", "value": 4000},
            {"type": "input_tokens_per_minute", "value": 5000000},
            {"type": "output_tokens_per_minute", "value": 1000000}]},
        {"model_group": "message-batches", "limits": [
            {"type": "requests_per_minute", "value": 100}]},
    ]}
    folded = configured(payload)
    assert folded["claude-sonnet-5"]["output_tokens_per_minute"] == 1000000
    # Absent from limits[] means it inherits, so it must read as None and get
    # printed as unpublished rather than silently becoming zero or infinity.
    assert folded["message-batches"]["input_tokens_per_minute"] is None
    assert folded["message-batches"]["requests_per_minute"] == 100
    assert configured({}) == {}
    assert configured(None) == {}


def test_the_repair_lists_only_headers_that_actually_arrived():
    names = log_headers({
        "Anthropic-RateLimit-Output-Tokens-Remaining": "12000",
        "anthropic-ratelimit-tokens-limit": "400000",
        "Retry-After": "12",
        "request-id": "req_fake123",
        "content-type": "application/json",
    })
    assert names == ["anthropic-ratelimit-output-tokens-remaining",
                     "anthropic-ratelimit-tokens-limit",
                     "request-id", "retry-after"]
anthropic-limiter-identify.test.mjs
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { configured, emptiest, logHeaders, mirrors, parseCount, readTriples,
         secondsUntil, shareLeft, verdict }
  from './anthropic-limiter-identify.mjs';

const NOW = new Date('2026-08-30T12:00:00Z');

test('the aggregate ceiling names the binding limiter', () => {
  const parsed = readTriples({
    'anthropic-ratelimit-requests-limit': '4000',
    'anthropic-ratelimit-requests-remaining': '3600',
    'anthropic-ratelimit-input-tokens-limit': '5000000',
    'anthropic-ratelimit-input-tokens-remaining': '4000000',
    'anthropic-ratelimit-output-tokens-limit': '400000',
    'anthropic-ratelimit-output-tokens-remaining': '12000',
    'anthropic-ratelimit-tokens-limit': '400000',
    'anthropic-ratelimit-tokens-remaining': '12000',
  });
  assert.equal(mirrors(parsed), 'output-tokens');
  assert.deepEqual(emptiest(parsed), ['output-tokens', 0.03]);
  const [state, detail] = verdict(parsed);
  assert.equal(state, 'identified');
  assert.match(detail, /output-tokens is the emptiest named bucket at 3% remaining/);
});

test('the tightest ceiling and the emptiest bucket can disagree', () => {
  const parsed = readTriples({
    'anthropic-ratelimit-requests-limit': '4000',
    'anthropic-ratelimit-requests-remaining': '40',
    'anthropic-ratelimit-input-tokens-limit': '5000000',
    'anthropic-ratelimit-input-tokens-remaining': '4900000',
    'anthropic-ratelimit-output-tokens-limit': '400000',
    'anthropic-ratelimit-output-tokens-remaining': '380000',
    'anthropic-ratelimit-tokens-limit': '400000',
    'anthropic-ratelimit-tokens-remaining': '380000',
  });
  const [state, detail] = verdict(parsed);
  assert.equal(state, 'disagreement');
  assert.match(detail, /requests is the emptiest named bucket at 1% remaining/);
  assert.match(detail, /mirrors output-tokens/);
});

test('an aggregate matching neither ceiling is a third limit', () => {
  const parsed = readTriples({
    'anthropic-ratelimit-requests-limit': '4000',
    'anthropic-ratelimit-requests-remaining': '3900',
    'anthropic-ratelimit-input-tokens-limit': '5000000',
    'anthropic-ratelimit-input-tokens-remaining': '4900000',
    'anthropic-ratelimit-output-tokens-limit': '400000',
    'anthropic-ratelimit-output-tokens-remaining': '390000',
    'anthropic-ratelimit-tokens-limit': '150000',
    'anthropic-ratelimit-tokens-remaining': '150000',
  });
  assert.equal(mirrors(parsed), 'unmatched');
  assert.equal(verdict(parsed)[0], 'aggregate-unmatched');
});

test('no headers is a finding and not a pass', () => {
  assert.deepEqual(readTriples({ 'content-type': 'application/json' }), {});
  assert.deepEqual(readTriples(null), {});
  const [state, detail] = verdict({});
  assert.equal(state, 'headers-missing');
  assert.match(detail, /retry-after would be missing too/);
  assert.deepEqual(logHeaders({ 'content-type': 'application/json' }), []);
});

test('the aggregate never competes to be the emptiest bucket', () => {
  const parsed = {
    requests: { limit: 100, remaining: 90 },
    'output-tokens': { limit: 1000, remaining: 500 },
    tokens: { limit: 1000, remaining: 1 },
  };
  assert.deepEqual(emptiest(parsed), ['output-tokens', 0.5]);
});

test('absent and empty are different readings', () => {
  assert.equal(parseCount(null), null);
  assert.equal(parseCount(''), null);
  assert.equal(parseCount('0'), 0);
  assert.equal(parseCount('2,000,000'), 2000000);
  assert.equal(parseCount('lots'), null);
  assert.equal(shareLeft({ limit: 100, remaining: null }), null);
  assert.equal(shareLeft({ limit: 0, remaining: 0 }), null);
  assert.equal(verdict({ requests: { limit: null, remaining: null } })[0], 'unreadable');
});

test('rfc3339 resets parse and unreadable ones stay unreadable', () => {
  assert.equal(secondsUntil('2026-08-30T12:00:30Z', NOW), 30);
  assert.equal(secondsUntil('2026-08-30T12:00:30+00:00', NOW), 30);
  assert.equal(secondsUntil('in a bit', NOW), null);
  assert.equal(secondsUntil('', NOW), null);
  assert.equal(secondsUntil(null, NOW), null);
});

test('an unpublished limiter is not an unlimited one', () => {
  const folded = configured({ data: [
    { model_group: 'claude-sonnet-5', limits: [
      { type: 'requests_per_minute', value: 4000 },
      { type: 'input_tokens_per_minute', value: 5000000 },
      { type: 'output_tokens_per_minute', value: 1000000 }] },
    { model_group: 'message-batches', limits: [
      { type: 'requests_per_minute', value: 100 }] },
  ] });
  assert.equal(folded['claude-sonnet-5'].output_tokens_per_minute, 1000000);
  assert.equal(folded['message-batches'].input_tokens_per_minute, null);
  assert.equal(folded['message-batches'].requests_per_minute, 100);
  assert.deepEqual(configured({}), {});
  assert.deepEqual(configured(null), {});
});

test('the repair lists only headers that actually arrived', () => {
  assert.deepEqual(logHeaders({
    'Anthropic-RateLimit-Output-Tokens-Remaining': '12000',
    'anthropic-ratelimit-tokens-limit': '400000',
    'Retry-After': '12',
    'request-id': 'req_fake123',
    'content-type': 'application/json',
  }), ['anthropic-ratelimit-output-tokens-remaining',
       'anthropic-ratelimit-tokens-limit',
       'request-id', 'retry-after']);
});

FAQ

Why is there no single TPM number on Anthropic?

Because input and output are metered separately and empty at different rates. Input tokens per minute is roughly five times output tokens per minute at every tier, so a summarisation workload and a generation workload run out of completely different things. One combined number would have to be the minimum of the two, which is exactly what the aggregate anthropic-ratelimit-tokens header already is.

The aggregate matches neither ceiling. What now?

Something lower than both is in effect. The usual cause is a workspace-level override, which can be set below the organization limit and which the aggregate faithfully reports. The other cause is that you probed a different limiter group than you think: batches, token counting, files and web search each carry their own. Read GET /v1/organizations/rate_limits and compare per model group before assuming a bug.

Can the script tell me which limiter caused yesterday's 429s?

No, and nothing can. Neither provider exposes a request log, so there is no endpoint that will return the status codes or error bodies of calls you already made. That is precisely why the repair is to start recording the headers now: the only version of this question that has an answer is the one asked about a 429 you captured.

Should the script just retry until it sees a 429 and read that?

No. Deliberately draining a production token bucket to inspect the error is an outage you scheduled, and it costs your colleagues the same headroom it costs you. Every header this note reads is present on a successful response, so one call to GET /v1/models answers the question without spending anything.

Does the retry-after header make the branching unnecessary?

It makes the sleeping correct, not the diagnosis. retry-after tells you when the bucket refills; it does not tell you which bucket, so it cannot tell you whether to reduce concurrency, cache the prefix or shorten the answers. And its absence is itself a signal: a 429 with no retry-after and error.details.error_code of enforced_spend_limit_reached is a monthly spend cap that will not clear until the month does.

Related field notes

Sources

Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.

Stuck on a tricky one?

If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.