Field notes
LLM API fix guides
An LLM API bills you whether or not the answer was any good, and most of the ways it goes wrong do not raise an exception. A model id retires on a date nobody diaried, a response is cut off mid-JSON because nobody checked finish_reason, a retry storm triples the invoice. Each note here explains one such problem and gives you a script that finds it through the API.
A script here holds a key that can spend real money on inference. So these read, they tell you exactly what is wrong, and they print the repair: the endpoint, the parameter, the model id to migrate to. You run it.
Several of them want an admin or organization-scoped read key rather than a project key, because usage and cost live on the organization. Each note says which it needs.
OpenAI and Anthropic
A model id in use is past its published shutdown date
GET /v1/models carries a shutdown_date per model. Once it passes, the id fails exactly like a typo: 404 model_not_found, with no retired error code.
A model you still call retires in under 90 days
The shutdown_date is real but still ahead of you. One GET turns that field into a migration schedule, ordered by days left and by the traffic each id carries.
A retired model id still sitting in the code
Anthropic drops a retired id from the models list entirely, with no date left to read. You find it by diffing your config strings against GET /v1/models.
A floating model alias silently changes model under you
An alias is not retiring, it moves. GET /v1/models/{alias} returns the snapshot it resolves to today, which is the id that should have been in the config.
429 credit_balance_exhausted retried forever as a rate limit
A 429 carrying insufficient_quota is a billing wall, not a throttle. Every SDK raises RateLimitError, so the retry loop hammers it until someone notices.
No hard spend limit is set, so the bill has no ceiling
OpenAI's hard spend limit is opt-in and lives on its own admin endpoint. Read it, the alerts, and month-to-date spend to see if anything stops a runaway.
Reasoning tokens are billed as output but never returned
Cost per request jumps after a model switch while the visible answers stay the same length. The reasoning tokens are billed as output and never returned.
Output tokens, not input, are what the bill is made of
Output is priced at five times input on every current Claude model, and thinking tokens bill as output. The cost report says which side the bill sits on.
Keys still work after their owner loses project access
Removing someone from a project revokes their console access, not their API key. One admin call lists the keys that are still live and still billing.
An archived project still holds live API keys
Archiving hides a project from the default listing without revoking anything inside it. Without include_archived=true your key audit never sees those keys.
Prompt caching was never switched on anywhere
cache_read_input_tokens is flat zero across every bucket while uncached input climbs. Caching is opt-in, and nothing tells you it is off.
Cache writes are paid for and never read back
A cache write costs 1.25x or 2x base input and a read costs 0.1x. When the reads never come, caching costs strictly more than leaving it off.
A batch reads completed while some of its rows failed
GET /v1/batches returns completed while request_counts.failed is non-zero. Completed means the run finished, not that every request inside it succeeded.
The batch left an error_file_id that nothing ever fetched
A completed batch carries a non-null error_file_id and no code ever called /content. The failures were written down and the downstream table is short.
A batch expired when the 24 hour completion window closed
status is expired and request_counts.completed is below total. The fixed 24h window closed on rows that had not run, and no HTTP error was raised.
Scheduled jobs pay full price for work the Batch API halves
group_by=batch returns one row, batch:false. Nightly enrichment and backfills go through the synchronous endpoint at roughly twice the batch price.
Fast mode billed at twice the rate and served as default
The tier you request and the tier you are served are different fields. Read the project setting against the invoice: both directions of the mismatch cost money.
Streamed responses report no usage and the dashboard undercounts
usage is null on every streamed chunk unless you ask for the totals. Reconcile the org token report against your own telemetry to size the hole.
Spend jumped week over week and no release explains it
Eight weeks of daily cost folded into whole weeks. The shape of the change is the finding: a spike, a step and a ramp want three different people.
One line item or project is most of the organization's bill
Ungrouped, the cost report is one opaque number per day. Grouped by line_item and project_id it is usually one row carrying most of the total.
A frontier model is answering twenty-token questions
Divide output_tokens by num_model_requests on the usage endpoint. A premium model whose mean answer is twenty tokens is the wrong size for the work.
Per-customer cost is unknowable because tenants share a key
user_id on the Usage API names your own org members and service accounts, never your end users. Resolve every principal and the gap becomes provable.
Audio and image usage never shows up in a token dashboard
Speech bills by characters, transcription by seconds, images by count. A dashboard built on usage/completions is structurally unable to see any of it.
A fine-tuned model was trained, billed, and never called once
Succeeded fine-tuning jobs name a model id. Thirty days of usage grouped by model shows zero requests against it. Training was paid for; inference never ran.
Web search is billing $10 per 1,000 searches unnoticed
Sum server_tool_use.web_search_requests per key on the Claude usage report. The tool fee is $10 per 1,000 searches and no token graph can show it.
Code execution has spent its free 1,550 container hours
A non-zero code_execution row on the Claude cost report means the free 1,550 container hours are gone. The messages usage report never shows it.
US inference geo is billing every token at 1.1x
Group the Claude usage report by inference_geo. A workspace whose data_residency defaults to us pays a 1.1x multiplier on every token category.
Most of your input tokens sit in the 200k-1M band
Group the Claude usage report by context_window. A growing 200k-1M share of uncached input is context bloat, not a price band, and compaction is the fix.
X-ratelimit-remaining sits near zero before any 429
OpenAI has no endpoint that returns your remaining quota. The headers on one cheap GET are the only forward-looking view, and nothing reads them.
429s are retried blindly without reading which limit hit
Anthropic enforces three independent limiters and a fourth header that mirrors the tightest. A handler that catches 429 and sleeps discards all four.
ITPM runs out because uncached input is never cached
Cache reads do not count toward the input limiter. A full ITPM bucket next to a zero cache-read share is a throughput problem caching fixes, not a discount.
Output tokens per minute is the real ceiling, not RPM
The output limiter is full while input and requests have room. More workers generate the same tokens against the same bucket, so concurrency changes nothing.
Prompts overflow the context window and 400 as too long
count_tokens is free and returns the exact number a window check needs. Nothing calls it, so overflow arrives as a 400 in production or a 200 nobody reads.
A 32 MB request is rejected with 413 before Anthropic sees it
The only ceiling in this set measured in bytes, not tokens. Cloudflare rejects it in front of the API, so the failure appears in no usage report at all.
Max_tokens is set above the model's own output cap
The model object publishes the legal ceiling for max_tokens and the docs table lags it. One shared config across two tiers 400s on the smaller model.
A non-streaming request over 10 minutes times out with 504
Not a size problem. A large max_tokens on a non-streaming path runs past a wall clock, and the repair is streaming rather than a shorter prompt.
Requests billed, zero output tokens: max_tokens refused
A reasoning model rejects max_tokens by name, so every call 400s before generation. In the usage report that is requests with no tokens either side.
Request count tripled while token volume stayed flat
A retry storm moves one of the two series the usage report carries. Tokens per request collapsing while requests climb is the only thing that does it.
529 overloaded errors arrive in clusters and get dropped
Anthropic bills no request count, so 5xx loss shows up only as your own attempt counter minus the work the usage report proves was actually done.
A live project's usage buckets have been empty for days
Nothing errored because nothing was sent. The usage endpoint answers with empty results arrays, and no monitor anywhere alerts on the absence of traffic.
JSON cut off mid-object because the ceiling was reached
A completed request whose answer was cut: status incomplete, reason max_output_tokens, and a body that is a valid prefix of the JSON you asked for.
The model refused and the refusal field was never read
A refusal arrives in its own content type, not as an error and not as text. A parser reaching for the answer finds nothing and writes an empty record.
Strict omitted, so the JSON schema is only a suggestion
Structured Outputs only guarantees the schema when strict is true. Absent or false, it is a hint the model usually follows, and nothing warns you.
Tool-call arguments that parse and still break the schema
Function arguments arrive as a JSON string that may be malformed, or may parse cleanly and violate the declared schema. Your dispatcher throws, not the API.
Tool shipped on every request and never once called
The declared tool names minus the names that ever come back as a function_call. A tool the model never picks is billed every turn and delivers nothing.
Tool schemas are most of the input tokens on every call
Count the same body with tools and without. The difference is what the schemas cost per call, and the counting endpoint is free and bills nothing.
Parallel tool calls void the strict schema guarantee
Structured Outputs is not supported alongside parallel function calls, and parallel_tool_calls defaults to true. The guarantee holds until the model fans out.
Cache written on every call by a prefix that keeps moving
Writes in every minute, back to back, and reads at zero. A run longer than the TTL proves the entry was alive and never matched: the prefix changes.
Prompt sits under the model's cache minimum, so nothing caches
cache_control is accepted and ignored below the floor. Caching that works on one model and stops on another brackets the prefix between two minimums.
Cache hits fall away exactly when the fleet scales out
Without prompt_cache_key, identical prompts scatter across backends. The signature is a cached share negatively correlated with the hour's request rate.
Every scheduled run starts on a cache that was evicted
The cold hours are the ones that resume after idle time. Binning cached share by preceding gap length says whether the repair is a parameter or a schedule.
Cache read share stepped down the day the model changed
A model switch starts cold, which is expected. A sustained collapse aligned with the day the new id first appears is a floor or a default that moved.
Priority Tier never covered the model you migrated to
Group the Claude usage report by service_tier and model. A model that never reports priority is not misconfigured, it has no coverage at all.
The 1M context window is capped at 200k in your own code
GET /v1/models gives max_input_tokens per id. Compare it with the ceiling your application enforces: a bought 1M window is often capped in software.
Claude Code sessions billed with zero cache reads
A different report entirely: the Claude Code usage report, per actor. Two or more sessions with cache_read at zero means the prefix is bought again every turn.
Claude Code edits rejected more often than they are kept
The same Claude Code report, read for tool_actions. Accepted over accepted plus rejected: output that was generated, billed, read by a person and dropped.
API keys that no request has ever used
OpenAI puts last_used_at on the key object. Anthropic has no such field, so the same question needs a set difference against the usage report.
Production keys owned by people, not service accounts
Filter the key list on owner.type == user and join it to cost by api_key_id. The finding is who owns the credential, not how much of the bill it holds.
A service account key that has never been rotated
Take the newest key age per service account. Past 180 days with only one key there is no overlap to rotate through, and the audit log confirms only per project.
Nobody has ever read the key lifecycle audit log
Both providers record every key and member lifecycle event and wait to be asked. Resolve each actor against the current roster and the unreviewed ones surface.
One project holds every environment, so nothing can be capped
Count the active projects before you rank them. One active project means there is no boundary to cap, alert or attribute against, only an org-wide total.
Cost lands in the default workspace and cannot be charged back
Rows with a null workspace_id are two different causes. One is a list of API keys you can move; the other is Console playground traffic with no key at all.
Almost everyone in the organization holds the owner role
The org role model is owner or reader, so every unblock is a promotion. Count owners with service accounts removed, then check the project level too.
Organization invites sat pending until they expired
An invite whose status still reads pending while its expires_at has passed is invisible to a status filter. Read the timestamp, and read the role it carried.
Nothing has ever called the free moderation endpoint
Compare the moderations usage report against completions. Zero moderation requests on a public product is one finding; a retired text-moderation-* id the other.
Zero data retention is claimed and no project resolves to it
The org has a default and each project overrides or inherits it. Read both and report what each project actually resolves to, not what was negotiated.
A model permission policy that has never excluded anything
mode deny_list with an empty model_ids looks configured and permits every model. Read the policy per project and grade it against what was actually called.
A CMEK key config is inert but assumed to be encrypting
attachment.type reads unattached, which the API describes as inert. Reconcile every external key config against the workspaces that actually name it.
Files failed to index and file_search quietly returns less
A vector_store.file carries last_error.code when parsing fails. The store's status still reads completed, because completed only means nothing is pending.
An empty vector store is still named in vector_store_ids
file_search returns 200 with zero citations and the model answers anyway. Read back the store ids your app configures and refuse to boot on an empty one.
A vector store with expires_after deletes itself on a clock
expires_after counts down from last_active_at, so an idle index evaporates and takes its file objects with it. Read expires_at, not your own arithmetic.
Vector store bytes grow while nobody queries the index
Storage is a stock, not a flow: it bills by the hour whether or not anything searches it. Trend usage_bytes over 90 days against file search calls.
A project or workspace ceiling set below the org limit
Anthropic returns each workspace override with org_limit beside it. OpenAI returns none, so peer projects stand in. Either way one container is throttled.
429s while every minute sits under the configured limit
Adjacent one-minute buckets show a steep ramp whose peak is far below every ceiling. The acceleration limit refused you, not the number in the tier table.
The gateway strips the header your backoff depends on
retry-after exists only on a 429. Probe the headers that arrive on every response, through the gateway and around it: if they die in transit, so does it.
The flex tier fails by not being served, and bills nothing
A 429 Resource Unavailable is not charged, so it never reaches the usage report. Group completions usage by service_tier and read the hours that are missing.
Anthropic-version is missing, ancient, or added in transit
Probe GET /v1/models three ways: no version header, 2023-06-01, 2023-01-01. Repeat through your gateway, because a header added in transit hides the fault.
The anthropic-beta value that 400s, and the one that went GA
GET /v1/models validates any anthropic-beta string for free. Loop every value your code sends, then diff one GET with and without it to find the GA ones.
Model visible, streaming refused: the org is unverified
The model resolves 200 while one key on it bills requests with zero tokens and a sibling key does not. That pairing is verification rather than access.
The same key works on your laptop and 403s in production
Run GET /v1/models from the production egress path and compare it with the same call from a known-good host. 403 here and 200 there is geography, not the key.
The Assistants API is shut down. Is yours still answering?
Past a shutdown date a 404 is the expected answer and a 200 is the finding. Probe /v1/assistants against a control path, then date the outage.
The Videos API closes and no successor model is listed
Sora 2 and /v1/videos shut down 2026-09-24 with an empty replacement column. Two clocks to read: the endpoint's date and each asset's own expires_at.
Prompts, Evals and Agent Builder close: export, not rewrite
Three surfaces close on 2026-11-30 and the content lives on OpenAI's side. Grade what the API can still export, and hand the rest to a person.
Fine-tuning stops taking new jobs while old ones keep serving
Two verbs, two deadlines. Whether you can still create a job depends on your own last 60 days of fine-tuned inference, and that is readable.
The same body counts 30% more tokens on the newer model
Claude 4.7 and later use a newer tokenizer. Call count_tokens twice on one identical body under two model ids and re-baseline every budget on the ratio.
System_fingerprint moved and seed stopped reproducing
Read system_fingerprint off the chat completions you already stored. Two values for one model void every seed-keyed cache and baseline spanning that day.
Previous_response_id 404s once the parent has aged out
Response objects are saved 30 days by default. Walk each recorded chain with GET /v1/responses/{id} to find the links already gone and the ones days away.
The fine-tuning job failed and error.code was never read
Job creation returns 200 and fails hours later. List the jobs, flag status failed, and print error.code, error.param and the events feed that explains them.
The batch failed validation and named the broken line
A batch that reads failed never reached the model. errors.data[] carries the code, the param and the line number of the input .jsonl it stopped on.
Cancelling a batch does not unbill the rows it already ran
A cancelled batch holds finished, paid-for output. Read completed vs total on OpenAI and succeeded vs canceled on Anthropic before you re-run the whole thing.
The background response is queued and nothing is polling
background: true returns 200 immediately and puts the result only on the response object. Four of the six status values are not success. Poll your ids.
The batch finished and nobody ever collected the output
Join output_file_id and results_url against your ingest ledger. Results expire 30 days after an OpenAI batch completes, 29 after a Claude batch is created.
The batch queue is full, so the next submission is refused
Sum request_counts.processing across live batches and compare it with the org's enqueued_batch_requests ceiling. A full queue refuses new submissions.
Files pile up against a storage ceiling nothing reports
No endpoint returns your file quota. Sum bytes across every page of /v1/files, group by purpose, and grade the total against the documented limit.
Purpose=assistants files outlived the API that owned them
The Assistants API is gone and its files are not. List both assistants purposes, subtract every id a surviving vector store holds, grade the remainder.
An expired file still answers metadata and fails every use
Content stops being retrievable at expires_at while metadata answers for 30 more days. Check the ids your app holds, up to 100 per request, and read the date.
Every response is stored and no endpoint will list them
store defaults to true and retention is at least 30 days, a floor not a deadline. Neither collection can be listed, so probe the ids you recorded.
These are the ones I keep hitting. If yours is broken in a way none of them describes, tell me on LinkedIn — it is usually how the next note gets written.