Field notes
LLM API fix guides
An LLM API bills you whether or not the answer was any good, and most of the ways it goes wrong do not raise an exception. A model id retires on a date nobody diaried, a response is cut off mid-JSON because nobody checked finish_reason, a retry storm triples the invoice. Each note here explains one such problem and gives you a script that finds it through the API.
A script here holds a key that can spend real money on inference. So these read, they tell you exactly what is wrong, and they print the repair: the endpoint, the parameter, the model id to migrate to. You run it.
Several of them want an admin or organization-scoped read key rather than a project key, because usage and cost live on the organization. Each note says which it needs.
OpenAI and Anthropic
A model id in use is past its published shutdown date
GET /v1/models carries a shutdown_date per model. Once it passes, the id fails exactly like a typo: 404 model_not_found, with no retired error code.
A model you still call retires in under 90 days
The shutdown_date is real but still ahead of you. One GET turns that field into a migration schedule, ordered by days left and by the traffic each id carries.
A retired model id still sitting in the code
Anthropic drops a retired id from the models list entirely, with no date left to read. You find it by diffing your config strings against GET /v1/models.
A floating model alias silently changes model under you
An alias is not retiring, it moves. GET /v1/models/{alias} returns the snapshot it resolves to today, which is the id that should have been in the config.
429 credit_balance_exhausted retried forever as a rate limit
A 429 carrying insufficient_quota is a billing wall, not a throttle. Every SDK raises RateLimitError, so the retry loop hammers it until someone notices.
No hard spend limit is set, so the bill has no ceiling
OpenAI's hard spend limit is opt-in and lives on its own admin endpoint. Read it, the alerts, and month-to-date spend to see if anything stops a runaway.
Reasoning tokens are billed as output but never returned
Cost per request jumps after a model switch while the visible answers stay the same length. The reasoning tokens are billed as output and never returned.
Output tokens, not input, are what the bill is made of
Output is priced at five times input on every current Claude model, and thinking tokens bill as output. The cost report says which side the bill sits on.
Keys still work after their owner loses project access
Removing someone from a project revokes their console access, not their API key. One admin call lists the keys that are still live and still billing.
An archived project still holds live API keys
Archiving hides a project from the default listing without revoking anything inside it. Without include_archived=true your key audit never sees those keys.
Prompt caching was never switched on anywhere
cache_read_input_tokens is flat zero across every bucket while uncached input climbs. Caching is opt-in, and nothing tells you it is off.
Cache writes are paid for and never read back
A cache write costs 1.25x or 2x base input and a read costs 0.1x. When the reads never come, caching costs strictly more than leaving it off.
A batch reads completed while some of its rows failed
GET /v1/batches returns completed while request_counts.failed is non-zero. Completed means the run finished, not that every request inside it succeeded.
The batch left an error_file_id that nothing ever fetched
A completed batch carries a non-null error_file_id and no code ever called /content. The failures were written down and the downstream table is short.
A batch expired when the 24 hour completion window closed
status is expired and request_counts.completed is below total. The fixed 24h window closed on rows that had not run, and no HTTP error was raised.
Scheduled jobs pay full price for work the Batch API halves
group_by=batch returns one row, batch:false. Nightly enrichment and backfills go through the synchronous endpoint at roughly twice the batch price.
Fast mode billed at twice the rate and served as default
The tier you request and the tier you are served are different fields. Read the project setting against the invoice: both directions of the mismatch cost money.
Streamed responses report no usage and the dashboard undercounts
usage is null on every streamed chunk unless you ask for the totals. Reconcile the org token report against your own telemetry to size the hole.
Spend jumped week over week and no release explains it
Eight weeks of daily cost folded into whole weeks. The shape of the change is the finding: a spike, a step and a ramp want three different people.
One line item or project is most of the organization's bill
Ungrouped, the cost report is one opaque number per day. Grouped by line_item and project_id it is usually one row carrying most of the total.
A frontier model is answering twenty-token questions
Divide output_tokens by num_model_requests on the usage endpoint. A premium model whose mean answer is twenty tokens is the wrong size for the work.
Per-customer cost is unknowable because tenants share a key
user_id on the Usage API names your own org members and service accounts, never your end users. Resolve every principal and the gap becomes provable.
Audio and image usage never shows up in a token dashboard
Speech bills by characters, transcription by seconds, images by count. A dashboard built on usage/completions is structurally unable to see any of it.
A fine-tuned model was trained, billed, and never called once
Succeeded fine-tuning jobs name a model id. Thirty days of usage grouped by model shows zero requests against it. Training was paid for; inference never ran.
These are the ones I keep hitting. If yours is broken in a way none of them describes, tell me on LinkedIn — it is usually how the next note gets written.