Diagnostic LLM APIs
a model id in use is past its published shutdown date
Nothing was deployed. The key did not rotate. One model id started returning 404 on the same morning for every request, and the message reads exactly like a typo: The model does not exist or you do not have access to it. It existed yesterday. There is no distinct error code for a retired model, no deprecation warning on the successful calls that came before it, and nothing in the response that tells the difference between a model that was shut down and a model name somebody misspelled.
GET /v1/models returns a shutdown_date on each entry in data[]. Any id whose shutdown_date is non-null and earlier than today is already dead, and every call naming it is already failing.
That one field is the whole check. It needs a project key set to Read Only, it is a single request, and it answers the question the 404 refuses to: this id was retired on a published date, it was not mistyped.
The problem in plain words
The damage is not that the model went away — that was announced months earlier. The damage is that the failure is ambiguous. A 404 with model_not_found is the same response you get for a genuine typo, for a model your organization has never been granted, and for a model that exists only on a partner platform. So the first hour goes into checking the spelling, then the key, then the project, then whether somebody changed an environment variable. The one explanation nobody reaches for is that the string is correct and the model is gone.
It also arrives all at once. Retirement is not a ramp: the id routes normally until the shutdown date and then stops, so every code path naming it breaks in the same minute. A fallback branch that names the same retired snapshot fails with it, which is how a graceful degradation path turns into a second outage.
Why it happens
The retirement is a date, not an event. OpenAI publishes shutdown dates on the deprecations page, usually three to six months ahead. Nothing in the API pushes that at you: successful inference responses carry no deprecation header and no warnings array, so a service can run for months against an id with a shutdown date already published and get no runtime hint at all.
The 404 is shared with three other causes. model_not_found means "not routable for you", which covers retired, misspelled, never-granted and wrong-endpoint. The error body cannot distinguish them because it does not know which one applies. Only the models list can, and only if you read it.
Pinning a snapshot does not exempt you. Pinning is the correct thing to do and it is exactly what gets bitten here: a dated snapshot has a fixed lifetime by construction. Teams that took the advice and pinned are the ones holding an id with a real shutdown date, while teams on a floating alias were quietly migrated for them — and got a different problem in exchange.
The list entry outlives the model, and then it does not. Immediately after a shutdown the entry can still appear in data[] with the date in the past, which is what makes this check possible. Once the entry is dropped from the list entirely there is no date left to read, and the only evidence is absence. Catching it while the date is still readable is much cheaper than reconstructing it afterwards.
The fix, as a flow
The script compares one field against one date, and the reason that is worth writing down is the ambiguity it removes: the 404 alone cannot tell a retired model from a typo, and the models list can.
How to fix it
Read the models list once
GET https://api.openai.com/v1/models with a project key set to Read Only. The response is a single page of data[] entries; the field that matters is shutdown_date, and on most entries it is null.
Compare each date against today, not against a hardcoded year
A date in the past means the id is dead now. A date of today means it dies during today, which is an outage in progress rather than a warning. Everything else is future work and belongs in the 90-day check instead, or the report is too noisy to read.
Restrict the output to ids you actually name
The list carries every model the key can see, including families you have never called. Pass the ids that appear in your configuration with --model so the report is about your code rather than about OpenAI's catalogue. With an admin-read key you can get that list from usage instead: GET /v1/organization/usage/completions?bucket_width=1d&group_by[]=model and read data[].results[].model.
Find every place the string is written, including the fallbacks
Grep the whole tree, not just the main call path. The id hides in default arguments, in a retry branch that picks a cheaper model, in a batch request body, in an infrastructure variable and in a test fixture. A migration that misses the fallback branch converts one outage into two.
Replace with a pinned successor, then diary the new date
Take the replacement from the deprecations page and pin it. Then read shutdown_date on the id you just pinned and put it in a calendar, because the successor has one too. A migration that lands on a floating alias to avoid this has traded a dated failure for an undated one.
How to check it worked
Re-run the script. Nothing should be reported as retired, and every id you passed should come back with either a future date or none.
python3 openai_model_shutdown_audit.py --model gpt-5.6-sol --model gpt-5.6-terra
# 2 model id(s) checked, 0 past their shutdown dateThe full code
One GET and no writes: a project key set to Read Only is enough, and is what this should hold. The classifier is pure and takes the date to compare against as an argument, because a rule about whether a day has passed is only testable if the day can be fixed — and because the interesting case, a shutdown date that lands on today, exists for exactly 24 hours a year and will never show up in a test that uses the real clock.
"""Report OpenAI model ids whose published shutdown date has already passed.
Read only. One GET request, no writes: give this a project key set to Read Only.
The repair is printed, never performed, because this script holds a credential
that can spend real money on inference.
"""
import argparse
import datetime as dt
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("openai_model_shutdown_audit")
API = "https://api.openai.com/v1"
# Printed beside a dead id so the reader is not sent back to the deprecations
# page for the obvious part. Matched longest prefix first, and deliberately
# family-level: this says where a line went, not that any one snapshot is a
# drop-in replacement for another.
SUCCESSORS = (
("gpt-image-1", "gpt-image-2"),
("chatgpt-image", "gpt-image-2"),
("dall-e", "gpt-image-2"),
("gpt-5-nano", "gpt-5.6-luna"),
("gpt-5-mini", "gpt-5.6-terra"),
("gpt-5-pro", "gpt-5.6-sol"),
("gpt-5", "gpt-5.6-sol"),
("o4-mini", "gpt-5.6-terra"),
("o3-pro", "gpt-5.6-sol"),
("o3", "gpt-5.6-sol"),
("o1", "gpt-5.6-sol"),
("gpt-4", "gpt-5.6-sol"),
)
FAILING = ("retired", "retiring-today")
def successor(model_id):
"""The family a retired id was folded into, or None if this script has no
opinion. An unknown id is left without a suggestion rather than pointed at
a guess."""
for prefix, replacement in SUCCESSORS:
if model_id.startswith(prefix):
return replacement
return None
def parse_day(value):
"""Read a shutdown_date into a date, or None when it cannot be read.
The field is a plain YYYY-MM-DD string. A full timestamp is tolerated by
taking the date part. Anything else returns None rather than a guess,
because a guess here either invents an outage or hides one.
"""
raw = str(value or "").strip()
if not raw:
return None
try:
return dt.date.fromisoformat(raw.split("T")[0])
except ValueError:
return None
def verdict(model, today):
"""Classify one entry from GET /v1/models against a date you pass in.
Pure, so the boundary cases can be tested at a fixed date instead of at
whatever day the suite happens to run. Returns (state, detail).
"""
model_id = str(model.get("id") or "").strip()
if not model_id:
return ("unreadable", "entry has no id field")
raw = model.get("shutdown_date")
if raw is None or str(raw).strip() == "":
return ("open",
"no shutdown date published. That is the current state of the "
"field, not a guarantee: re-read it on a schedule.")
day = parse_day(raw)
if day is None:
return ("unreadable-date",
"shutdown_date is %r, which is not a date this script will "
"guess at. Check it by hand." % (raw,))
days = (day - today).days
if days < 0:
return ("retired",
"shut down on %s, %d day(s) ago. Calls naming this id return "
"404 model_not_found, which is the same error a misspelled "
"model name returns." % (day.isoformat(), -days))
if days == 0:
return ("retiring-today",
"shuts down today (%s). Requests may already be failing; treat "
"this as an outage in progress, not a warning."
% (day.isoformat(),))
return ("scheduled",
"shuts down on %s, %d day(s) from now. Still routable today."
% (day.isoformat(), days))
def get(session, path):
r = session.get(API + path, timeout=30)
if r.status_code == 401:
raise SystemExit("401 from OpenAI: the key is wrong, revoked, or belongs "
"to another organization")
r.raise_for_status()
return r.json()
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--model", action="append", default=[],
help="only report this id; repeatable. Pass the ids your "
"code actually names to keep the report about you")
ap.add_argument("--show-all", action="store_true",
help="also print ids that are fine")
args = ap.parse_args()
key = os.environ.get("OPENAI_API_KEY")
if not key:
log.error("set OPENAI_API_KEY (a project key set to Read Only)")
return 2
session = requests.Session()
session.headers.update({"Authorization": "Bearer " + key})
models = get(session, "/models").get("data", [])
if not models:
log.info("the models list came back empty for this key")
return 0
wanted = set(args.model)
if wanted:
listed = {str(m.get("id") or "") for m in models}
for missing in sorted(wanted - listed):
log.warning("%-15s %s not in the models list at all, so there is no "
"shutdown_date left to read. An id that has been dropped "
"from the list is already gone.", "absent", missing)
models = [m for m in models if str(m.get("id") or "") in wanted]
today = dt.date.today()
bad = 0
for model in sorted(models, key=lambda m: str(m.get("id") or "")):
state, detail = verdict(model, today)
model_id = str(model.get("id") or "?")
line = "%-15s %s %s" % (state, model_id, detail)
if state in FAILING:
bad += 1
log.warning(line)
replacement = successor(model_id)
if replacement:
log.warning(" repair: change model=%r to model=%r at every call "
"site, then read shutdown_date on the new id",
model_id, replacement)
else:
log.warning(" repair: take the replacement from the "
"deprecations page and pin it")
elif state in ("unreadable", "unreadable-date"):
log.warning(line)
elif args.show_all or state == "scheduled":
log.info(line)
log.info("%d model id(s) checked, %d past their shutdown date",
len(models), bad)
return 1 if bad else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Report OpenAI model ids whose published shutdown date has already passed.
*
* Read only. One GET request, no writes: give this a project key set to Read
* Only. The repair is printed, never performed.
*/
const API = 'https://api.openai.com/v1';
// Matched longest prefix first, and deliberately family-level: this says where a
// line went, not that any one snapshot is a drop-in replacement for another.
const SUCCESSORS = [
['gpt-image-1', 'gpt-image-2'],
['chatgpt-image', 'gpt-image-2'],
['dall-e', 'gpt-image-2'],
['gpt-5-nano', 'gpt-5.6-luna'],
['gpt-5-mini', 'gpt-5.6-terra'],
['gpt-5-pro', 'gpt-5.6-sol'],
['gpt-5', 'gpt-5.6-sol'],
['o4-mini', 'gpt-5.6-terra'],
['o3-pro', 'gpt-5.6-sol'],
['o3', 'gpt-5.6-sol'],
['o1', 'gpt-5.6-sol'],
['gpt-4', 'gpt-5.6-sol'],
];
const FAILING = ['retired', 'retiring-today'];
/** The family a retired id was folded into, or null if this script has no opinion. */
export function successor(modelId) {
for (const [prefix, replacement] of SUCCESSORS) {
if (modelId.startsWith(prefix)) return replacement;
}
return null;
}
/**
* Read a shutdown_date into a UTC date, or null when it cannot be read. The
* field is a plain YYYY-MM-DD string; a full timestamp is tolerated by taking
* the date part. Anything else returns null rather than a guess, because a
* guess here either invents an outage or hides one.
*/
export function parseDay(value) {
const raw = String(value ?? '').trim().split('T')[0];
if (!/^\d{4}-\d{2}-\d{2}$/.test(raw)) return null;
const ms = Date.parse(`${raw}T00:00:00Z`);
return Number.isNaN(ms) ? null : new Date(ms);
}
const DAY = 86400000;
/**
* Classify one entry from GET /v1/models against a date you pass in. Pure, so
* the boundary cases can be tested at a fixed date instead of at whatever day
* the suite happens to run. Returns [state, detail].
*/
export function verdict(model, today) {
const modelId = String(model.id ?? '').trim();
if (!modelId) return ['unreadable', 'entry has no id field'];
const raw = model.shutdown_date;
if (raw === null || raw === undefined || String(raw).trim() === '') {
return ['open',
'no shutdown date published. That is the current state of the field, ' +
'not a guarantee: re-read it on a schedule.'];
}
const day = parseDay(raw);
if (day === null) {
return ['unreadable-date',
`shutdown_date is ${JSON.stringify(raw)}, which is not a date this ` +
'script will guess at. Check it by hand.'];
}
const iso = day.toISOString().slice(0, 10);
const days = Math.round((day.getTime() - today.getTime()) / DAY);
if (days < 0) {
return ['retired',
`shut down on ${iso}, ${-days} day(s) ago. Calls naming this id return ` +
'404 model_not_found, which is the same error a misspelled model name returns.'];
}
if (days === 0) {
return ['retiring-today',
`shuts down today (${iso}). Requests may already be failing; treat this ` +
'as an outage in progress, not a warning.'];
}
return ['scheduled',
`shuts down on ${iso}, ${days} day(s) from now. Still routable today.`];
}
async function get(key, path) {
const res = await fetch(API + path, {
headers: { Authorization: `Bearer ${key}` },
});
if (res.status === 401) {
throw new Error('401 from OpenAI: the key is wrong, revoked, or belongs to ' +
'another organization');
}
if (!res.ok) throw new Error(`${res.status} from ${path}`);
return res.json();
}
async function main() {
const key = process.env.OPENAI_API_KEY;
if (!key) {
console.error('set OPENAI_API_KEY (a project key set to Read Only)');
process.exitCode = 2;
return;
}
const wanted = new Set(process.argv.reduce((acc, arg, i) => (
arg === '--model' && process.argv[i + 1] ? [...acc, process.argv[i + 1]] : acc
), []));
const showAll = process.argv.includes('--show-all');
const { data = [] } = await get(key, '/models');
if (data.length === 0) {
console.log('the models list came back empty for this key');
return;
}
let models = data;
if (wanted.size > 0) {
const listed = new Set(data.map((m) => String(m.id ?? '')));
for (const missing of [...wanted].filter((m) => !listed.has(m)).sort()) {
console.warn(`${'absent'.padEnd(15)} ${missing} not in the models list at ` +
'all, so there is no shutdown_date left to read. An id that has been ' +
'dropped from the list is already gone.');
}
models = data.filter((m) => wanted.has(String(m.id ?? '')));
}
const today = new Date(`${new Date().toISOString().slice(0, 10)}T00:00:00Z`);
let bad = 0;
for (const model of [...models].sort((a, b) =>
String(a.id ?? '').localeCompare(String(b.id ?? '')))) {
const [state, detail] = verdict(model, today);
const modelId = String(model.id ?? '?');
const line = `${state.padEnd(15)} ${modelId} ${detail}`;
if (FAILING.includes(state)) {
bad += 1;
console.warn(line);
const replacement = successor(modelId);
console.warn(replacement
? ` repair: change model="${modelId}" to model="${replacement}" at ` +
'every call site, then read shutdown_date on the new id'
: ' repair: take the replacement from the deprecations page and pin it');
} else if (state === 'unreadable' || state === 'unreadable-date') {
console.warn(line);
} else if (showAll || state === 'scheduled') {
console.log(line);
}
}
console.log(`${models.length} model id(s) checked, ${bad} past their shutdown date`);
process.exitCode = bad ? 1 : 0;
}
// Only run when invoked directly. The test file imports this module, and without
// the guard main() would run there too, fail on the missing key, and set a
// non-zero exit code that fails the whole test file even as every test passes.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
Every test runs against a fixed date. The two that earn their place are the boundaries: a shutdown date of today is an outage happening now rather than a warning, and a null shutdown_date is the absence of a published date rather than a promise — collapsing either into its neighbour is how a check like this gets ignored or, worse, believed.
import datetime as dt
from openai_model_shutdown_audit import parse_day, successor, verdict
TODAY = dt.date(2026, 8, 30)
def test_shutdown_date_is_read_as_a_plain_day():
assert parse_day("2026-12-11") == dt.date(2026, 12, 11)
assert parse_day("2026-12-11T00:00:00Z") == dt.date(2026, 12, 11)
assert parse_day("") is None
assert parse_day(None) is None
assert parse_day("December 2026") is None
def test_a_date_already_passed_is_retired():
state, detail = verdict({"id": "gpt-4-turbo", "shutdown_date": "2026-06-15"},
TODAY)
assert state == "retired"
assert "76 day(s) ago" in detail
assert "misspelled" in detail
def test_a_shutdown_date_of_today_is_its_own_state():
# The whole point of the note: this is happening now, not soon.
state, detail = verdict({"id": "gpt-5-2025-08-07", "shutdown_date": "2026-08-30"},
TODAY)
assert state == "retiring-today"
assert "outage in progress" in detail
def test_a_future_date_belongs_to_the_other_note():
state, detail = verdict({"id": "gpt-5-2025-08-07", "shutdown_date": "2026-12-11"},
TODAY)
assert state == "scheduled"
assert "103 day(s)" in detail
def test_no_shutdown_date_is_not_a_promise():
state, detail = verdict({"id": "gpt-5.6-sol", "shutdown_date": None}, TODAY)
assert state == "open"
assert "not a guarantee" in detail
assert verdict({"id": "gpt-5.6-sol"}, TODAY)[0] == "open"
def test_an_unreadable_date_is_not_silently_healthy():
assert verdict({"id": "x", "shutdown_date": "soon"}, TODAY)[0] == "unreadable-date"
assert verdict({"shutdown_date": "2026-01-01"}, TODAY)[0] == "unreadable"
def test_the_successor_is_family_level_and_admits_ignorance():
assert successor("gpt-5-mini-2025-08-07") == "gpt-5.6-terra"
assert successor("gpt-5-2025-08-07") == "gpt-5.6-sol"
assert successor("dall-e-3") == "gpt-image-2"
assert successor("some-vendor-model") is None
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { parseDay, successor, verdict } from './openai-model-shutdown-audit.mjs';
const TODAY = new Date('2026-08-30T00:00:00Z');
test('shutdown_date is read as a plain day', () => {
assert.equal(parseDay('2026-12-11').toISOString().slice(0, 10), '2026-12-11');
assert.equal(parseDay('2026-12-11T00:00:00Z').toISOString().slice(0, 10),
'2026-12-11');
assert.equal(parseDay(''), null);
assert.equal(parseDay(null), null);
assert.equal(parseDay('December 2026'), null);
});
test('a date already passed is retired', () => {
const [state, detail] = verdict(
{ id: 'gpt-4-turbo', shutdown_date: '2026-06-15' }, TODAY);
assert.equal(state, 'retired');
assert.match(detail, /76 day\(s\) ago/);
assert.match(detail, /misspelled/);
});
test('a shutdown date of today is its own state', () => {
const [state, detail] = verdict(
{ id: 'gpt-5-2025-08-07', shutdown_date: '2026-08-30' }, TODAY);
assert.equal(state, 'retiring-today');
assert.match(detail, /outage in progress/);
});
test('a future date belongs to the other note', () => {
const [state, detail] = verdict(
{ id: 'gpt-5-2025-08-07', shutdown_date: '2026-12-11' }, TODAY);
assert.equal(state, 'scheduled');
assert.match(detail, /103 day\(s\)/);
});
test('no shutdown date is not a promise', () => {
const [state, detail] = verdict({ id: 'gpt-5.6-sol', shutdown_date: null }, TODAY);
assert.equal(state, 'open');
assert.match(detail, /not a guarantee/);
assert.equal(verdict({ id: 'gpt-5.6-sol' }, TODAY)[0], 'open');
});
test('an unreadable date is not silently healthy', () => {
assert.equal(verdict({ id: 'x', shutdown_date: 'soon' }, TODAY)[0],
'unreadable-date');
assert.equal(verdict({ shutdown_date: '2026-01-01' }, TODAY)[0], 'unreadable');
});
test('the successor is family level and admits ignorance', () => {
assert.equal(successor('gpt-5-mini-2025-08-07'), 'gpt-5.6-terra');
assert.equal(successor('gpt-5-2025-08-07'), 'gpt-5.6-sol');
assert.equal(successor('dall-e-3'), 'gpt-image-2');
assert.equal(successor('some-vendor-model'), null);
});
FAQ
How do I tell a retired model from a typo, when both return the same 404?
By reading GET /v1/models. The error body cannot tell them apart because it does not know which applies, but the list can: an entry with a shutdown_date in the past was retired on a published date, while a string that never appears in the list at any point was never a model this key could call.
Does OpenAI warn me before the shutdown date on a successful call?
No. Successful inference responses carry no deprecation header and no warnings array. The notice is published on the deprecations page, and the machine-readable form of it is the shutdown_date field on the models list. If nothing reads that field, nothing warns you.
A model has no shutdown_date at all. Is it safe?
It is unscheduled, which is not the same as permanent. OpenAI typically publishes three to six months of notice, so a null today can be a date tomorrow. The value of the check is that it is one request, so running it weekly costs nothing and turns the announcement into something your pipeline sees.
Should I switch to an unpinned alias so this never happens again?
That trades a dated failure for an undated one. An alias is repointed at new weights without notice, so the model changes underneath you with no deploy and no error; the symptoms are drifting evals and token counts rather than a 404. Pinning plus a scheduled read of shutdown_date is the combination that gives you both stability and warning.
Does Anthropic have the same field?
No. Anthropic's model object carries created_at, max_input_tokens and max_output_tokens but no retirement date, so on that side the date has to come from the published deprecation table and the API can only tell you whether the id is still callable at all. That is why the Anthropic check in this section is built on the id disappearing from the list rather than on a date.
Related field notes
- A model still in production retiring in under 90 days
- A retired model id still sitting in the code
- A floating alias where a pinned snapshot belongs
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.