Diagnostic LLM APIs
Prompts, Evals and Agent Builder close: export, not rewrite
The deprecation notice reads like the others and gets triaged like the others: three surfaces, one date, put it in the sprint after next. What is different does not appear anywhere in the notice. The reusable prompt referenced as pmpt_a1b2 in four services is four characters of your source code and several hundred words of somebody else's, and those words are on OpenAI's side of the line. On 1 December the code still compiles, the deploy still succeeds, and the prompt is not in the repository because it never was.
Treat this as an export problem, because the thing at risk is content rather than code. Reusable Prompts, the Evals platform and Agent Builder all close on 2026-11-30, and what disappears with them is stored material you did not write down: prompt versions, eval definitions and grader configuration, and published workflows.
The API reaches exactly one of the three. GET /v1/evals is documented, paginates on after, and returns the full eval object including data_source_config and testing_criteria — so the listing is the export, and one script can take the whole set.
Reusable prompts are the awkward one. There is no documented list endpoint for them anywhere in the API reference, so the script does not assert one: it probes GET /v1/prompts and grades whatever comes back, then falls back to the pmpt_ ids in your own call sites and probes each individually. That is a real difference from the other two surfaces and it changes the plan, because a set you cannot enumerate is a set you can only be sure about by grepping your tree.
Agent Builder has no REST surface at all. Not a closed one, not an undocumented one: there is nothing to call. The script says so on its own line and assigns that surface to a person working in the dashboard, rather than quietly leaving it out of a report that otherwise looks complete.
So the output is a plan with an owner against each row: what a script can take, what a script can take only for ids you already hold, and what somebody has to open a browser for. Then the code change, which is small and second: prompt={"id": "pmpt_a1b2", "version": "3"} becomes an instructions string you now have in the repository, eval suites move to Promptfoo, and Agent Builder flows become Agents SDK code. None of that is possible until the export is done, which is why the export is the note.
The problem in plain words
Announced 3 June 2026, closing 30 November 2026. Three things go at once and they are usually described together, which hides the fact that they fail differently. A prompt object is content. An eval suite is content plus configuration. An Agent Builder workflow is a published artefact with an id somebody else's code references. The only thing they share is a date and the direction of the loss.
The reusable prompt is the sharpest case because it is invisible in review. A call site reads prompt={"id": "pmpt_a1b2", "version": "3"}, which is a perfectly ordinary-looking line, and the several hundred words it stands for are stored server-side and versioned server-side. Nobody diffs them. Nobody has them in the repository. After the date, the same line is an invalid request against a prompt object that no longer resolves, and the text is not somewhere else — it is gone.
Evals are the surface people notice last and miss longest. An eval suite is not production traffic, so it is not on any dashboard anyone watches, and its absence does not break a deploy. It breaks the next time somebody wants to know whether a model change was safe, which is exactly when they cannot afford to reconstruct six months of graders from memory.
And the coverage is uneven in a way that a tidy report will conceal. One of the three enumerates cleanly, one has no documented listing at all, and one has no API. A script that reports on what it can reach and silently omits the rest produces a green summary for an organization that is going to lose an Agent Builder workflow on the last day of November.
Why it happens
The unit here is exportability, not validity. Every other note in this batch asks whether an endpoint still answers. This one asks whether the content behind it can be got out, which is a different question with a different answer per surface and a different owner per answer. A surface that answers 200 and holds nothing you need is fine; a surface that cannot be listed and holds your prompt text is the problem, and status codes alone do not sort those two.
An undocumented endpoint gets probed and reported, never asserted. The API reference index lists no path for reusable prompts. So the script issues the probe and prints the status it got, and the state it produces on a 404 is no list endpoint rather than gone — because those imply different next steps and only one of them is supported by the evidence. If the path does answer, the script says that too and the plan gets better.
A surface with no API is a finding with a name, not an omission. Agent Builder is graded without a request, because there is nothing to request. That row exists so the report cannot look complete while covering two thirds of the problem, and it is asserted in a test: passing a 200 for that surface still produces no-api-surface, so a stray status from somewhere else can never promote it to covered.
A structural fault is graded before the network, exactly as in the header notes. An id that does not begin pmpt_ is a configuration bug and needs no request to prove it. Probing it first would spend a call and, worse, bury a definite finding under a status code.
The listing is the export, and that is worth saying out loud. GET /v1/evals returns whole eval objects rather than stubs, so there is no second pass and no per-id fetch to get the definition. That is why the evals half of this is a solved problem and the prompts half is not, and stating the reason keeps somebody from writing the fetch loop that is not needed.
The repair is a two-step and the steps are not interchangeable. Export, then inline. Nobody can replace prompt={"id": ...} with an instructions string they do not have. The script prints the export commands first and the code change second, and the code change is the short part.
The fix, as a flow
Three things close on one date and the code change is the small half of the work. What is actually at risk is content held on the provider's side: prompt versions, graders, published workflows. So the useful question is not whether an endpoint answers but how far the API reaches into each surface, and the answer is different three times. One lists cleanly, one has no documented listing, and one has no endpoints at all.
How to fix it
Count the days, and read the date as the export deadline
2026-11-30. The script prints days remaining and treats the date as the last day the content is retrievable, not the last day the code works — those are the same date here, and the first one is the one with a queue behind it.
List the evals, which is the export
GET /v1/evals?limit=100, paginating on after. The response carries name, data_source_config and testing_criteria per eval, so the page itself is the material. Save it; there is no second call to make.
Probe the prompts path rather than assuming it
GET /v1/prompts?limit=1 and record the status. The API reference documents no listing for reusable prompts, so this is a probe with a reported result, and a 404 means ids must come from your own tree rather than that the content is already gone.
Declare the pmpt_ ids your code actually passes
OPENAI_PROMPT_IDS as a comma-separated list, or repeated --prompt-id. Take them from the deployed call sites. Anything that does not start pmpt_ is graded as a configuration bug without a request; the rest are probed individually.
Print the plan with an owner per surface
Three rows: what a script exports, what a script exports only by id, and what a person exports in the dashboard. Agent Builder is always the third. Then the export commands, and then the code change — in that order, because the second one is impossible before the first.
How to check it worked
Re-run after the export pass. The eval count should not move, because listing does not consume anything, and that is the point: the check is idempotent and the artefact lives in your repository now. What should change is the prompt roster, which shrinks as call sites move from prompt={"id": ...} to inline instructions and the declared id list gets shorter. The row that will never go green is Agent Builder, and it should not: no rerun of a script can close a surface that has no API.
OPENAI_PROMPT_IDS=pmpt_a1b2,pmpt_c3d4,promptx \
python3 sunset_export_audit.py
# three surfaces close 2026-11-30, 91 day(s) left
# evals 200 enumerable the listing answered, so these can be exported
# by script
# prompts 404 no-list-endpoint nothing answered at this path, so ids have to
# come from your own call sites
# agent-builder --- no-api-surface no documented REST endpoints exist, so nothing
# here can inventory or export it
# 12 eval(s) listed, and the listing carries the full definition
# curl -s -H "Authorization: Bearer $OPENAI_API_KEY" \
# https://api.openai.com/v1/evals?limit=100 > export/evals.json
# 3 declared prompt id(s)
# pmpt_a1b2 200 readable the stored content came back
# pmpt_c3d4 404 not-readable nothing answered, so its text comes out of the
# dashboard before the date
# promptx --- not-a-prompt-id reusable prompt ids start pmpt_, so this is a
# configuration bug and not an id
# plan
# evals a script one GET per page dumps the full objects
# prompts a script, by id probe the ids you hold; the rest is the dashboard
# agent-builder a person there is no endpoint, so nothing automates this
# 4 finding(s)The full code
One paginated listing, one probe, one probe per declared id, and five pure functions. days_left, arithmetic against the published date; surface_reach, which grades how far the API gets on one surface and returns no-api-surface for Agent Builder whatever status it is handed; prompt_id_state, which grades the shape of an id before it grades a response, so a string that is not a prompt id never costs a request; export_plan, the only function that turns reach into an owner, which is the output somebody can act on; and export_command, which builds the exact GET to run and is asserted by a test to be a read.
"""Audit three closing surfaces for what can still be exported, and by whom.
Read only. Every request is a GET: the evals listing, one probe of the prompts
path, and one probe per prompt id you declare. Nothing here creates an eval, a
run or a prompt version.
The unit is exportability rather than validity, because what closes on
2026-11-30 is content held on the provider's side. The three surfaces are not
equally reachable and the script refuses to hide that: evals list cleanly,
reusable prompts have no documented list endpoint so the path is probed rather
than assumed, and Agent Builder has no REST surface at all and is graded
without a request.
"""
import argparse
import datetime as dt
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("sunset_export_audit")
API = "https://api.openai.com/v1"
# Announced 3 June 2026. Reusable Prompts, the Evals dashboard and API, and
# Agent Builder all close on this date. Published, not readable.
SHUTDOWN = "2026-11-30"
# Checked against the API reference index: there is a documented listing for
# evals and none for reusable prompts, and Agent Builder has no REST endpoints
# at all. That asymmetry is the reason this script grades reach per surface
# instead of running one loop over three paths.
AGENT_BUILDER = "agent-builder"
FINDINGS = ("no-api-surface", "no-list-endpoint", "not-readable",
"not-a-prompt-id", "malformed", "credentials", "refused",
"unreachable", "content-to-export")
REPAIRS = {
"no-api-surface":
"there is no endpoint, so nothing here automates it. Somebody has to "
"open Agent Builder, export each published workflow, and rebuild it "
"with the Agents SDK before the date.",
"no-list-endpoint":
"the API reference documents no listing for reusable prompts, so the "
"authoritative roster is a grep of your own tree for pmpt_ ids. "
"Anything only a colleague remembers comes out of the dashboard.",
"not-readable":
"nothing answered for this id, so its text is not retrievable by "
"script. Copy it out of the dashboard and put it in the repository "
"before the date, because after it there is nowhere to copy from.",
"not-a-prompt-id":
"reusable prompt ids start pmpt_. Fix the configuration; this one was "
"never going to resolve, shutdown or no shutdown.",
"content-to-export":
"the listing carries the full definition, so one paginated GET is the "
"whole export. Save it into the repository, then migrate the suites "
"to Promptfoo.",
}
def days_left(today, when=SHUTDOWN):
"""Whole days from today to the date. Pure. Negative once it has passed."""
return (dt.date.fromisoformat(str(when))
- dt.date.fromisoformat(str(today))).days
def surface_reach(name, status):
"""How far the API gets on one surface. Pure. Returns (state, detail).
Agent Builder is graded without a request and cannot be promoted by one:
passing a 200 here still returns no-api-surface, because there is no path
that 200 could have come from. That is asserted in a test, so a stray
status from somewhere else can never make the report look complete.
"""
if str(name) == AGENT_BUILDER:
return ("no-api-surface",
"no documented REST endpoints exist, so nothing here can "
"inventory or export it")
if status is None:
return ("unreachable", "no response at all from this path")
status = int(status)
if status == 200:
return ("enumerable",
"the listing answered, so these can be exported by script")
if status == 404:
return ("no-list-endpoint",
"nothing answered at this path, so ids have to come from your "
"own call sites")
if status in (401, 403):
return ("credentials",
"%d, so the reach of this surface was not established" % status)
return ("refused", "%d, so the reach of this surface is unknown" % status)
def prompt_id_state(pid, status):
"""Grade one declared prompt id. Pure. Returns (state, detail).
Shape first, response second. An id that is not a prompt id is a bug in the
configuration and needs no request to prove it, and probing it anyway would
bury a definite finding underneath a status code.
"""
if not isinstance(pid, str) or not pid.strip():
return ("malformed",
"not a usable string, so this is a configuration bug rather "
"than an id")
pid = pid.strip()
if not pid.startswith("pmpt_"):
return ("not-a-prompt-id",
"reusable prompt ids start pmpt_, so this is something else")
if status is None:
return ("not-probed", "no request was made for this id")
status = int(status)
if status == 200:
return ("readable", "the stored content came back")
if status == 404:
return ("not-readable",
"nothing answered, so its text comes out of the dashboard "
"before the date")
if status in (401, 403):
return ("credentials", "%d, which is the key and not the id" % status)
return ("refused", "%d" % status)
def export_plan(rows):
"""Turn reach into an owner per surface. Pure. [(name, owner, line)].
The output somebody can actually act on: three rows, three owners, and no
surface silently missing from the report.
"""
plan = []
for name, state in rows or []:
if state == "enumerable":
plan.append((name, "a script",
"one GET per page dumps the full objects"))
elif state == "no-list-endpoint":
plan.append((name, "a script, by id",
"probe the ids you hold; the rest is the dashboard"))
elif state == "no-api-surface":
plan.append((name, "a person",
"there is no endpoint, so nothing automates this"))
else:
plan.append((name, "a person, until proven otherwise",
"the reach could not be established, so assume the "
"dashboard"))
return plan
def export_command(kind, ident=None):
"""The exact GET to run for one export. Pure. Printed, never performed."""
auth = '-H "Authorization: Bearer $OPENAI_API_KEY"'
if kind == "evals":
return ("curl -s %s %s/evals?limit=100 > export/evals.json"
% (auth, API))
if kind == "prompt":
return ("curl -s %s %s/prompts/%s > export/%s.json"
% (auth, API, ident, ident))
return ""
def repair_lines(state):
"""The repair for one verdict. Pure. Printed, never performed."""
line = REPAIRS.get(state)
if not line:
return []
if state in ("no-list-endpoint", "not-readable"):
return [line,
"then inline it: prompt={id: pmpt_...} becomes an instructions "
"string you hold, which is the short half of this job and the "
"half that is impossible before the export."]
return [line]
def get_json(session, path, key, params=None, timeout=30):
"""One GET. Returns (status, parsed body). Never raises on a 4xx."""
try:
r = session.get(API + path,
headers={"Authorization": "Bearer " + key},
params=params or {}, timeout=timeout)
except requests.RequestException as exc:
log.debug("GET %s failed: %s", path, exc)
return (None, {})
try:
return (r.status_code, r.json())
except ValueError:
return (r.status_code, {})
def all_evals(session, key, pages=50):
"""Walk GET /v1/evals to the end. Returns (status, [eval objects]).
The listing carries data_source_config and testing_criteria, so the page is
the export and there is no per-id fetch to write.
"""
out, after, first = [], None, None
for _ in range(pages):
params = {"limit": 100, "order": "asc"}
if after:
params["after"] = after
status, body = get_json(session, "/evals", key, params)
if first is None:
first = status
if status != 200:
break
page = body.get("data") or []
out.extend(page)
if not page or not body.get("has_more"):
break
after = page[-1].get("id")
if not after:
break
return (first, out)
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--prompt-id", action="append", default=[],
help="a pmpt_ id your code passes (repeatable)")
ap.add_argument("--today", default=dt.date.today().isoformat(),
help="override the date the arithmetic is done against")
args = ap.parse_args()
key = os.environ.get("OPENAI_API_KEY")
if not key:
log.error("set OPENAI_API_KEY to a project read key. This script only "
"issues GET requests")
return 2
left = days_left(args.today)
log.info("three surfaces close %s, %d day(s) %s", SHUTDOWN, abs(left),
"left" if left >= 0 else "past")
session = requests.Session()
findings = 0
reach = []
eval_status, evals = all_evals(session, key)
prompt_status, _ = get_json(session, "/prompts", key, {"limit": 1})
probes = [("evals", eval_status), ("prompts", prompt_status),
(AGENT_BUILDER, None)]
for name, status in probes:
state, detail = surface_reach(name, status)
reach.append((name, state))
emit = log.warning if state in FINDINGS else log.info
emit(" %-14s %s %-17s %s", name,
"---" if status is None else status, state, detail)
for line in repair_lines(state):
emit(" repair: %s", line)
if state in FINDINGS:
findings += 1
if evals:
log.warning("%d eval(s) listed, and the listing carries the full "
"definition", len(evals))
log.warning(" %s", export_command("evals"))
for line in repair_lines("content-to-export"):
log.warning(" repair: %s", line)
findings += 1
declared = list(args.prompt_id)
declared += [p.strip() for p in
(os.environ.get("OPENAI_PROMPT_IDS") or "").split(",")
if p.strip()]
if declared:
log.info("%d declared prompt id(s)", len(declared))
for pid in declared:
text = pid.strip() if isinstance(pid, str) else pid
status = None
if isinstance(text, str) and text.startswith("pmpt_"):
status, _ = get_json(session, "/prompts/" + text, key)
state, detail = prompt_id_state(text, status)
emit = log.warning if state in FINDINGS else log.info
emit(" %-12s %s %-16s %s", text,
"---" if status is None else status, state, detail)
if state == "readable":
log.info(" %s", export_command("prompt", text))
for line in repair_lines(state):
emit(" repair: %s", line)
if state in FINDINGS:
findings += 1
log.info("plan")
for name, owner, line in export_plan(reach):
log.info(" %-14s %-28s %s", name, owner, line)
log.info("%d finding(s)", findings)
return 1 if findings else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Audit three closing surfaces for what can still be exported, and by whom.
*
* Read only. Every request is a GET: the evals listing, one probe of the
* prompts path, and one probe per declared prompt id. Nothing here creates an
* eval, a run or a prompt version.
*
* The unit is exportability rather than validity, because what closes on
* 2026-11-30 is content held on the provider's side. Evals list cleanly,
* reusable prompts have no documented list endpoint so the path is probed
* rather than assumed, and Agent Builder has no REST surface at all.
*/
export const API = 'https://api.openai.com/v1';
// Announced 3 June 2026. Published, not readable.
export const SHUTDOWN = '2026-11-30';
export const AGENT_BUILDER = 'agent-builder';
const FINDINGS = new Set(['no-api-surface', 'no-list-endpoint', 'not-readable',
'not-a-prompt-id', 'malformed', 'credentials', 'refused', 'unreachable',
'content-to-export']);
const REPAIRS = {
'no-api-surface':
'there is no endpoint, so nothing here automates it. Somebody has to open '
+ 'Agent Builder, export each published workflow, and rebuild it with the '
+ 'Agents SDK before the date.',
'no-list-endpoint':
'the API reference documents no listing for reusable prompts, so the '
+ 'authoritative roster is a grep of your own tree for pmpt_ ids. Anything '
+ 'only a colleague remembers comes out of the dashboard.',
'not-readable':
'nothing answered for this id, so its text is not retrievable by script. '
+ 'Copy it out of the dashboard and put it in the repository before the '
+ 'date, because after it there is nowhere to copy from.',
'not-a-prompt-id':
'reusable prompt ids start pmpt_. Fix the configuration; this one was never '
+ 'going to resolve, shutdown or no shutdown.',
'content-to-export':
'the listing carries the full definition, so one paginated GET is the whole '
+ 'export. Save it into the repository, then migrate the suites to Promptfoo.',
};
const day = (iso) => Date.parse(`${iso}T00:00:00Z`);
/** Whole days from today to the date. Pure. Negative once it has passed. */
export function daysLeft(today, when = SHUTDOWN) {
return Math.round((day(String(when)) - day(String(today))) / 86400000);
}
/** How far the API gets on one surface. Pure. [state, detail]. */
export function surfaceReach(name, status) {
if (String(name) === AGENT_BUILDER) {
return ['no-api-surface',
'no documented REST endpoints exist, so nothing here can inventory or export it'];
}
if (status === null || status === undefined) {
return ['unreachable', 'no response at all from this path'];
}
const s = Number(status);
if (s === 200) {
return ['enumerable', 'the listing answered, so these can be exported by script'];
}
if (s === 404) {
return ['no-list-endpoint',
'nothing answered at this path, so ids have to come from your own call sites'];
}
if (s === 401 || s === 403) {
return ['credentials', `${s}, so the reach of this surface was not established`];
}
return ['refused', `${s}, so the reach of this surface is unknown`];
}
/** Grade one declared prompt id. Pure. Shape first, response second. */
export function promptIdState(pid, status) {
if (typeof pid !== 'string' || !pid.trim()) {
return ['malformed',
'not a usable string, so this is a configuration bug rather than an id'];
}
const id = pid.trim();
if (!id.startsWith('pmpt_')) {
return ['not-a-prompt-id', 'reusable prompt ids start pmpt_, so this is something else'];
}
if (status === null || status === undefined) {
return ['not-probed', 'no request was made for this id'];
}
const s = Number(status);
if (s === 200) return ['readable', 'the stored content came back'];
if (s === 404) {
return ['not-readable',
'nothing answered, so its text comes out of the dashboard before the date'];
}
if (s === 401 || s === 403) return ['credentials', `${s}, which is the key and not the id`];
return ['refused', `${s}`];
}
/** Turn reach into an owner per surface. Pure. [[name, owner, line]]. */
export function exportPlan(rows) {
return (rows || []).map(([name, state]) => {
if (state === 'enumerable') {
return [name, 'a script', 'one GET per page dumps the full objects'];
}
if (state === 'no-list-endpoint') {
return [name, 'a script, by id', 'probe the ids you hold; the rest is the dashboard'];
}
if (state === 'no-api-surface') {
return [name, 'a person', 'there is no endpoint, so nothing automates this'];
}
return [name, 'a person, until proven otherwise',
'the reach could not be established, so assume the dashboard'];
});
}
/** The exact GET to run for one export. Pure. Printed, never performed. */
export function exportCommand(kind, ident = null) {
const auth = '-H "Authorization: Bearer $OPENAI_API_KEY"';
if (kind === 'evals') return `curl -s ${auth} ${API}/evals?limit=100 > export/evals.json`;
if (kind === 'prompt') {
return `curl -s ${auth} ${API}/prompts/${ident} > export/${ident}.json`;
}
return '';
}
/** The repair for one verdict. Pure. Printed, never performed. */
export function repairLines(state) {
const line = REPAIRS[state];
if (!line) return [];
if (state === 'no-list-endpoint' || state === 'not-readable') {
return [line,
'then inline it: prompt={id: pmpt_...} becomes an instructions string you '
+ 'hold, which is the short half of this job and the half that is '
+ 'impossible before the export.'];
}
return [line];
}
async function getJson(path, key, params = {}) {
const url = new URL(API + path);
for (const [k, v] of Object.entries(params)) {
for (const one of Array.isArray(v) ? v : [v]) url.searchParams.append(k, String(one));
}
try {
const r = await fetch(url, { headers: { Authorization: `Bearer ${key}` } });
let body = {};
try { body = await r.json(); } catch { body = {}; }
return [r.status, body];
} catch {
return [null, {}];
}
}
async function allEvals(key, pages = 50) {
const out = [];
let after = null;
let first = null;
for (let i = 0; i < pages; i += 1) {
const params = { limit: 100, order: 'asc' };
if (after) params.after = after;
const [status, body] = await getJson('/evals', key, params);
if (first === null) first = status;
if (status !== 200) break;
const page = body.data || [];
out.push(...page);
if (!page.length || !body.has_more) break;
after = page[page.length - 1].id;
if (!after) break;
}
return [first, out];
}
async function main() {
const key = process.env.OPENAI_API_KEY;
if (!key) {
console.error('set OPENAI_API_KEY to a project read key. This script only '
+ 'issues GET requests');
process.exitCode = 2;
return;
}
const today = process.env.TODAY || new Date().toISOString().slice(0, 10);
const left = daysLeft(today);
console.log(`three surfaces close ${SHUTDOWN}, ${Math.abs(left)} day(s) `
+ `${left >= 0 ? 'left' : 'past'}`);
let findings = 0;
const reach = [];
const [evalStatus, evals] = await allEvals(key);
const [promptStatus] = await getJson('/prompts', key, { limit: 1 });
for (const [name, status] of [['evals', evalStatus], ['prompts', promptStatus],
[AGENT_BUILDER, null]]) {
const [state, detail] = surfaceReach(name, status);
reach.push([name, state]);
console.log(` ${name.padEnd(14)} ${status ?? '---'} ${state.padEnd(17)} ${detail}`);
for (const line of repairLines(state)) console.log(` repair: ${line}`);
if (FINDINGS.has(state)) findings += 1;
}
if (evals.length) {
console.log(`${evals.length} eval(s) listed, and the listing carries the full definition`);
console.log(` ${exportCommand('evals')}`);
for (const line of repairLines('content-to-export')) console.log(` repair: ${line}`);
findings += 1;
}
const declared = (process.env.OPENAI_PROMPT_IDS ?? '')
.split(',').map((s) => s.trim()).filter(Boolean);
if (declared.length) console.log(`${declared.length} declared prompt id(s)`);
for (const pid of declared) {
let status = null;
if (pid.startsWith('pmpt_')) [status] = await getJson(`/prompts/${pid}`, key);
const [state, detail] = promptIdState(pid, status);
console.log(` ${pid.padEnd(12)} ${status ?? '---'} ${state.padEnd(16)} ${detail}`);
if (state === 'readable') console.log(` ${exportCommand('prompt', pid)}`);
for (const line of repairLines(state)) console.log(` repair: ${line}`);
if (FINDINGS.has(state)) findings += 1;
}
console.log('plan');
for (const [name, owner, line] of exportPlan(reach)) {
console.log(` ${name.padEnd(14)} ${owner.padEnd(28)} ${line}`);
}
console.log(`${findings} finding(s)`);
process.exitCode = findings ? 1 : 0;
}
if (import.meta.url === `file://${process.argv[1]}`) await main();
Add a test
The first test is the one that keeps the report honest: Agent Builder grades as no-api-surface even when a 200 is handed to it, because there is no path that 200 could have come from, and a surface with no API must never be promoted to covered by a stray status. The second is the correction this note was written around — a 404 on the prompts path is no-list-endpoint, not gone, and the detail sends you to your own call sites rather than to a conclusion the evidence does not support. Then the plan, asserted to put a person against exactly the surfaces a script cannot reach. Then the shape check, which catches an id that is not a prompt id with no request made at all. And finally the export command, asserted to be a read: a GET, with no write verb anywhere in it.
from sunset_export_audit import (AGENT_BUILDER, SHUTDOWN, days_left,
export_command, export_plan, prompt_id_state,
repair_lines, surface_reach)
TODAY = "2026-08-31"
def test_a_surface_with_no_api_is_never_promoted_by_a_stray_status():
# There is no path a 200 could have come from, so one must not make the
# report look complete. This is the whole reason the row exists.
for status in (None, 200, 404, 401):
state, detail = surface_reach(AGENT_BUILDER, status)
assert state == "no-api-surface"
assert "no documented REST endpoints" in detail
assert any("open Agent Builder" in line
for line in repair_lines("no-api-surface"))
def test_a_404_on_the_prompts_path_means_no_listing_and_not_gone():
# Those imply different next steps and only one is supported by the
# evidence: the API reference documents no listing for reusable prompts.
state, detail = surface_reach("prompts", 404)
assert state == "no-list-endpoint"
assert "your own call sites" in detail
assert "gone" not in detail
lines = repair_lines(state)
assert any("grep of your own tree" in line for line in lines)
assert any("impossible before the export" in line for line in lines)
def test_the_plan_puts_a_person_against_what_no_script_can_reach():
plan = export_plan([("evals", "enumerable"),
("prompts", "no-list-endpoint"),
(AGENT_BUILDER, "no-api-surface"),
("something", "credentials")])
owners = {name: owner for name, owner, _ in plan}
assert owners["evals"] == "a script"
assert owners["prompts"] == "a script, by id"
assert owners[AGENT_BUILDER] == "a person"
assert owners["something"].startswith("a person, until")
assert len(plan) == 4
def test_an_id_that_is_not_a_prompt_id_is_caught_without_a_request():
state, detail = prompt_id_state("promptx", None)
assert state == "not-a-prompt-id"
assert "start pmpt_" in detail
assert prompt_id_state("", None)[0] == "malformed"
assert prompt_id_state(None, 200)[0] == "malformed"
# A real id with no probe is honestly reported as not probed, which is a
# different thing from unreadable.
assert prompt_id_state("pmpt_a1b2", None)[0] == "not-probed"
def test_a_declared_id_is_graded_by_what_answered_for_it():
assert prompt_id_state("pmpt_a1b2", 200)[0] == "readable"
state, detail = prompt_id_state(" pmpt_c3d4 ", 404)
assert state == "not-readable"
assert "out of the dashboard" in detail
assert prompt_id_state("pmpt_c3d4", 401)[0] == "credentials"
assert prompt_id_state("pmpt_c3d4", 500)[0] == "refused"
def test_the_export_command_is_a_read():
line = export_command("evals")
assert line.startswith("curl -s ")
assert "/v1/evals?limit=100" in line
assert "$OPENAI_API_KEY" in line
assert "-X" not in line
assert export_command("prompt", "pmpt_a1b2").endswith("export/pmpt_a1b2.json")
assert export_command("agent-builder") == ""
def test_the_date_is_the_export_deadline_and_the_arithmetic_says_so():
assert days_left(TODAY) == 91
assert days_left("2026-11-30") == 0
assert days_left("2026-12-05") == -5
assert SHUTDOWN == "2026-11-30"
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { AGENT_BUILDER, SHUTDOWN, daysLeft, exportCommand, exportPlan,
promptIdState, repairLines, surfaceReach } from './sunset-export-audit.mjs';
const TODAY = '2026-08-31';
test('a surface with no api is never promoted by a stray status', () => {
for (const status of [null, 200, 404, 401]) {
const [state, detail] = surfaceReach(AGENT_BUILDER, status);
assert.equal(state, 'no-api-surface');
assert.ok(detail.includes('no documented REST endpoints'));
}
assert.ok(repairLines('no-api-surface').some((l) => l.includes('open Agent Builder')));
});
test('a 404 on the prompts path means no listing and not gone', () => {
const [state, detail] = surfaceReach('prompts', 404);
assert.equal(state, 'no-list-endpoint');
assert.ok(detail.includes('your own call sites'));
assert.ok(!detail.includes('gone'));
const lines = repairLines(state);
assert.ok(lines.some((l) => l.includes('grep of your own tree')));
assert.ok(lines.some((l) => l.includes('impossible before the export')));
});
test('the plan puts a person against what no script can reach', () => {
const plan = exportPlan([['evals', 'enumerable'],
['prompts', 'no-list-endpoint'],
[AGENT_BUILDER, 'no-api-surface'],
['something', 'credentials']]);
const owners = Object.fromEntries(plan.map(([n, o]) => [n, o]));
assert.equal(owners.evals, 'a script');
assert.equal(owners.prompts, 'a script, by id');
assert.equal(owners[AGENT_BUILDER], 'a person');
assert.ok(owners.something.startsWith('a person, until'));
assert.equal(plan.length, 4);
});
test('an id that is not a prompt id is caught without a request', () => {
const [state, detail] = promptIdState('promptx', null);
assert.equal(state, 'not-a-prompt-id');
assert.ok(detail.includes('start pmpt_'));
assert.equal(promptIdState('', null)[0], 'malformed');
assert.equal(promptIdState(null, 200)[0], 'malformed');
assert.equal(promptIdState('pmpt_a1b2', null)[0], 'not-probed');
});
test('a declared id is graded by what answered for it', () => {
assert.equal(promptIdState('pmpt_a1b2', 200)[0], 'readable');
const [state, detail] = promptIdState(' pmpt_c3d4 ', 404);
assert.equal(state, 'not-readable');
assert.ok(detail.includes('out of the dashboard'));
assert.equal(promptIdState('pmpt_c3d4', 401)[0], 'credentials');
assert.equal(promptIdState('pmpt_c3d4', 500)[0], 'refused');
});
test('the export command is a read', () => {
const line = exportCommand('evals');
assert.ok(line.startsWith('curl -s '));
assert.ok(line.includes('/v1/evals?limit=100'));
assert.ok(line.includes('$OPENAI_API_KEY'));
assert.ok(!line.includes('-X'));
assert.ok(exportCommand('prompt', 'pmpt_a1b2').endsWith('export/pmpt_a1b2.json'));
assert.equal(exportCommand('agent-builder'), '');
});
test('the date is the export deadline and the arithmetic says so', () => {
assert.equal(daysLeft(TODAY), 91);
assert.equal(daysLeft('2026-11-30'), 0);
assert.equal(daysLeft('2026-12-05'), -5);
assert.equal(SHUTDOWN, '2026-11-30');
});
FAQ
Can I list my reusable prompts through the API?
Not according to the reference. There is no documented endpoint for reusable prompts anywhere in the API reference index, which is why this script probes GET /v1/prompts and prints the status it got rather than asserting a listing exists. If the path answers, the script says so and your job gets easier. If it 404s, that is reported as no list endpoint rather than as gone, because the two mean different things: the content may be perfectly alive in the dashboard while being unenumerable from code. Either way the authoritative roster is a grep of your own tree for pmpt_ ids.
Why is Agent Builder in a report about API calls at all?
Because leaving it out is how a report about three surfaces becomes a green summary about two. Agent Builder has no REST endpoints, so the script grades it without a request and assigns it to a person in the dashboard. There is a test that this row cannot be promoted by a status code — hand it a 200 and it still returns no-api-surface, because there is no path that 200 could have come from.
Do I need to fetch each eval individually to export it?
No, and that is worth knowing before you write the loop. GET /v1/evals returns the full eval object per row, including name, data_source_config and testing_criteria, so the paginated listing is itself the export. Save the pages, put them in the repository, and migrate the suites to Promptfoo, which is the replacement OpenAI names. The prompts half of this job is the hard half precisely because it has no equivalent.
Is a stored prompt the same thing as prompt caching?
No, and confusing them is easy because both involve a prompt living somewhere other than your request. A reusable prompt is a content object with an id and versions that you reference instead of sending the text; prompt caching is a billing and latency mechanism for text you do send. This closure affects the first and not the second. Nothing here changes your cached share, and nothing about caching survives or replaces a pmpt_ id.
What actually breaks on 1 December if I do nothing?
Any call that passes a pmpt_ id to the Responses API fails as an invalid request, because the referenced object no longer resolves. The code does not stop compiling and the deploy does not fail; the request does. The worse loss is quieter: the prompt text and the eval definitions were never in your repository, so there is nothing to restore from. That is why the script orders the output as export first and code change second — the second is a few lines and is impossible without the first.
Related field notes
- The same closure one date earlier, and already past it
- Prompt caching, which is a different thing from a stored prompt
- The diary entry version, for ids rather than endpoints
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Deprecations — OpenAI platform docs
- List evals — OpenAI API reference
- Migrate to the Responses API — OpenAI platform docs
- Admin APIs — OpenAI platform docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.