Diagnostic LLM APIs
The background response is queued and nothing is polling
The queue worker takes a job, starts a background response, writes the id somewhere, and returns. It is a good design: the request is accepted in a few hundred milliseconds and the model can take twenty minutes without holding a socket open. Then the worker is redeployed mid-shift, or the process that was going to poll gets an unhandled exception on a different code path, and the ids stop being read. The jobs keep running. They keep billing. And the only place their results exist is on an object nobody is asking about.
Poll every id you hold. GET /v1/responses/{response_id} with a project key set to Read Only, one call per id, and bucket the answers by status. There is no list endpoint for /v1/responses, so the ids have to come from your own job table — which is the first thing the script checks, because if you cannot produce a list of ids then the jobs are already unreachable.
The status enum has six values and four of them are not success. A background response moves through queued and in_progress and lands on one of completed, incomplete, failed or cancelled. The documentation's own instruction is to keep polling while it is queued or in_progress, because leaving those two states is the definition of terminal.
Read the reason, not just the status. failed carries an error object whose code separates the retryable from the escalatable: server_error and rate_limit_exceeded want a retry, invalid_prompt wants a person. incomplete carries incomplete_details.reason, which is max_output_tokens or content_filter.
A 404 is two different things and the script makes you say which. On an ordinary project, an id that does not resolve is a lost job. On a zero-data-retention project, background responses run with store false and are kept on disk only for roughly ten minutes to make polling possible at all, so a 404 on an old id is the platform behaving exactly as documented. Pass --zdr and the script reports those separately instead of alarming on them.
The problem in plain words
Background mode moves the failure surface from your process to a stored object. Without it, a request that takes twenty minutes fails as a timeout somewhere in your stack, loudly and in a place with a stack trace. With it, the request is accepted in milliseconds, the 200 says only that the job was queued, and everything that happens afterwards happens on a response object that has no way to tell you about it. There is no callback in the basic flow. Nothing pushes.
So the abandoned job is invisible from both ends. Your side has a row in a table with no terminal status and no error, which looks the same as a job that is simply still running. The platform's side has a response that reached failed two hours ago and has been sitting there ever since. Neither half raises anything, and the two halves never meet unless somebody writes the reconciler.
The costs stack up in three ways. Work that completed is billed and thrown away, because the output existed only on the object nobody read. Work that is genuinely still queued holds capacity you are not using. And work that failed on something retryable — a server_error, a rate_limit_exceeded — is not retried, so a transient blip becomes a permanently missing result.
The awkward part is that the API cannot help you find them. /v1/responses has no list endpoint. You cannot ask for every response your project created, cannot filter by status, and cannot enumerate what you have forgotten. The reconciliation is bounded by the ids you wrote down, which makes the durability of that write the actual failure point: if the id is persisted after the job starts rather than transactionally with it, a crash in between produces a job that is running, billing, and unreferenced anywhere.
Why it happens
Four of the six statuses are not success, and the two that look temporary are the dangerous ones. queued and in_progress are perfectly normal for a job that started forty seconds ago and are a finding for one that started yesterday. That means the check needs a service level to compare against rather than a fixed number, so the threshold is an argument, and the same id is graded differently depending on what your queue promises.
The error object is the difference between a retry and a page. The Responses error code is an enumeration, and it separates infrastructure from input: server_error and rate_limit_exceeded are things that will probably work on the next attempt, while invalid_prompt will fail identically forever. A reconciler that retries everything hammers a broken request, and one that retries nothing loses recoverable work, so the script prints the code and sorts the two apart.
There is no list endpoint, and pretending otherwise would produce a clean run on an empty audit. /v1/responses is reachable only by an id you already hold. So the script takes a file of ids, reports how many it was given, and treats an empty file as a finding in its own right — the absence of a job table is a worse problem than anything in it.
A 404 needs context the API will not give you. Retrieving a response that was never stored, or whose retention has elapsed, returns a not-found. Whether that is a lost job or documented behaviour depends on the project's data-retention posture: on a zero-data-retention project a background response runs with store false and is retained for roughly ten minutes purely to make polling work. The script therefore takes --zdr as a declaration from you and grades the same HTTP status differently, rather than guessing and being confidently wrong half the time.
Only background responses can be cancelled, which is why abandonment is worth finding. The cancel endpoint is documented as applying to responses created with background set to true and to no others. So the stranded jobs this script finds are exactly the population you can actually do something about, and the repair for a queued job that nobody wants any more is a real action rather than waiting for it to finish and paying for it.
The fix, as a flow
Background mode moves the failure out of your process and onto a stored object. The request is accepted in milliseconds, the model works for twenty minutes, and the result exists in exactly one place: a response whose status has six values, four of which are not success. There is no list endpoint, so the audit can only reach the ids you wrote down, which makes that write the real failure point.
How to fix it
Produce the id list first
One response id per line, optionally followed by a comma and the unix timestamp your table recorded at creation. There is no list endpoint, so this file is the audit's entire universe. If you cannot produce it, stop here: that is the finding, and the repair is to persist the id transactionally with the job row rather than after the call returns.
Use a project key set to Read Only
One GET /v1/responses/{response_id} per id. The script does not cancel anything, does not re-run anything and does not create a response. Cancelling an abandoned job is often the right repair, but it is destructive, and it is printed for you rather than performed.
Declare your service level
--sla-minutes is the age past which queued or in_progress stops being normal. A minute-scale interactive queue and an overnight document pipeline are both correct and want very different numbers, so this is an argument rather than a constant. The value used is printed with the result.
Say whether the project is zero-data-retention
--zdr changes how a 404 is graded. On a ZDR project, background responses run unstored and are kept for roughly ten minutes to allow polling at all, so an old id that does not resolve is documented behaviour rather than a lost job. Without the flag, the same 404 is reported as a job whose result is gone.
Read the reasons and split the retries from the escalations
The output groups by bucket and prints the error.code or incomplete_details.reason next to each id. The repair is printed: retry server_error and rate_limit_exceeded, escalate invalid_prompt, raise max_output_tokens or accept the truncation, and cancel the queued jobs nobody wants.
How to check it worked
Re-run after the reconciler exists. What should change is not this script's verdict but the size of its input: a job table that is being driven to terminal states has few open ids in it, and the ones it has are younger than the service level. Run it on a schedule against the open rows only, and treat a stranded count above zero as the alarm rather than as a report.
python3 openai_background_response_audit.py --ids open_jobs.txt --sla-minutes 30
# resp_68f1a2c4 stranded in_progress for 19 h
# resp_68f1a2d9 stranded queued for 19 h
# resp_68e94411 failed error.code server_error
# resp_68e944a7 failed error.code invalid_prompt
# resp_68e94500 incomplete incomplete_details.reason max_output_tokens
# resp_68d21a03 gone HTTP 404, no longer retrievable
# resp_68f9bb10 running queued for 4 min, inside the 30 min service level
# resp_68f9bb2a completed
# background-stranded 2 of 8 ids have been queued or in_progress past the 30
# minute service level, 2 failed and 1 is no longer
# retrievable
# measured: status, error.code and incomplete_details.reason from one GET per id
# inferred: nothing about ids not in the file, because /v1/responses has no
# list endpoint and cannot be enumerated
# repair: retry server_error and rate_limit_exceeded; escalate invalid_prompt,
# which will fail identically on every attempt.
# repair: cancel the stranded jobs you no longer want. Only responses created
# with background true can be cancelled, so these ones can be.
# 5 finding(s)The full code
One GET per id and six pure functions. read_ids, which takes a plain id or an id with the creation timestamp your table recorded, and de-duplicates while keeping order so the output reads like your job table; age_of, which prefers the response's own created_at and falls back to your hint, because a 404 has no object to read an age from; reason_for, which reaches for error.code and then incomplete_details.reason and returns an empty string rather than a null that gets printed; classify, which is the whole grading and takes now, the service level and the ZDR declaration as arguments; summarise, which counts the buckets in a fixed order so the output does not reshuffle between runs; and verdict, which puts a stranded job ahead of a failed one because it is still spending money.
"""Drive every background response id you hold to a terminal status.
Read only. One GET /v1/responses/{response_id} per id and nothing else. This
script does not create a response, does not cancel one and does not retry
anything: cancelling is destructive and retrying costs money, so both are
printed for a human to run.
/v1/responses has no list endpoint. The ids come from your own job table, which
means the audit is bounded by what you wrote down, and the script says so with
every result rather than implying it enumerated anything.
A 404 is graded differently on a zero-data-retention project, where background
responses run unstored and are retained for roughly ten minutes purely so that
polling works at all. Pass --zdr and those stop being reported as lost jobs.
"""
import argparse
import logging
import os
import sys
import time
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("openai_background_response_audit")
RESPONSES_URL = "https://api.openai.com/v1/responses"
# The documented status enum. Four of the six are not success, and the two that
# look temporary are the ones worth an alarm when they stop being temporary.
OPEN_STATES = ("queued", "in_progress")
TERMINAL_STATES = ("completed", "incomplete", "failed", "cancelled")
# Roughly how long a ZDR project keeps a background response on disk so that it
# can be polled at all. Past this, a 404 there is documented behaviour.
ZDR_WINDOW = 600
BUCKET_ORDER = ("stranded", "failed", "incomplete", "gone", "cancelled",
"running", "completed", "aged-out", "unreadable")
RETRYABLE = ("server_error", "rate_limit_exceeded")
FINDINGS = ("background-stranded", "background-failed", "background-gone",
"background-no-ids")
def read_ids(text):
"""[(id, created_hint)] from a file body. Pure. Order kept, ids deduped.
A line is either an id or "id,<unix timestamp>". The timestamp is what your
own table recorded at creation, and it is the only way to age a 404, which
has no object behind it to read a created_at from.
"""
out, seen = [], set()
for raw in (text or "").splitlines():
line = raw.strip()
if not line or line.startswith("#"):
continue
ident, _, stamp = line.partition(",")
ident = ident.strip()
if not ident or ident in seen:
continue
seen.add(ident)
try:
hint = int(float(stamp.strip())) if stamp.strip() else None
except ValueError:
hint = None
out.append((ident, hint))
return out
def age_of(response, hint, now):
"""Seconds since creation. Pure. None when neither source has a time."""
created = (response or {}).get("created_at")
try:
created = int(created)
except (TypeError, ValueError):
created = None
if created is None:
created = hint
if created is None:
return None
return max(0, int(now) - int(created))
def reason_for(response):
"""The failure reason, or "". Pure. Never returns None to be printed."""
response = response or {}
error = response.get("error") or {}
if isinstance(error, dict) and error.get("code"):
return "error.code %s" % error["code"]
details = response.get("incomplete_details") or {}
if isinstance(details, dict) and details.get("reason"):
return "incomplete_details.reason %s" % details["reason"]
return ""
def error_code(response):
"""Just the error code, or "". Pure. Used to sort retry from escalate."""
error = (response or {}).get("error") or {}
return str(error.get("code") or "") if isinstance(error, dict) else ""
def classify(record, now, sla_seconds, zdr=False):
"""(bucket, detail) for one id. Pure. now and the SLA are arguments.
The same HTTP 404 is a lost job on an ordinary project and documented
behaviour on a ZDR one, so the declaration is taken from the caller rather
than guessed from a field that does not exist.
"""
http = (record or {}).get("http")
response = (record or {}).get("response") or {}
hint = (record or {}).get("created_hint")
age = age_of(response, hint, now)
if http == 404:
if zdr and (age is None or age > ZDR_WINDOW):
return ("aged-out",
"HTTP 404, and on a ZDR project a background response is "
"kept only about ten minutes")
return ("gone", "HTTP 404, no longer retrievable")
if http != 200:
return ("unreadable", "HTTP %s" % http)
status = str(response.get("status") or "")
if status in OPEN_STATES:
shown = "%d min" % (age // 60) if age is not None else "an unknown time"
if age is not None and age > sla_seconds:
return ("stranded", "%s for %s" % (status, shown))
return ("running", "%s for %s, inside the service level" % (status, shown))
if status == "failed":
return ("failed", reason_for(response) or "failed with no error object")
if status == "incomplete":
return ("incomplete", reason_for(response) or "incomplete with no reason")
if status == "cancelled":
return ("cancelled", "cancelled")
if status == "completed":
return ("completed", "")
return ("unreadable", "status %r is not one of the six documented values"
% status)
def summarise(rows):
"""{bucket: count} in a fixed order. Pure. Empty buckets are omitted."""
counts = {}
for row in rows or []:
counts[row.get("bucket")] = counts.get(row.get("bucket"), 0) + 1
return {b: counts[b] for b in BUCKET_ORDER if b in counts}
def verdict(rows, sla_seconds):
"""Grade the run. Pure. Returns (state, detail)."""
rows = list(rows or [])
if not rows:
return ("background-no-ids",
"no response ids were supplied. /v1/responses has no list "
"endpoint, so an empty id file means those jobs are already "
"unreachable")
counts = summarise(rows)
minutes = max(1, int(sla_seconds // 60))
stranded = counts.get("stranded", 0)
failed = counts.get("failed", 0)
gone = counts.get("gone", 0)
tail = ""
if failed or gone:
parts = []
if failed:
parts.append("%d failed" % failed)
if gone:
parts.append("%d is no longer retrievable" % gone)
tail = ", " + " and ".join(parts)
if stranded:
return ("background-stranded",
"%d of %d ids have been queued or in_progress past the %d "
"minute service level%s" % (stranded, len(rows), minutes, tail))
if failed:
return ("background-failed",
"%d of %d ids reached failed and nothing read the error code%s"
% (failed, len(rows), ", %d is no longer retrievable" % gone
if gone else ""))
if gone:
return ("background-gone",
"%d of %d ids no longer resolve, so whatever they produced is "
"gone" % (gone, len(rows)))
return ("background-drained",
"all %d ids are terminal or inside the %d minute service level"
% (len(rows), minutes))
def repair_lines(state, rows):
"""The repair for one verdict. Pure. Printed, never performed."""
rows = list(rows or [])
if state == "background-no-ids":
return ["persist the response id transactionally with the job row, not "
"after the call returns. A crash in between leaves a job that "
"runs, bills, and is referenced nowhere.",
"there is no list endpoint for /v1/responses, so an id you did "
"not write down cannot be recovered by any read call."]
if state == "background-drained":
return ["nothing stranded. Keep the reconciler running: the failure "
"mode here is a poller that stops, not one that is wrong."]
lines = []
codes = {row.get("code") for row in rows if row.get("code")}
if codes & set(RETRYABLE):
lines.append("retry the transient codes (%s), which will usually "
"succeed on a second attempt."
% ", ".join(sorted(codes & set(RETRYABLE))))
if codes - set(RETRYABLE):
lines.append("escalate %s. These fail identically on every attempt, so "
"a retry loop only spends money."
% ", ".join(sorted(codes - set(RETRYABLE))))
if any(row.get("bucket") == "stranded" for row in rows):
lines.append("cancel the stranded jobs you no longer want, at "
"/v1/responses/{response_id}/cancel. Only responses "
"created with background true can be cancelled, so these "
"ones can be.")
if any(row.get("bucket") == "incomplete" for row in rows):
lines.append("an incomplete response was cut rather than refused. Read "
"incomplete_details.reason: max_output_tokens wants a "
"bigger cap, content_filter wants a person.")
if any(row.get("bucket") == "gone" for row in rows):
lines.append("an id that no longer resolves cannot be recovered by any "
"read call. Archive the output at the moment a response "
"reaches completed, not on the next run of a nightly job.")
return lines
def fetch(response_id, key, timeout=30):
"""One GET. Returns (http_status, payload). Read only, always."""
try:
r = requests.get("%s/%s" % (RESPONSES_URL, response_id),
headers={"Authorization": "Bearer %s" % key},
timeout=timeout)
except requests.RequestException:
return (None, {})
try:
return (r.status_code, r.json())
except ValueError:
return (r.status_code, {})
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--ids", help="file of response ids, one per line, "
"optionally id,<unix created_at>")
ap.add_argument("--sla-minutes", type=int, default=30,
help="age past which queued or in_progress is a finding")
ap.add_argument("--zdr", action="store_true",
help="this project is zero data retention, so a 404 on an "
"old background response is documented behaviour")
args = ap.parse_args()
key = os.environ.get("OPENAI_API_KEY")
if not key:
log.error("set OPENAI_API_KEY to a project key set to Read Only")
return 2
raw = ""
if args.ids:
try:
with open(args.ids, "r", encoding="utf-8") as fh:
raw = fh.read()
except OSError as exc:
log.error("could not read %s: %s", args.ids, exc)
return 2
else:
raw = os.environ.get("OPENAI_RESPONSE_IDS", "").replace(",", "\n")
pairs = read_ids(raw)
now = int(time.time())
sla = max(1, args.sla_minutes) * 60
rows = []
for ident, hint in pairs:
http, payload = fetch(ident, key)
bucket, detail = classify({"http": http, "response": payload,
"created_hint": hint}, now, sla, args.zdr)
rows.append({"id": ident, "bucket": bucket, "detail": detail,
"code": error_code(payload)})
emit = log.warning if bucket in ("stranded", "failed", "gone") else log.info
emit("%-16s %-12s %s", ident[:16], bucket, detail)
state, detail = verdict(rows, sla)
emit = log.warning if state in FINDINGS else log.info
emit("%-20s %s", state, detail)
counts = summarise(rows)
if counts:
emit(" buckets: %s", ", ".join("%s %d" % (k, v) for k, v in counts.items()))
emit(" measured: status, error.code and incomplete_details.reason from one "
"GET per id")
emit(" inferred: nothing about ids not in the file, because /v1/responses "
"has no list endpoint and cannot be enumerated")
for line in repair_lines(state, rows):
emit(" repair: %s", line)
findings = sum(counts.get(b, 0) for b in ("stranded", "failed", "gone"))
if state == "background-no-ids":
findings = 1
log.info("%d finding(s)", findings)
return 1 if findings else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Drive every background response id you hold to a terminal status.
*
* Read only. One GET /v1/responses/{response_id} per id. Nothing is created,
* cancelled or retried.
*
* /v1/responses has no list endpoint, so the ids come from your own job table
* and the audit is bounded by what you wrote down.
*
* A 404 is graded differently on a zero-data-retention project, where a
* background response is kept for roughly ten minutes so polling can work.
*/
import { readFile } from 'node:fs/promises';
const RESPONSES_URL = 'https://api.openai.com/v1/responses';
export const OPEN_STATES = new Set(['queued', 'in_progress']);
export const TERMINAL_STATES = new Set(['completed', 'incomplete', 'failed', 'cancelled']);
export const ZDR_WINDOW = 600;
export const BUCKET_ORDER = ['stranded', 'failed', 'incomplete', 'gone', 'cancelled',
'running', 'completed', 'aged-out', 'unreadable'];
export const RETRYABLE = ['server_error', 'rate_limit_exceeded'];
const FINDINGS = new Set(['background-stranded', 'background-failed',
'background-gone', 'background-no-ids']);
/** [[id, createdHint]] from a file body. Pure. Order kept, ids deduped. */
export function readIds(text) {
const out = [];
const seen = new Set();
for (const raw of String(text ?? '').split('\n')) {
const line = raw.trim();
if (!line || line.startsWith('#')) continue;
const at = line.indexOf(',');
const ident = (at < 0 ? line : line.slice(0, at)).trim();
const stamp = at < 0 ? '' : line.slice(at + 1).trim();
if (!ident || seen.has(ident)) continue;
seen.add(ident);
const parsed = Number.parseFloat(stamp);
out.push([ident, Number.isFinite(parsed) ? Math.trunc(parsed) : null]);
}
return out;
}
/** Seconds since creation. Pure. Null when neither source has a time. */
export function ageOf(response, hint, now) {
const raw = Number((response ?? {}).created_at);
const created = Number.isFinite(raw) ? Math.trunc(raw)
: (Number.isFinite(Number(hint)) && hint !== null ? Math.trunc(Number(hint)) : null);
if (created === null) return null;
return Math.max(0, Math.trunc(now) - created);
}
/** The failure reason, or ''. Pure. Never returns null to be printed. */
export function reasonFor(response) {
const r = response ?? {};
const error = r.error;
if (error && typeof error === 'object' && error.code) return `error.code ${error.code}`;
const details = r.incomplete_details;
if (details && typeof details === 'object' && details.reason) {
return `incomplete_details.reason ${details.reason}`;
}
return '';
}
/** Just the error code, or ''. Pure. */
export function errorCode(response) {
const error = (response ?? {}).error;
return error && typeof error === 'object' ? String(error.code ?? '') : '';
}
/** [bucket, detail] for one id. Pure. now, the SLA and ZDR are arguments. */
export function classify(record, now, slaSeconds, zdr = false) {
const http = (record ?? {}).http;
const response = (record ?? {}).response ?? {};
const age = ageOf(response, (record ?? {}).created_hint, now);
if (http === 404) {
if (zdr && (age === null || age > ZDR_WINDOW)) {
return ['aged-out', 'HTTP 404, and on a ZDR project a background response '
+ 'is kept only about ten minutes'];
}
return ['gone', 'HTTP 404, no longer retrievable'];
}
if (http !== 200) return ['unreadable', `HTTP ${http}`];
const status = String(response.status ?? '');
if (OPEN_STATES.has(status)) {
const shown = age === null ? 'an unknown time' : `${Math.floor(age / 60)} min`;
if (age !== null && age > slaSeconds) return ['stranded', `${status} for ${shown}`];
return ['running', `${status} for ${shown}, inside the service level`];
}
if (status === 'failed') {
return ['failed', reasonFor(response) || 'failed with no error object'];
}
if (status === 'incomplete') {
return ['incomplete', reasonFor(response) || 'incomplete with no reason'];
}
if (status === 'cancelled') return ['cancelled', 'cancelled'];
if (status === 'completed') return ['completed', ''];
return ['unreadable',
`status ${JSON.stringify(status)} is not one of the six documented values`];
}
/** {bucket: count} in a fixed order. Pure. Empty buckets are omitted. */
export function summarise(rows) {
const counts = new Map();
for (const row of rows ?? []) {
counts.set(row?.bucket, (counts.get(row?.bucket) ?? 0) + 1);
}
const out = {};
for (const b of BUCKET_ORDER) if (counts.has(b)) out[b] = counts.get(b);
return out;
}
/** Grade the run. Pure. Returns [state, detail]. */
export function verdict(rows, slaSeconds) {
const list = rows ?? [];
if (!list.length) {
return ['background-no-ids',
'no response ids were supplied. /v1/responses has no list endpoint, so an '
+ 'empty id file means those jobs are already unreachable'];
}
const counts = summarise(list);
const minutes = Math.max(1, Math.trunc(slaSeconds / 60));
const stranded = counts.stranded ?? 0;
const failed = counts.failed ?? 0;
const gone = counts.gone ?? 0;
let tail = '';
if (failed || gone) {
const parts = [];
if (failed) parts.push(`${failed} failed`);
if (gone) parts.push(`${gone} is no longer retrievable`);
tail = `, ${parts.join(' and ')}`;
}
if (stranded) {
return ['background-stranded',
`${stranded} of ${list.length} ids have been queued or in_progress past `
+ `the ${minutes} minute service level${tail}`];
}
if (failed) {
return ['background-failed',
`${failed} of ${list.length} ids reached failed and nothing read the error `
+ `code${gone ? `, ${gone} is no longer retrievable` : ''}`];
}
if (gone) {
return ['background-gone',
`${gone} of ${list.length} ids no longer resolve, so whatever they `
+ 'produced is gone'];
}
return ['background-drained',
`all ${list.length} ids are terminal or inside the ${minutes} minute service level`];
}
/** The repair for one verdict. Pure. Printed, never performed. */
export function repairLines(state, rows) {
const list = rows ?? [];
if (state === 'background-no-ids') {
return ['persist the response id transactionally with the job row, not after '
+ 'the call returns. A crash in between leaves a job that runs, bills, and '
+ 'is referenced nowhere.',
'there is no list endpoint for /v1/responses, so an id you did not write '
+ 'down cannot be recovered by any read call.'];
}
if (state === 'background-drained') {
return ['nothing stranded. Keep the reconciler running: the failure mode '
+ 'here is a poller that stops, not one that is wrong.'];
}
const lines = [];
const codes = new Set(list.map((r) => r?.code).filter(Boolean));
const retry = [...codes].filter((c) => RETRYABLE.includes(c)).sort();
const escalate = [...codes].filter((c) => !RETRYABLE.includes(c)).sort();
if (retry.length) {
lines.push(`retry the transient codes (${retry.join(', ')}), which will `
+ 'usually succeed on a second attempt.');
}
if (escalate.length) {
lines.push(`escalate ${escalate.join(', ')}. These fail identically on every `
+ 'attempt, so a retry loop only spends money.');
}
if (list.some((r) => r?.bucket === 'stranded')) {
lines.push('cancel the stranded jobs you no longer want, at '
+ '/v1/responses/{response_id}/cancel. Only responses created with '
+ 'background true can be cancelled, so these ones can be.');
}
if (list.some((r) => r?.bucket === 'incomplete')) {
lines.push('an incomplete response was cut rather than refused. Read '
+ 'incomplete_details.reason: max_output_tokens wants a bigger cap, '
+ 'content_filter wants a person.');
}
if (list.some((r) => r?.bucket === 'gone')) {
lines.push('an id that no longer resolves cannot be recovered by any read '
+ 'call. Archive the output at the moment a response reaches completed, '
+ 'not on the next run of a nightly job.');
}
return lines;
}
async function fetchOne(id, key) {
let res;
try {
res = await fetch(`${RESPONSES_URL}/${id}`, {
headers: { Authorization: `Bearer ${key}` },
});
} catch {
return [null, {}];
}
try {
return [res.status, await res.json()];
} catch {
return [res.status, {}];
}
}
function args(argv) {
const out = { slaMinutes: 30, zdr: false };
for (let i = 0; i < argv.length; i += 1) {
if (argv[i] === '--ids') out.ids = argv[i += 1];
else if (argv[i] === '--sla-minutes') out.slaMinutes = Number.parseInt(argv[i += 1], 10);
else if (argv[i] === '--zdr') out.zdr = true;
}
return out;
}
async function main() {
const opts = args(process.argv.slice(2));
const key = process.env.OPENAI_API_KEY;
if (!key) {
console.error('set OPENAI_API_KEY to a project key set to Read Only');
process.exitCode = 2;
return;
}
let raw = '';
if (opts.ids) {
try {
raw = await readFile(opts.ids, 'utf8');
} catch (err) {
console.error(`could not read ${opts.ids}: ${err.message}`);
process.exitCode = 2;
return;
}
} else {
raw = (process.env.OPENAI_RESPONSE_IDS ?? '').split(',').join('\n');
}
const pairs = readIds(raw);
const now = Math.floor(Date.now() / 1000);
const sla = Math.max(1, opts.slaMinutes) * 60;
const rows = [];
for (const [ident, hint] of pairs) {
const [http, payload] = await fetchOne(ident, key);
const [bucket, detail] = classify({ http, response: payload, created_hint: hint },
now, sla, opts.zdr);
rows.push({ id: ident, bucket, detail, code: errorCode(payload) });
console.log(`${ident.slice(0, 16).padEnd(16)} ${bucket.padEnd(12)} ${detail}`);
}
const [state, detail] = verdict(rows, sla);
console.log(`${state.padEnd(20)} ${detail}`);
const counts = summarise(rows);
if (Object.keys(counts).length) {
console.log(` buckets: ${Object.entries(counts).map(([k, v]) => `${k} ${v}`).join(', ')}`);
}
console.log(' measured: status, error.code and incomplete_details.reason from '
+ 'one GET per id');
console.log(' inferred: nothing about ids not in the file, because '
+ '/v1/responses has no list endpoint and cannot be enumerated');
for (const line of repairLines(state, rows)) console.log(` repair: ${line}`);
let findings = ['stranded', 'failed', 'gone'].reduce((n, b) => n + (counts[b] ?? 0), 0);
if (state === 'background-no-ids') findings = 1;
console.log(`${findings} finding(s)`);
process.exitCode = findings ? 1 : 0;
void FINDINGS;
void TERMINAL_STATES;
}
if (import.meta.url === `file://${process.argv[1]}`) await main();
Add a test
The first test walks one id through each of the six documented statuses and asserts a distinct bucket for each, because collapsing incomplete into failed or cancelled into completed would hide exactly the cases that need a different repair. The second is the service level: the same in_progress response is running against a thirty minute promise and stranded against a three minute one, with now and the threshold passed in so the assertion does not depend on when it runs. The third is the 404, graded twice — a lost job on an ordinary project and documented behaviour on a ZDR one — and it also covers the case where no timestamp is available at all. The fourth is the id file parser, given blank lines, comments, duplicates, a bare id and an id with a timestamp. The fifth is that an empty id list is a finding rather than a clean run, which is the single most important behaviour in the script. And the last checks the retry split, where server_error and invalid_prompt must not end up in the same repair line.
from openai_background_response_audit import (age_of, classify, error_code,
read_ids, reason_for,
repair_lines, summarise, verdict)
NOW = 1_800_000_000
SLA = 30 * 60
def record(status, created=None, http=200, **extra):
body = {"id": "resp_x", "status": status}
if created is not None:
body["created_at"] = created
body.update(extra)
return {"http": http, "response": body, "created_hint": None}
def test_each_documented_status_gets_its_own_bucket():
assert classify(record("completed", NOW - 60), NOW, SLA)[0] == "completed"
assert classify(record("cancelled", NOW - 60), NOW, SLA)[0] == "cancelled"
incomplete = record("incomplete", NOW - 60,
incomplete_details={"reason": "max_output_tokens"})
bucket, detail = classify(incomplete, NOW, SLA)
assert bucket == "incomplete" and "max_output_tokens" in detail
failed = record("failed", NOW - 60,
error={"code": "server_error", "message": "boom"})
bucket, detail = classify(failed, NOW, SLA)
assert bucket == "failed" and "error.code server_error" in detail
assert error_code(failed["response"]) == "server_error"
# A status outside the enum is not silently treated as success.
assert classify(record("weird", NOW), NOW, SLA)[0] == "unreadable"
assert reason_for({}) == ""
def test_queued_is_normal_until_the_service_level_says_it_is_not():
running = record("in_progress", NOW - 4 * 60)
bucket, detail = classify(running, NOW, SLA)
assert bucket == "running" and "inside the service level" in detail
bucket, detail = classify(running, NOW, 3 * 60)
assert bucket == "stranded" and detail.startswith("in_progress for 4 min")
queued = record("queued", NOW - 19 * 3600)
assert classify(queued, NOW, SLA)[0] == "stranded"
# The hint from your own table stands in when the object has no created_at.
no_stamp = {"http": 200, "response": {"status": "queued"},
"created_hint": NOW - 7200}
assert classify(no_stamp, NOW, SLA)[0] == "stranded"
assert age_of({}, None, NOW) is None
assert classify({"http": 200, "response": {"status": "queued"},
"created_hint": None}, NOW, SLA)[0] == "running"
def test_a_404_means_two_different_things_and_zdr_decides_which():
lost = {"http": 404, "response": {}, "created_hint": NOW - 86400}
assert classify(lost, NOW, SLA)[0] == "gone"
bucket, detail = classify(lost, NOW, SLA, zdr=True)
assert bucket == "aged-out" and "ten minutes" in detail
# Inside the ZDR window a 404 is still a real miss.
fresh = {"http": 404, "response": {}, "created_hint": NOW - 60}
assert classify(fresh, NOW, SLA, zdr=True)[0] == "gone"
assert classify({"http": 500, "response": {}}, NOW, SLA)[0] == "unreadable"
def test_the_id_file_takes_bare_ids_timestamps_comments_and_duplicates():
text = "\n".join(["# open jobs", "", "resp_a", "resp_b,1799990000",
"resp_a", " resp_c , not-a-number "])
assert read_ids(text) == [("resp_a", None), ("resp_b", 1799990000),
("resp_c", None)]
assert read_ids("") == []
assert read_ids(None) == []
def test_an_empty_id_list_is_a_finding_and_not_a_clean_run():
state, detail = verdict([], SLA)
assert state == "background-no-ids"
assert "no list endpoint" in detail
lines = repair_lines(state, [])
assert any("transactionally" in line for line in lines)
drained = [{"id": "a", "bucket": "completed", "code": ""}]
assert verdict(drained, SLA)[0] == "background-drained"
assert summarise(drained) == {"completed": 1}
def test_transient_and_permanent_error_codes_get_different_repairs():
rows = [
{"id": "a", "bucket": "stranded", "code": ""},
{"id": "b", "bucket": "failed", "code": "server_error"},
{"id": "c", "bucket": "failed", "code": "invalid_prompt"},
{"id": "d", "bucket": "gone", "code": ""},
{"id": "e", "bucket": "incomplete", "code": ""},
]
state, detail = verdict(rows, SLA)
assert state == "background-stranded"
assert "2 failed" in detail and "no longer retrievable" in detail
lines = repair_lines(state, rows)
retry = [line for line in lines if line.startswith("retry")]
escalate = [line for line in lines if line.startswith("escalate")]
assert retry and "server_error" in retry[0] and "invalid_prompt" not in retry[0]
assert escalate and "invalid_prompt" in escalate[0]
assert any("background true can be cancelled" in line for line in lines)
assert any("incomplete_details.reason" in line for line in lines)
assert summarise(rows)["stranded"] == 1
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { ageOf, classify, errorCode, readIds, reasonFor, repairLines, summarise,
verdict } from './openai-background-response-audit.mjs';
const NOW = 1800000000;
const SLA = 30 * 60;
function record(status, created = null, http = 200, extra = {}) {
const body = { id: 'resp_x', status, ...extra };
if (created !== null) body.created_at = created;
return { http, response: body, created_hint: null };
}
test('each documented status gets its own bucket', () => {
assert.equal(classify(record('completed', NOW - 60), NOW, SLA)[0], 'completed');
assert.equal(classify(record('cancelled', NOW - 60), NOW, SLA)[0], 'cancelled');
const incomplete = record('incomplete', NOW - 60, 200,
{ incomplete_details: { reason: 'max_output_tokens' } });
const [b1, d1] = classify(incomplete, NOW, SLA);
assert.equal(b1, 'incomplete');
assert.ok(d1.includes('max_output_tokens'));
const failed = record('failed', NOW - 60, 200,
{ error: { code: 'server_error', message: 'boom' } });
const [b2, d2] = classify(failed, NOW, SLA);
assert.equal(b2, 'failed');
assert.ok(d2.includes('error.code server_error'));
assert.equal(errorCode(failed.response), 'server_error');
assert.equal(classify(record('weird', NOW), NOW, SLA)[0], 'unreadable');
assert.equal(reasonFor({}), '');
});
test('queued is normal until the service level says it is not', () => {
const running = record('in_progress', NOW - 4 * 60);
const [b1, d1] = classify(running, NOW, SLA);
assert.equal(b1, 'running');
assert.ok(d1.includes('inside the service level'));
const [b2, d2] = classify(running, NOW, 3 * 60);
assert.equal(b2, 'stranded');
assert.ok(d2.startsWith('in_progress for 4 min'));
assert.equal(classify(record('queued', NOW - 19 * 3600), NOW, SLA)[0], 'stranded');
const noStamp = { http: 200, response: { status: 'queued' }, created_hint: NOW - 7200 };
assert.equal(classify(noStamp, NOW, SLA)[0], 'stranded');
assert.equal(ageOf({}, null, NOW), null);
assert.equal(classify({ http: 200, response: { status: 'queued' }, created_hint: null },
NOW, SLA)[0], 'running');
});
test('a 404 means two different things and zdr decides which', () => {
const lost = { http: 404, response: {}, created_hint: NOW - 86400 };
assert.equal(classify(lost, NOW, SLA)[0], 'gone');
const [b, d] = classify(lost, NOW, SLA, true);
assert.equal(b, 'aged-out');
assert.ok(d.includes('ten minutes'));
const fresh = { http: 404, response: {}, created_hint: NOW - 60 };
assert.equal(classify(fresh, NOW, SLA, true)[0], 'gone');
assert.equal(classify({ http: 500, response: {} }, NOW, SLA)[0], 'unreadable');
});
test('the id file takes bare ids timestamps comments and duplicates', () => {
const text = ['# open jobs', '', 'resp_a', 'resp_b,1799990000', 'resp_a',
' resp_c , not-a-number '].join('\n');
assert.deepEqual(readIds(text),
[['resp_a', null], ['resp_b', 1799990000], ['resp_c', null]]);
assert.deepEqual(readIds(''), []);
assert.deepEqual(readIds(null), []);
});
test('an empty id list is a finding and not a clean run', () => {
const [state, detail] = verdict([], SLA);
assert.equal(state, 'background-no-ids');
assert.ok(detail.includes('no list endpoint'));
assert.ok(repairLines(state, []).some((l) => l.includes('transactionally')));
const drained = [{ id: 'a', bucket: 'completed', code: '' }];
assert.equal(verdict(drained, SLA)[0], 'background-drained');
assert.deepEqual(summarise(drained), { completed: 1 });
});
test('transient and permanent error codes get different repairs', () => {
const rows = [
{ id: 'a', bucket: 'stranded', code: '' },
{ id: 'b', bucket: 'failed', code: 'server_error' },
{ id: 'c', bucket: 'failed', code: 'invalid_prompt' },
{ id: 'd', bucket: 'gone', code: '' },
{ id: 'e', bucket: 'incomplete', code: '' },
];
const [state, detail] = verdict(rows, SLA);
assert.equal(state, 'background-stranded');
assert.ok(detail.includes('2 failed') && detail.includes('no longer retrievable'));
const lines = repairLines(state, rows);
const retry = lines.filter((l) => l.startsWith('retry'));
const escalate = lines.filter((l) => l.startsWith('escalate'));
assert.ok(retry.length && retry[0].includes('server_error')
&& !retry[0].includes('invalid_prompt'));
assert.ok(escalate.length && escalate[0].includes('invalid_prompt'));
assert.ok(lines.some((l) => l.includes('background true can be cancelled')));
assert.ok(lines.some((l) => l.includes('incomplete_details.reason')));
assert.equal(summarise(rows).stranded, 1);
});
FAQ
Can I list all my background responses?
No, and this is the constraint the whole note is built around. <code>/v1/responses</code> has no list endpoint: a stored response is reachable only by an id you already hold. There is no way to ask the API which responses your project created, no status filter and no way to enumerate the ones you have forgotten about. That makes the durability of the write that records the id the real failure point, which is why the repair for an empty id file is to persist it transactionally with the job row rather than after the call returns.
What are all the statuses a background response can have?
Six: <code>queued</code>, <code>in_progress</code>, <code>completed</code>, <code>incomplete</code>, <code>failed</code> and <code>cancelled</code>. The documented polling rule is to keep going while it is <code>queued</code> or <code>in_progress</code>, because leaving those two states is what terminal means. Of the four terminal values only <code>completed</code> is success, so a reconciler that treats \u201cnot queued any more\u201d as done will silently accept a refusal, a truncation and a cancellation as finished work.
Why does my background response 404 when I know it existed?
Most often because the project is zero data retention. Background requests there run with <code>store</code> set to false, and the response is held on disk only for roughly ten minutes so that polling can work at all; after that the id resolves to nothing, by design. On an ordinary project a 404 means the response was never stored or its retention elapsed, and that is a genuinely lost result. The script cannot tell the two apart from the API, so it takes <code>--zdr</code> from you and grades accordingly.
Should I cancel the stranded jobs?
That is the usual repair, and it is one of the few actions available: the cancel endpoint is documented as applying only to responses created with <code>background</code> set to true, so this population is exactly the cancellable one. The script prints it rather than doing it, because cancelling destroys work that might still be wanted and only you know whether the job's result still has a consumer. Cancellation is also idempotent, so a later call simply returns the final response object.
Is this the same problem as a batch nobody polled?
The shape is the same and the surface is not. A batch is a queue of thousands of requests with a 24 hour window, a results file and a retention clock, and its abandonment is found by listing batches. A background response is a single job with no list endpoint, a six value status and an <code>error</code> object, and its abandonment is found by walking ids you kept. The repairs differ too: a batch wants a reconciler over the list, a background response wants the id persisted transactionally in the first place.
Related field notes
- Another stored response object with a retention clock on it
- Reading incomplete_details on a response that came back 200
- Work you paid for and never collected, on both providers
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Background mode — OpenAI developer docs
- Responses — OpenAI API reference
- Conversation state and stored response retention
- Error codes — OpenAI platform docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.