Diagnostic LLM APIs
Cancelling a batch does not unbill the rows it already ran
Somebody hit cancel during the deploy, which was the right call, and then the runbook said re-run it in the morning. So it was re-run in the morning. What nobody checked is that the batch had already processed sixty-one thousand of its ninety thousand rows before the cancel landed, that those rows are sitting in the output file, and that the morning re-run paid for all sixty-one thousand of them a second time. Cancel is not a rollback. It is a stop.
Read the cancellation states before you re-run anything. On OpenAI, GET /v1/batches?limit=100 and keep status of cancelling or cancelled; the row that matters is request_counts.completed against total, alongside a non-null output_file_id. On Anthropic, GET /v1/messages/batches?limit=1000 with a workspace key and keep every batch where cancel_initiated_at is not null; the row that matters is request_counts.succeeded against canceled.
Anything with a non-zero completed or succeeded count is finished work you already have. It is in the output file or at results_url, keyed by custom_id, and re-running those rows spends the money again. The repair is a subtraction, not a re-submission: download what came back, take the custom_ids out of the input file, and submit the remainder.
Anthropic states the billing rule outright, and it is the good news: requests that were canceled or expired before reaching the model are not billed. Only what actually ran costs anything. OpenAI documents the partial results but not the billing split, so for that provider the completed count is a floor on what you paid, and the cost report for the day is the confirmation.
Then check that the cancel actually finished. OpenAI holds a batch in cancelling for up to ten minutes before it reaches cancelled. Anthropic holds it in canceling until the in-flight requests drain. A batch that has been mid-cancel for an hour has not stopped, and treating it as stopped is how a second batch gets submitted alongside the first.
The problem in plain words
The mental model that causes this is that cancel undoes the job. It does not. Both Batch APIs are queues of independent requests, and cancellation stops the ones that have not been dispatched yet; the ones already through the model are done, written down, and paid for. The API is completely honest about it — OpenAI's cancelled batch carries an output_file_id and Anthropic's carries a results_url — but the operator who typed the cancel is not reading the batch object afterwards, because in their head the job is gone.
So there are two costs, and the second one is bigger. The first is the output that is thrown away: rows that were generated, billed and never collected. The second is the re-run, which pays for every one of those rows again. On a ninety thousand row batch cancelled two thirds of the way through, that is two thirds of the batch's cost spent twice, and it does not appear as an incident anywhere. It appears as a slightly larger invoice.
The mid-flight state is the other half. Cancellation is asynchronous on both providers: OpenAI's documented window is up to ten minutes in cancelling, and Anthropic's canceling lasts until the requests already in flight finish. During that window the batch is still working, still consuming its share of the enqueued queue, and still producing output. A deploy script that cancels and immediately submits a replacement has, briefly, two batches doing the same work, and a batch stuck in cancelling long past the window is a sign that something is wrong with the job rather than with your patience.
None of this raises. Cancelling is a successful operation, the batch object updates as documented, and the only party who could notice the abandoned output is the code that was going to read it, which was switched off at the same moment as the batch.
Why it happens
The counts are the whole reading, and they are spelled differently on each provider. OpenAI's request_counts is {total, completed, failed}, so the salvage number is completed and the cancelled remainder is total minus the rest. Anthropic's is {processing, succeeded, errored, canceled, expired}, whose values are documented to sum to the total number of requests in the batch, so the salvage number is succeeded and the cancelled remainder is already its own field. The script normalises both into the same row rather than pretending one shape fits.
Anthropic tells you the billing rule; OpenAI does not, and the script says which is which. The Claude documentation states that canceled and expired requests are not billed, and that a cancelled batch ends with processing_status: "ended" and may contain partial results. OpenAI's guide documents the ten-minute cancelling window and the partial output, and says nothing about the billing split. Printing a confident claim about OpenAI billing would be inventing it, so the output reports the completed count as a floor and points at the cost report.
A cancel that has not landed is a different finding from a cancel that has. cancelling_at on OpenAI and cancel_initiated_at on Anthropic are timestamps, and both providers keep a batch in the intermediate state while work drains. Fifteen minutes is a generous read of OpenAI's documented ten, and Anthropic publishes no bound at all, so on that side the threshold is a heuristic and the real ceiling is the batch's own 24-hour expires_at. The script prints the age rather than a verdict dressed up as a fact.
Cancelling before anything completed is a clean outcome and gets said so. A batch cancelled thirty seconds after submission has completed: 0, no salvage and no double billing, and reporting it in the same breath as one that is two thirds done trains people to skim the output. It gets its own line and no finding.
This note stops at the salvage arithmetic and hands retention to its sibling. A cancelled batch's output is on the same expiry clock as any other — thirty days on OpenAI, twenty-nine on Anthropic — but that clock, and the join against your ingest ledger, belong to the unclaimed-output note. Here the question is only whether there is anything worth collecting before you pay for it twice.
The fix, as a flow
Cancel stops the requests that have not been dispatched. It does not reach back for the ones already through the model, which are finished, billed and written to the output. The expensive step is the one after the cancel, where the whole batch is submitted again and the overlap is paid for twice. Two counts on the batch object are the entire reading.
How to fix it
Bring whichever keys you have
OPENAI_API_KEY as a project key set to Read Only, ANTHROPIC_API_KEY as a workspace key, or both. The script reads what it is given and says which providers it looked at, so a single-provider shop gets a single-provider answer rather than a misleading clean run.
List the batches and keep the cancellations
OpenAI: page /v1/batches on after and keep status in cancelling or cancelled. Anthropic: page /v1/messages/batches on after_id and keep every batch with a non-null cancel_initiated_at, which is set only when cancellation was initiated and stays set afterwards.
Split finished work from cancelled work
completed versus total on OpenAI, succeeded versus canceled on Anthropic. A non-zero finished count with an artifact to read — output_file_id or results_url — is the finding, and the number printed next to it is how many rows a naive re-run would pay for twice.
Check the cancel actually landed
Anything still in cancelling or canceling gets its age printed from cancelling_at or cancel_initiated_at. Past about fifteen minutes on OpenAI the documented window has gone by; on Anthropic there is no published window, so the number is information rather than a verdict.
Subtract, then re-submit the remainder
The repair is printed. Download the partial output, collect the custom_ids that came back, remove those lines from the input file, and submit what is left. Results are not returned in request order on either provider, so custom_id is the only join key that works.
How to check it worked
Re-run after the remainder batch is submitted. The old cancelled batches stay in the list — batch objects are not deleted by cancelling them — so what changes is that the finished counts have been collected and the re-run is smaller than the original by exactly the salvaged number. Keep this in the deploy path rather than in a weekly report: the moment it is worth running is between the cancel and the re-run.
python3 batch_cancellation_audit.py
# openai batch_68f4b2 cancelled 61,204 of 90,000 done, 28,796 cancelled
# output_file_id file_7ac1
# openai batch_68f51a cancelling for 68 min, past the 10 minute window
# anthropic msgbatch_01Hq ended 41,880 succeeded, 12,120 canceled
# results_url present
# cancel-stuck 1 batch has been mid cancel longer than the documented
# window, and 2 cancelled batches hold 103,084 finished
# rows nothing has collected
# measured: request_counts and the cancellation timestamps from both batch
# lists
# inferred: that a re-run would repeat the finished rows, since neither API
# records whether the partial output was ever downloaded
# repair: download the partial output, drop those custom_ids from the input
# file, and submit only the remainder.
# repair: on Anthropic, canceled and expired requests are not billed, so the
# succeeded count is the whole cost. On OpenAI the completed count is
# a floor: confirm the day against the cost report.
# 3 finding(s)The full code
Two paged GETs, one per provider, and seven pure functions. parse_time, which takes OpenAI's unix integers and Anthropic's RFC 3339 strings and returns one kind of number; openai_cancel_rows and anthropic_cancel_rows, which normalise two different request_counts shapes into the same row rather than branching everywhere downstream; salvage_rows, which is the subtraction the repair depends on; stuck_rows, which takes both now and the threshold as arguments because a clock in a pure function is a test that fails on a Tuesday; verdict, which puts an unlanded cancel ahead of a salvageable one because billing may not have stopped; and repair_lines, which states the Anthropic billing rule as documented and refuses to state an OpenAI one that is not.
"""Find billed, salvageable output left behind by cancelled batches.
Read only, on both providers. Two GET endpoints, /v1/batches on OpenAI and
/v1/messages/batches on Anthropic. This script never cancels a batch, never
submits one, and never downloads a result file.
Cancel is a stop, not a rollback. Requests that reached the model before the
cancel landed are finished and are in the output; re-running the whole batch
pays for them twice. Anthropic documents that canceled and expired requests are
not billed. OpenAI documents the partial output but not the billing split, so
the completed count is reported as a floor rather than as a total.
Retention of that partial output, and the join against your own ingest ledger,
belong to the unclaimed-output note. This one stops at the arithmetic.
"""
import argparse
import calendar
import datetime
import logging
import os
import sys
import time
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("batch_cancellation_audit")
OPENAI_BATCHES_URL = "https://api.openai.com/v1/batches"
ANTHROPIC_BATCHES_URL = "https://api.anthropic.com/v1/messages/batches"
# OpenAI documents up to ten minutes in "cancelling" before a batch reaches
# "cancelled". Fifteen is a generous read of ten. Anthropic publishes no bound
# for "canceling" at all, so on that side this is a heuristic and the output
# says so rather than dressing a guess up as a rule.
STUCK_SECONDS = 15 * 60
OPENAI_CANCEL_STATES = ("cancelling", "cancelled")
FINDINGS = ("cancel-stuck", "cancel-partial-unclaimed")
def parse_time(value):
"""Epoch seconds from a unix number or an RFC 3339 string. Pure.
OpenAI stamps integers, Anthropic stamps RFC 3339. Everything downstream
wants one kind of number, so the difference is absorbed once, here. A
string with no offset is read as UTC rather than as whatever the machine
running the audit happens to be set to.
"""
if value is None or value == "" or isinstance(value, bool):
return None
if isinstance(value, (int, float)):
return int(value)
text = str(value).strip()
if text[-1:] in ("Z", "z"):
text = text[:-1] + "+00:00"
try:
stamp = datetime.datetime.fromisoformat(text)
except (ValueError, TypeError):
return None
if stamp.tzinfo is None:
return calendar.timegm(stamp.timetuple())
return int(stamp.timestamp())
def openai_cancel_rows(batches):
"""Normalised rows for OpenAI batches under cancellation. Pure."""
out = []
for b in batches or []:
status = (b or {}).get("status")
if status not in OPENAI_CANCEL_STATES:
continue
counts = b.get("request_counts") or {}
total = int(counts.get("total") or 0)
done = int(counts.get("completed") or 0)
failed = int(counts.get("failed") or 0)
out.append({
"provider": "openai",
"id": str(b.get("id")),
"status": status,
"in_flight": status == "cancelling",
"done": done,
"stopped": max(0, total - done - failed),
"total": total,
"artifact": b.get("output_file_id"),
"cancel_started": parse_time(b.get("cancelling_at")),
"billing_known": False,
})
return sorted(out, key=lambda r: r["id"])
def anthropic_cancel_rows(batches):
"""Normalised rows for Claude batches under cancellation. Pure.
cancel_initiated_at is set only when cancellation was initiated, and stays
set once the batch has ended, so it is the whole filter.
"""
out = []
for b in batches or []:
started = (b or {}).get("cancel_initiated_at")
if not started:
continue
counts = b.get("request_counts") or {}
done = int(counts.get("succeeded") or 0)
stopped = int(counts.get("canceled") or 0)
status = str(b.get("processing_status") or "")
out.append({
"provider": "anthropic",
"id": str(b.get("id")),
"status": status or "unknown",
"in_flight": status == "canceling",
"done": done,
"stopped": stopped,
"total": done + stopped + int(counts.get("errored") or 0)
+ int(counts.get("expired") or 0)
+ int(counts.get("processing") or 0),
"artifact": b.get("results_url"),
"cancel_started": parse_time(started),
"billing_known": True,
})
return sorted(out, key=lambda r: r["id"])
def salvage_rows(rows):
"""Rows holding finished work a re-run would pay for again. Pure."""
return [r for r in rows or [] if int(r.get("done") or 0) > 0]
def stuck_rows(rows, now, seconds=STUCK_SECONDS):
"""Rows still mid cancel past the threshold. Pure. now is an argument."""
out = []
for r in rows or []:
if not r.get("in_flight"):
continue
started = r.get("cancel_started")
if started is None or now - started > seconds:
out.append(r)
return out
def salvaged_total(rows):
"""Finished rows across everything cancelled. Pure."""
return sum(int(r.get("done") or 0) for r in salvage_rows(rows))
def verdict(rows, stuck, salvage):
"""Grade the run. Pure. Returns (state, detail)."""
rows = list(rows or [])
stuck = list(stuck or [])
salvage = list(salvage or [])
if not rows:
return ("no-cancels",
"no batch on the providers checked has had a cancellation "
"initiated")
if stuck:
detail = ("%d batch(es) have been mid cancel longer than the documented "
"window" % len(stuck))
if salvage:
detail += (", and %d cancelled batch(es) hold %d finished rows "
"nothing has collected"
% (len(salvage), salvaged_total(salvage)))
return ("cancel-stuck", detail)
if salvage:
return ("cancel-partial-unclaimed",
"%d cancelled batch(es) hold %d finished rows a re-run would "
"pay for again" % (len(salvage), salvaged_total(salvage)))
return ("cancel-clean",
"%d cancellation(s) found, none of which had completed a single "
"request, so there is nothing to salvage and nothing to double pay"
% len(rows))
def repair_lines(state, rows):
"""The repair for one verdict. Pure. Printed, never performed."""
rows = list(rows or [])
if state == "no-cancels":
return []
if state == "cancel-clean":
return ["nothing to collect. Keep cancelling early: a batch stopped "
"before its first request completed costs nothing."]
lines = []
if state == "cancel-stuck":
lines.append("a batch still in cancelling or canceling has not stopped. "
"Poll it to a terminal state before you submit a "
"replacement, or the two will run the same rows at once.")
if any(int(r.get("done") or 0) > 0 for r in rows):
lines.append("download the partial output, drop those custom_ids from "
"the input file, and submit only the remainder. Results "
"are not returned in request order, so custom_id is the "
"only join key that works.")
if any(r.get("provider") == "anthropic" and int(r.get("done") or 0) > 0
for r in rows):
lines.append("on Anthropic, canceled and expired requests are not "
"billed, so the succeeded count is the whole cost of the "
"cancelled batch.")
if any(r.get("provider") == "openai" and int(r.get("done") or 0) > 0
for r in rows):
lines.append("on OpenAI the billing split for a cancelled batch is not "
"documented, so treat the completed count as a floor and "
"confirm the day against the cost report.")
return lines
def get_json(url, headers, params=None, timeout=30):
"""One GET. Returns (payload, error). Read only, always."""
try:
r = requests.get(url, headers=headers, params=params or {}, timeout=timeout)
except requests.RequestException as exc:
return (None, "request failed: %s" % exc)
if r.status_code != 200:
return (None, "HTTP %d %s" % (r.status_code, (r.text or "")[:160]))
try:
return (r.json(), None)
except ValueError:
return (None, "response was not JSON")
def page_openai(key, max_pages):
"""/v1/batches, following the after cursor. GETs only."""
rows, after = [], None
headers = {"Authorization": "Bearer %s" % key}
for _ in range(max(1, max_pages)):
params = {"limit": 100}
if after:
params["after"] = after
payload, err = get_json(OPENAI_BATCHES_URL, headers, params)
if err:
return (rows, err)
data = payload.get("data") or []
rows.extend(data)
if not payload.get("has_more") or not data:
break
after = data[-1].get("id")
return (rows, None)
def page_anthropic(key, max_pages):
"""/v1/messages/batches, following after_id. GETs only."""
rows, after = [], None
headers = {"x-api-key": key, "anthropic-version": "2023-06-01"}
for _ in range(max(1, max_pages)):
params = {"limit": 1000}
if after:
params["after_id"] = after
payload, err = get_json(ANTHROPIC_BATCHES_URL, headers, params)
if err:
return (rows, err)
data = payload.get("data") or []
rows.extend(data)
if not payload.get("has_more") or not data:
break
after = payload.get("last_id") or data[-1].get("id")
return (rows, None)
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--stuck-minutes", type=int, default=15,
help="age past which a batch still mid cancel is reported")
ap.add_argument("--max-pages", type=int, default=20)
args = ap.parse_args()
openai_key = os.environ.get("OPENAI_API_KEY")
anthropic_key = os.environ.get("ANTHROPIC_API_KEY")
if not openai_key and not anthropic_key:
log.error("set OPENAI_API_KEY (project key, Read Only) or "
"ANTHROPIC_API_KEY (workspace key), or both")
return 2
now = int(time.time())
rows = []
checked = []
if openai_key:
checked.append("openai")
batches, err = page_openai(openai_key, args.max_pages)
if err:
log.warning("openai batch list stopped early: %s", err)
rows.extend(openai_cancel_rows(batches))
if anthropic_key:
checked.append("anthropic")
batches, err = page_anthropic(anthropic_key, args.max_pages)
if err:
log.warning("anthropic batch list stopped early: %s", err)
rows.extend(anthropic_cancel_rows(batches))
stuck = stuck_rows(rows, now, max(1, args.stuck_minutes) * 60)
salvage = salvage_rows(rows)
stuck_ids = {r["id"] for r in stuck}
for r in rows:
log.info("%-11s %-14s %-11s %s of %s done, %s stopped", r["provider"],
r["id"][:14], r["status"], format(r["done"], ","),
format(r["total"], ","), format(r["stopped"], ","))
if r["artifact"]:
label = "output_file_id" if r["provider"] == "openai" else "results_url"
log.info("%-11s %-14s %s present", "", "", label)
if r["id"] in stuck_ids:
started = r.get("cancel_started")
age = "an unknown time" if started is None \
else "%d min" % ((now - started) // 60)
log.warning("%-11s %-14s mid cancel for %s", r["provider"],
r["id"][:14], age)
state, detail = verdict(rows, stuck, salvage)
emit = log.warning if state in FINDINGS else log.info
emit("%-20s %s", state, detail)
emit(" checked: %s", ", ".join(checked))
emit(" measured: request_counts and the cancellation timestamps from the "
"batch lists")
emit(" inferred: that a re-run would repeat the finished rows, since "
"neither API records whether the partial output was downloaded")
for line in repair_lines(state, rows):
emit(" repair: %s", line)
total = len(stuck) + len(salvage)
log.info("%d finding(s)", total)
return 1 if total else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Find billed, salvageable output left behind by cancelled batches.
*
* Read only, on both providers: /v1/batches on OpenAI and /v1/messages/batches
* on Anthropic. Nothing is cancelled, submitted or downloaded.
*
* Cancel is a stop, not a rollback. Anthropic documents that canceled and
* expired requests are not billed; OpenAI documents the partial output but not
* the billing split, so its completed count is reported as a floor.
*/
const OPENAI_BATCHES_URL = 'https://api.openai.com/v1/batches';
const ANTHROPIC_BATCHES_URL = 'https://api.anthropic.com/v1/messages/batches';
export const STUCK_SECONDS = 15 * 60;
const OPENAI_CANCEL_STATES = new Set(['cancelling', 'cancelled']);
const FINDINGS = new Set(['cancel-stuck', 'cancel-partial-unclaimed']);
/** Epoch seconds from a unix number or an RFC 3339 string. Pure. */
export function parseTime(value) {
if (value === null || value === undefined || value === '' || typeof value === 'boolean') {
return null;
}
if (typeof value === 'number') return Math.trunc(value);
const ms = Date.parse(String(value));
return Number.isFinite(ms) ? Math.floor(ms / 1000) : null;
}
/** Normalised rows for OpenAI batches under cancellation. Pure. */
export function openaiCancelRows(batches) {
return (batches ?? [])
.filter((b) => OPENAI_CANCEL_STATES.has((b ?? {}).status))
.map((b) => {
const counts = b.request_counts ?? {};
const total = Number(counts.total) || 0;
const done = Number(counts.completed) || 0;
const failed = Number(counts.failed) || 0;
return {
provider: 'openai',
id: String(b.id),
status: b.status,
inFlight: b.status === 'cancelling',
done,
stopped: Math.max(0, total - done - failed),
total,
artifact: b.output_file_id ?? null,
cancelStarted: parseTime(b.cancelling_at),
billingKnown: false,
};
})
.sort((a, b) => a.id.localeCompare(b.id));
}
/** Normalised rows for Claude batches under cancellation. Pure. */
export function anthropicCancelRows(batches) {
return (batches ?? [])
.filter((b) => (b ?? {}).cancel_initiated_at)
.map((b) => {
const counts = b.request_counts ?? {};
const done = Number(counts.succeeded) || 0;
const stopped = Number(counts.canceled) || 0;
const status = String(b.processing_status ?? '') || 'unknown';
return {
provider: 'anthropic',
id: String(b.id),
status,
inFlight: status === 'canceling',
done,
stopped,
total: done + stopped + (Number(counts.errored) || 0)
+ (Number(counts.expired) || 0) + (Number(counts.processing) || 0),
artifact: b.results_url ?? null,
cancelStarted: parseTime(b.cancel_initiated_at),
billingKnown: true,
};
})
.sort((a, b) => a.id.localeCompare(b.id));
}
/** Rows holding finished work a re-run would pay for again. Pure. */
export function salvageRows(rows) {
return (rows ?? []).filter((r) => (Number(r?.done) || 0) > 0);
}
/** Rows still mid cancel past the threshold. Pure. now is an argument. */
export function stuckRows(rows, now, seconds = STUCK_SECONDS) {
return (rows ?? []).filter((r) => {
if (!r?.inFlight) return false;
const started = r.cancelStarted;
return started === null || started === undefined || now - started > seconds;
});
}
/** Finished rows across everything cancelled. Pure. */
export function salvagedTotal(rows) {
return salvageRows(rows).reduce((n, r) => n + (Number(r.done) || 0), 0);
}
/** Grade the run. Pure. Returns [state, detail]. */
export function verdict(rows, stuck, salvage) {
const all = rows ?? [];
const s = stuck ?? [];
const v = salvage ?? [];
if (!all.length) {
return ['no-cancels',
'no batch on the providers checked has had a cancellation initiated'];
}
if (s.length) {
let detail = `${s.length} batch(es) have been mid cancel longer than the `
+ 'documented window';
if (v.length) {
detail += `, and ${v.length} cancelled batch(es) hold ${salvagedTotal(v)} `
+ 'finished rows nothing has collected';
}
return ['cancel-stuck', detail];
}
if (v.length) {
return ['cancel-partial-unclaimed',
`${v.length} cancelled batch(es) hold ${salvagedTotal(v)} finished rows a `
+ 're-run would pay for again'];
}
return ['cancel-clean',
`${all.length} cancellation(s) found, none of which had completed a single `
+ 'request, so there is nothing to salvage and nothing to double pay'];
}
/** The repair for one verdict. Pure. Printed, never performed. */
export function repairLines(state, rows) {
const all = rows ?? [];
if (state === 'no-cancels') return [];
if (state === 'cancel-clean') {
return ['nothing to collect. Keep cancelling early: a batch stopped before '
+ 'its first request completed costs nothing.'];
}
const lines = [];
if (state === 'cancel-stuck') {
lines.push('a batch still in cancelling or canceling has not stopped. Poll '
+ 'it to a terminal state before you submit a replacement, or the two '
+ 'will run the same rows at once.');
}
if (all.some((r) => (Number(r?.done) || 0) > 0)) {
lines.push('download the partial output, drop those custom_ids from the '
+ 'input file, and submit only the remainder. Results are not returned in '
+ 'request order, so custom_id is the only join key that works.');
}
if (all.some((r) => r?.provider === 'anthropic' && (Number(r.done) || 0) > 0)) {
lines.push('on Anthropic, canceled and expired requests are not billed, so '
+ 'the succeeded count is the whole cost of the cancelled batch.');
}
if (all.some((r) => r?.provider === 'openai' && (Number(r.done) || 0) > 0)) {
lines.push('on OpenAI the billing split for a cancelled batch is not '
+ 'documented, so treat the completed count as a floor and confirm the '
+ 'day against the cost report.');
}
return lines;
}
async function getJson(url, headers, params) {
const target = new URL(url);
for (const [k, v] of Object.entries(params ?? {})) target.searchParams.set(k, String(v));
let res;
try {
res = await fetch(target, { headers });
} catch (err) {
return [null, `request failed: ${err.message}`];
}
if (res.status !== 200) return [null, `HTTP ${res.status}`];
try {
return [await res.json(), null];
} catch {
return [null, 'response was not JSON'];
}
}
async function pageOpenai(key, maxPages) {
const rows = [];
let after = null;
for (let i = 0; i < Math.max(1, maxPages); i += 1) {
const params = { limit: 100 };
if (after) params.after = after;
const [payload, err] = await getJson(OPENAI_BATCHES_URL,
{ Authorization: `Bearer ${key}` }, params);
if (err) return [rows, err];
const data = payload.data ?? [];
rows.push(...data);
if (!payload.has_more || !data.length) break;
after = data[data.length - 1]?.id;
}
return [rows, null];
}
async function pageAnthropic(key, maxPages) {
const rows = [];
let after = null;
for (let i = 0; i < Math.max(1, maxPages); i += 1) {
const params = { limit: 1000 };
if (after) params.after_id = after;
const [payload, err] = await getJson(ANTHROPIC_BATCHES_URL,
{ 'x-api-key': key, 'anthropic-version': '2023-06-01' }, params);
if (err) return [rows, err];
const data = payload.data ?? [];
rows.push(...data);
if (!payload.has_more || !data.length) break;
after = payload.last_id ?? data[data.length - 1]?.id;
}
return [rows, null];
}
function args(argv) {
const out = { stuckMinutes: 15, maxPages: 20 };
for (let i = 0; i < argv.length; i += 1) {
if (argv[i] === '--stuck-minutes') out.stuckMinutes = Number.parseInt(argv[i += 1], 10);
else if (argv[i] === '--max-pages') out.maxPages = Number.parseInt(argv[i += 1], 10);
}
return out;
}
async function main() {
const opts = args(process.argv.slice(2));
const openaiKey = process.env.OPENAI_API_KEY;
const anthropicKey = process.env.ANTHROPIC_API_KEY;
if (!openaiKey && !anthropicKey) {
console.error('set OPENAI_API_KEY (project key, Read Only) or '
+ 'ANTHROPIC_API_KEY (workspace key), or both');
process.exitCode = 2;
return;
}
const now = Math.floor(Date.now() / 1000);
const rows = [];
const checked = [];
if (openaiKey) {
checked.push('openai');
const [batches, err] = await pageOpenai(openaiKey, opts.maxPages);
if (err) console.log(`openai batch list stopped early: ${err}`);
rows.push(...openaiCancelRows(batches));
}
if (anthropicKey) {
checked.push('anthropic');
const [batches, err] = await pageAnthropic(anthropicKey, opts.maxPages);
if (err) console.log(`anthropic batch list stopped early: ${err}`);
rows.push(...anthropicCancelRows(batches));
}
const stuck = stuckRows(rows, now, Math.max(1, opts.stuckMinutes) * 60);
const salvage = salvageRows(rows);
const stuckIds = new Set(stuck.map((r) => r.id));
for (const r of rows) {
console.log(`${r.provider.padEnd(11)} ${r.id.slice(0, 14).padEnd(14)} `
+ `${r.status.padEnd(11)} ${r.done} of ${r.total} done, ${r.stopped} stopped`);
if (r.artifact) {
const label = r.provider === 'openai' ? 'output_file_id' : 'results_url';
console.log(`${''.padEnd(27)} ${label} present`);
}
if (stuckIds.has(r.id)) {
const age = r.cancelStarted === null || r.cancelStarted === undefined
? 'an unknown time'
: `${Math.floor((now - r.cancelStarted) / 60)} min`;
console.log(`${r.provider.padEnd(11)} ${r.id.slice(0, 14).padEnd(14)} `
+ `mid cancel for ${age}`);
}
}
const [state, detail] = verdict(rows, stuck, salvage);
console.log(`${state.padEnd(20)} ${detail}`);
console.log(` checked: ${checked.join(', ')}`);
console.log(' measured: request_counts and the cancellation timestamps from '
+ 'the batch lists');
console.log(' inferred: that a re-run would repeat the finished rows, since '
+ 'neither API records whether the partial output was downloaded');
for (const line of repairLines(state, rows)) console.log(` repair: ${line}`);
const total = stuck.length + salvage.length;
console.log(`${total} finding(s)`);
process.exitCode = total ? 1 : 0;
void FINDINGS;
}
if (import.meta.url === `file://${process.argv[1]}`) await main();
Add a test
The first test is the arithmetic the repair rests on, run twice because the two providers spell it differently: OpenAI's {total, completed, failed} and Anthropic's {processing, succeeded, errored, canceled, expired} have to normalise to the same row, with the same finished count and the same stopped count. The second is the timestamp parser, which is handed a unix integer, an RFC 3339 string with a fractional second, one with an offset, and several kinds of rubbish, because a cancel age computed from a misparsed string is worse than no age at all. The third pins the stuck threshold to an argument rather than a clock, and includes the case where the timestamp is missing entirely — which has to count as stuck rather than as fine. The fourth is the ordering of verdicts: an unlanded cancel outranks a salvageable one, because billing may not have stopped. Then the clean cancel that produces no finding. And last the repair text, which must state the Anthropic billing rule and must not state an OpenAI one, since that one is not documented.
from batch_cancellation_audit import (anthropic_cancel_rows, openai_cancel_rows,
parse_time, repair_lines, salvage_rows,
salvaged_total, stuck_rows, verdict)
NOW = 1_800_000_000
OPENAI = [
{"id": "batch_c1", "status": "cancelled",
"request_counts": {"total": 90000, "completed": 61204, "failed": 0},
"output_file_id": "file_7ac1", "cancelling_at": NOW - 7200,
"cancelled_at": NOW - 6900},
{"id": "batch_c2", "status": "cancelling",
"request_counts": {"total": 400, "completed": 0, "failed": 0},
"cancelling_at": NOW - 68 * 60},
{"id": "batch_ok", "status": "completed",
"request_counts": {"total": 10, "completed": 10, "failed": 0}},
]
ANTHROPIC = [
{"id": "msgbatch_01Hq", "processing_status": "ended",
"cancel_initiated_at": "2026-08-20T18:37:24.100435Z",
"request_counts": {"processing": 0, "succeeded": 41880, "errored": 0,
"canceled": 12120, "expired": 0},
"results_url": "https://api.anthropic.com/v1/messages/batches/x/results"},
{"id": "msgbatch_02Zz", "processing_status": "in_progress",
"cancel_initiated_at": None,
"request_counts": {"processing": 500, "succeeded": 0, "errored": 0,
"canceled": 0, "expired": 0}},
]
def test_two_providers_normalise_to_one_row_shape():
rows = openai_cancel_rows(OPENAI) + anthropic_cancel_rows(ANTHROPIC)
assert [r["id"] for r in rows] == ["batch_c1", "batch_c2", "msgbatch_01Hq"]
first = rows[0]
assert first["done"] == 61204 and first["stopped"] == 28796
assert first["total"] == 90000 and first["artifact"] == "file_7ac1"
last = rows[2]
# succeeded/canceled on Anthropic, and the counts sum to the total.
assert last["done"] == 41880 and last["stopped"] == 12120
assert last["total"] == 54000
assert salvaged_total(rows) == 61204 + 41880
# A batch with no cancellation initiated is never a row here.
assert all(r["id"] != "msgbatch_02Zz" for r in rows)
def test_the_timestamp_parser_takes_both_providers_and_refuses_rubbish():
assert parse_time(NOW) == NOW
assert parse_time("2026-08-20T18:37:24Z") == 1787251044
assert parse_time("2026-08-20T18:37:24.100435Z") == 1787251044
assert parse_time("2026-08-20T18:37:24+00:00") == 1787251044
for junk in (None, "", "yesterday", True, {}):
assert parse_time(junk) is None
def test_a_stuck_cancel_is_measured_against_an_argument_not_a_clock():
rows = openai_cancel_rows(OPENAI)
stuck = stuck_rows(rows, NOW, 15 * 60)
assert [r["id"] for r in stuck] == ["batch_c2"]
# Generous threshold: the same batch is not stuck against a two hour one.
assert stuck_rows(rows, NOW, 3 * 3600) == []
# A missing cancelling_at on an in-flight cancel counts as stuck, because
# "we cannot tell how long" is not the same as "it is fine".
unknown = [{"id": "batch_x", "in_flight": True, "cancel_started": None,
"done": 0}]
assert stuck_rows(unknown, NOW, 15 * 60) == unknown
# A terminal cancellation is never stuck however old it is.
assert stuck_rows([rows[0]], NOW, 1) == []
def test_an_unlanded_cancel_outranks_a_salvageable_one():
rows = openai_cancel_rows(OPENAI) + anthropic_cancel_rows(ANTHROPIC)
stuck = stuck_rows(rows, NOW, 15 * 60)
salvage = salvage_rows(rows)
state, detail = verdict(rows, stuck, salvage)
assert state == "cancel-stuck"
assert "mid cancel" in detail and "103084 finished rows" in detail
# Without the stuck one it drops to the salvage verdict.
state2, detail2 = verdict(rows, [], salvage)
assert state2 == "cancel-partial-unclaimed"
assert "pay for again" in detail2
def test_a_cancel_that_landed_before_anything_ran_is_not_a_finding():
early = [{"id": "batch_z", "provider": "openai", "status": "cancelled",
"in_flight": False, "done": 0, "stopped": 400, "total": 400,
"artifact": None, "cancel_started": NOW - 86400}]
assert salvage_rows(early) == []
state, detail = verdict(early, [], [])
assert state == "cancel-clean"
assert "nothing to salvage" in detail
assert repair_lines(state, early)[0].startswith("nothing to collect")
assert verdict([], [], []) == ("no-cancels",
"no batch on the providers checked has had a "
"cancellation initiated")
assert repair_lines("no-cancels", []) == []
def test_the_repair_states_the_documented_billing_rule_and_only_that():
rows = openai_cancel_rows(OPENAI) + anthropic_cancel_rows(ANTHROPIC)
lines = repair_lines("cancel-partial-unclaimed", rows)
assert any("custom_id is the only join key" in line for line in lines)
assert any("canceled and expired requests are not billed" in line
for line in lines)
assert any("not documented" in line and "floor" in line for line in lines)
# Anthropic only: no claim is made about OpenAI billing.
only_anthropic = anthropic_cancel_rows(ANTHROPIC)
lines2 = repair_lines("cancel-partial-unclaimed", only_anthropic)
assert not any("floor" in line for line in lines2)
assert any("cancelling or canceling has not stopped" in line
for line in repair_lines("cancel-stuck", rows))
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { anthropicCancelRows, openaiCancelRows, parseTime, repairLines,
salvageRows, salvagedTotal, stuckRows,
verdict } from './batch-cancellation-audit.mjs';
const NOW = 1800000000;
const OPENAI = [
{ id: 'batch_c1', status: 'cancelled',
request_counts: { total: 90000, completed: 61204, failed: 0 },
output_file_id: 'file_7ac1', cancelling_at: NOW - 7200, cancelled_at: NOW - 6900 },
{ id: 'batch_c2', status: 'cancelling',
request_counts: { total: 400, completed: 0, failed: 0 },
cancelling_at: NOW - 68 * 60 },
{ id: 'batch_ok', status: 'completed',
request_counts: { total: 10, completed: 10, failed: 0 } },
];
const ANTHROPIC = [
{ id: 'msgbatch_01Hq', processing_status: 'ended',
cancel_initiated_at: '2026-08-20T18:37:24.100435Z',
request_counts: { processing: 0, succeeded: 41880, errored: 0, canceled: 12120,
expired: 0 },
results_url: 'https://api.anthropic.com/v1/messages/batches/x/results' },
{ id: 'msgbatch_02Zz', processing_status: 'in_progress', cancel_initiated_at: null,
request_counts: { processing: 500, succeeded: 0, errored: 0, canceled: 0,
expired: 0 } },
];
test('two providers normalise to one row shape', () => {
const rows = [...openaiCancelRows(OPENAI), ...anthropicCancelRows(ANTHROPIC)];
assert.deepEqual(rows.map((r) => r.id), ['batch_c1', 'batch_c2', 'msgbatch_01Hq']);
assert.equal(rows[0].done, 61204);
assert.equal(rows[0].stopped, 28796);
assert.equal(rows[0].total, 90000);
assert.equal(rows[0].artifact, 'file_7ac1');
assert.equal(rows[2].done, 41880);
assert.equal(rows[2].stopped, 12120);
assert.equal(rows[2].total, 54000);
assert.equal(salvagedTotal(rows), 61204 + 41880);
assert.ok(rows.every((r) => r.id !== 'msgbatch_02Zz'));
});
test('the timestamp parser takes both providers and refuses rubbish', () => {
assert.equal(parseTime(NOW), NOW);
assert.equal(parseTime('2026-08-20T18:37:24Z'), 1787251044);
assert.equal(parseTime('2026-08-20T18:37:24.100435Z'), 1787251044);
assert.equal(parseTime('2026-08-20T18:37:24+00:00'), 1787251044);
for (const junk of [null, undefined, '', 'yesterday', true, {}]) {
assert.equal(parseTime(junk), null);
}
});
test('a stuck cancel is measured against an argument not a clock', () => {
const rows = openaiCancelRows(OPENAI);
assert.deepEqual(stuckRows(rows, NOW, 15 * 60).map((r) => r.id), ['batch_c2']);
assert.deepEqual(stuckRows(rows, NOW, 3 * 3600), []);
const unknown = [{ id: 'batch_x', inFlight: true, cancelStarted: null, done: 0 }];
assert.deepEqual(stuckRows(unknown, NOW, 15 * 60), unknown);
assert.deepEqual(stuckRows([rows[0]], NOW, 1), []);
});
test('an unlanded cancel outranks a salvageable one', () => {
const rows = [...openaiCancelRows(OPENAI), ...anthropicCancelRows(ANTHROPIC)];
const stuck = stuckRows(rows, NOW, 15 * 60);
const salvage = salvageRows(rows);
const [state, detail] = verdict(rows, stuck, salvage);
assert.equal(state, 'cancel-stuck');
assert.ok(detail.includes('mid cancel') && detail.includes('103084 finished rows'));
const [state2, detail2] = verdict(rows, [], salvage);
assert.equal(state2, 'cancel-partial-unclaimed');
assert.ok(detail2.includes('pay for again'));
});
test('a cancel that landed before anything ran is not a finding', () => {
const early = [{ id: 'batch_z', provider: 'openai', status: 'cancelled',
inFlight: false, done: 0, stopped: 400, total: 400,
artifact: null, cancelStarted: NOW - 86400 }];
assert.deepEqual(salvageRows(early), []);
const [state, detail] = verdict(early, [], []);
assert.equal(state, 'cancel-clean');
assert.ok(detail.includes('nothing to salvage'));
assert.ok(repairLines(state, early)[0].startsWith('nothing to collect'));
assert.deepEqual(verdict([], [], []),
['no-cancels', 'no batch on the providers checked has had a cancellation initiated']);
assert.deepEqual(repairLines('no-cancels', []), []);
});
test('the repair states the documented billing rule and only that', () => {
const rows = [...openaiCancelRows(OPENAI), ...anthropicCancelRows(ANTHROPIC)];
const lines = repairLines('cancel-partial-unclaimed', rows);
assert.ok(lines.some((l) => l.includes('custom_id is the only join key')));
assert.ok(lines.some((l) => l.includes('canceled and expired requests are not billed')));
assert.ok(lines.some((l) => l.includes('not documented') && l.includes('floor')));
const lines2 = repairLines('cancel-partial-unclaimed', anthropicCancelRows(ANTHROPIC));
assert.ok(!lines2.some((l) => l.includes('floor')));
assert.ok(repairLines('cancel-stuck', rows)
.some((l) => l.includes('cancelling or canceling has not stopped')));
});
FAQ
If I cancel a batch, am I charged for it?
For the requests that had already been processed, yes. Anthropic documents the rule precisely: requests that end up <code>canceled</code> or <code>expired</code> were never sent to the model and are not billed, while the ones that succeeded were, and their results are in the batch's output. OpenAI documents that a cancelled batch keeps the partial results it produced but does not publish a billing split, so this script treats the <code>completed</code> count as a floor on what you paid there and tells you to confirm the day against the cost report rather than guessing.
Why is my batch stuck in cancelling?
Because cancellation is asynchronous. OpenAI's documentation gives the window explicitly: the batch moves to <code>cancelling</code> and may stay there for up to ten minutes before it reaches <code>cancelled</code>. Anthropic's <code>canceling</code> state lasts until the requests already in flight finish and publishes no bound, so the only hard ceiling on that side is the batch's own <code>expires_at</code>, which is 24 hours after creation. Past the window, poll it to a terminal state before you assume the work has stopped.
Can I re-run only the rows that did not complete?
Yes, and that is the whole repair. Download the partial output, collect the <code>custom_id</code> of every result that came back, remove those lines from the original input file, and submit the remainder. Results are explicitly not guaranteed to come back in request order on either provider, so <code>custom_id</code> is the only join key that works — which is also why a batch with duplicate or missing <code>custom_id</code> values cannot be reconciled at all and fails validation on OpenAI in the first place.
Does this script cancel or clean anything up?
No. It issues GET requests to the two batch list endpoints and nothing else. It does not cancel a batch, does not download a result file, does not delete anything and does not re-submit the remainder, because every one of those either costs money or destroys evidence. What it produces is a list of ids with counts next to them and a printed repair you run yourself, after you have decided the rows are still wanted.
The batch shows cancelled but request_counts is all zeros. Is that a problem?
No, that is the good outcome and the script reports it as <code>cancel-clean</code> with no finding. A cancellation that landed before any request completed leaves nothing to salvage, nothing that was billed, and nothing a re-run would pay for twice. It is worth distinguishing loudly from the two-thirds-finished case, because if both print the same way people stop reading the output, and then the case that matters gets skimmed past too.
Related field notes
- A batch that ran, reads completed, and has failed rows inside it
- Work you paid for and never collected, on both providers
- Why the work was worth batching in the first place
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Batch — OpenAI API reference
- Batch API guide — OpenAI developer docs
- Batch processing — Claude platform docs
- List Message Batches — Claude API reference
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.