Diagnostic LLM APIs
The flex tier fails by not being served, and bills nothing
The nightly enrichment job was moved to flex processing because it is not urgent and flex is priced like batch. It has been fine for six weeks. It is still fine, in the sense that nothing has ever paged: the job logs a count at the end, the count is lower some nights, and the difference is a few thousand records that quietly did not get enriched. The invoice is lower too, which is exactly what everybody expected to see, and which is why nobody looked.
One GET with an admin key: GET /v1/organization/usage/completions?start_time={T-7d}&bucket_width=1h&limit=168&group_by[]=service_tier&group_by[]=model. Every result carries num_model_requests, input_tokens, output_tokens and, because of the grouping, the service_tier the request was actually served on.
Two findings, and they are opposite mistakes. The first is a model you configured for flex that has no flex rows at all while it has plenty on default: the parameter is not reaching the API, so you are paying standard rates for a workload you believe is discounted. That needs the model names as input, which is why the script takes --flex-model; the platform cannot tell you what you meant to ask for.
The second is the one this note is named after. A 429 Resource Unavailable means flex had no capacity right now, and it is explicitly not charged. An unbilled request does not appear in a usage report, so those failures leave no row, no error count and no token anywhere in the API. The only trace is a hole: hours where flex volume collapses while the same organization is demonstrably still serving traffic on other tiers.
Absence is a weaker signal than an error, and the script treats it that way. It compares each hour's flex request count against the median of the hours that did get served, requires the organization to have been active in that hour before calling it a gap, and refuses to grade at all on fewer than a handful of served hours.
The cost report cannot help here: its group_by accepts project_id, line_item and api_key_id, and nothing about service tiers. The usage endpoint is the only place the tiers are separated, which is why this note reads one endpoint and not two.
The problem in plain words
Flex processing trades latency and availability for much cheaper tokens, which is a good trade for evals, backfills, enrichment and anything else that runs overnight. The part that surprises people is how it declines. It does not queue and it does not degrade: when there is no capacity it answers 429 Resource Unavailable, and you are not charged for it.
Not being charged is the humane behaviour and it is also why this is invisible. The organization usage endpoints report what was processed. A request that was refused for lack of capacity was not processed, produced no tokens, and does not increment num_model_requests. There is no error counter on the API, no failed-request feed, and no field anywhere that says how many times flex said no. The only place those failures exist is in your own client logs, and the client that swallowed them is usually a batch script whose logs roll over.
The second failure mode is the clock. The official SDKs default to a ten-minute request timeout, and flex responses regularly take longer than standard ones, so the recommendation is to raise the client timeout to at least fifteen minutes. A client left on the default aborts a request that the server may well go on to finish, which means the tokens can be billed to an answer nobody received. The SDKs then retry a 408 automatically, which triples the wall clock before anything surfaces to a human.
And the third is the quiet one: flex not being applied at all. service_tier is a request parameter, and a gateway that normalises request bodies, an SDK wrapper with its own defaults, or a code path that never set it will simply send the request at standard tier. Every one of those requests succeeds. The bill is the only place it shows up, and it shows up as a number that is higher than expected rather than as anything that looks like a fault.
Why it happens
The evidence is a hole, so the script has to be honest about how weak that is. Every other reading in this section counts something that happened. This one counts something that did not, which is one inference further from the data, and inference dressed up as measurement is how a diagnostic loses its credibility. So the gap test has three guards: the hour must be below half the median of the hours that were served, the organization must have been serving something on some tier in that same hour, and there must be enough served hours to have a median worth comparing against. Below that, the script says it cannot tell.
The tier in the report is the tier that was served, not the tier that was asked for. That is the property that makes the whole reading possible. The service_tier on a response reflects the processing mode actually used, and the grouped usage report inherits that. So flex rows are proof of flex being served, and their absence next to a healthy default row for the same model is proof that something dropped your parameter on the way.
Which models you meant to run on flex is not knowable from the API, so it is an input. Nothing OpenAI returns knows what your code sends. A script that guessed — by treating any model with no flex rows as misconfigured, say — would report every model you deliberately run at standard tier as a fault. Passing the model ids explicitly is the difference between a finding and a list of every model you own.
The cost report is the wrong instrument and saying so is part of the answer. It is the natural place to look, and its group_by supports project_id, line_item and api_key_id only. There is no service tier dimension on it, so flex spend cannot be separated from standard spend there at any granularity. The reading has to be done in the usage endpoint, in requests and tokens, and converted to money by you.
The timeout cannot be read from the API at all, so it is printed as a repair rather than detected. Client configuration lives in your source tree and nothing either provider returns describes it. The script says what the floor should be and why, and does not pretend to have measured it.
The fix, as a flow
This chain ends in a box that is not an error, which is unusual enough to be the point. Capacity was refused, nothing was charged, and an unbilled request never reaches a usage report, so there is no row to find and no counter to read. What is left is a hole in the hours, and reading a hole responsibly means proving the organization was awake at the time.
How to fix it
Use an admin key and name the models you configured for flex
/v1/organization/usage/completions needs an admin key. Pass --flex-model gpt-5.6 once per model id your code sends service_tier: "flex" for. Without them the script can still find shortfalls, but it cannot tell a tier that was never served from a tier you never requested, and it says which of the two checks it had to skip.
Read seven days of hourly buckets grouped by tier and model
bucket_width=1h allows up to 168 buckets, which is exactly a week. Group by service_tier and model; both appear as fields on each result, null when you did not group by them. Page on next_page until has_more is false.
Check the models you named for any flex row at all
A named model with zero flex requests and a healthy default count is the parameter never arriving. Print the tiers it actually ran on and the request counts, because that is the sentence that ends the argument about whether the gateway rewrites bodies.
Compare each hour's flex volume against the median served hour
Build the per-hour flex request counts, take the median over the hours that were served, and flag hours at or below half of it — but only where some tier served something that hour, so a quiet night is not reported as a capacity failure. Fewer than a handful of served hours means no median worth having, and the script declines rather than guessing.
Read the repairs, none of which the script performs
Raise the client timeout to at least fifteen minutes, back off on 429 Resource Unavailable rather than treating it as a limit you exceeded, and fall back to service_tier: "auto" when completion matters more than price. Keep flex on evals, enrichment and background work, and off anything a person is waiting for.
How to check it worked
Fix the parameter or the timeout and re-run a week later. A model that was flex-never-served should now carry flex rows with request counts in the same order as its old default counts, and the shortfall count should fall. A run that stays at flex-shortfall after the client is correct is telling you something real about capacity at those hours, and the repair for that is scheduling rather than code.
python3 openai_flex_tier_served.py --flex-model gpt-5.6 --days 7
# 168 hourly bucket(s), 4 tier(s) observed: batch, default, flex, priority
# flex-never-served gpt-5.6: 0 flex request(s) in 7 days, and 41,208 on
# default. The service_tier parameter is not reaching
# the API.
# repair: the tier in this report is the tier that was served. Check for a
# gateway that rewrites request bodies, an SDK wrapper with its own
# defaults, or a code path that never set service_tier at all.
# flex-shortfall gpt-5.3-mini: 9 hour(s) at or below half the median
# served hour (median 2,140 requests)
# 2026-08-27T02:00Z 0 requests, other tiers served 8,400 that hour
# 2026-08-28T02:00Z 130 requests, other tiers served 7,900 that hour
# note: a 429 Resource Unavailable is not charged and never reaches this
# report, so these hours are absence rather than error counts.
# repair: back off and retry on 429 Resource Unavailable, which means no
# capacity rather than a limit you exceeded.
# repair: raise the client timeout to at least 15 minutes. The SDK default
# is 10 and flex responses regularly exceed it.
# 2 finding(s)The full code
One paged GET and seven pure functions. tier_rows, which folds the buckets into {(model, tier): {hour: counts}} and keeps hours with no rows out rather than inventing zeros; totals_by_tier; hours_active, which records how much every tier together served in each hour, because that is the control that stops a quiet night reading as a capacity failure; median, chosen over the mean because one enormous backfill hour would drag a mean far enough to hide the gaps; flex_gaps, which applies the three guards and returns nothing when there is not enough served history to have an opinion; never_served, which needs the configured model ids as input because the API cannot know them; and verdict.
"""Find flex tier work that was never served, and flex you never actually asked for.
Read only. One paged GET with an admin key:
GET /v1/organization/usage/completions
?bucket_width=1h&group_by[]=service_tier&group_by[]=model
The tier on each result is the tier the request was actually served on, which is
what makes this readable at all. Two opposite findings come out of it: a model
configured for flex with no flex rows anywhere (the parameter is not arriving),
and hours where flex volume collapses while other tiers keep serving (capacity
was refused).
A 429 Resource Unavailable is explicitly not charged, so it never appears in any
usage report. The evidence for it is a hole, which is one inference further from
the data than everything else in this section, so the gap test is deliberately
conservative: below half the median served hour, in an hour the organization was
otherwise active, with enough served hours to have a median worth comparing to.
The cost report cannot substitute: its group_by accepts project_id, line_item
and api_key_id and has no service tier dimension at all.
"""
import argparse
import datetime as dt
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("openai_flex_tier_served")
API = "https://api.openai.com/v1"
FLEX = "flex"
# Enough served hours to have a median worth comparing against. Below this the
# script says it cannot tell rather than grading two data points.
MIN_SERVED_HOURS = 6
FINDINGS = ("flex-never-served", "flex-shortfall")
def num(value):
"""A float, or 0.0. Pure."""
if value is None or isinstance(value, bool):
return 0.0
try:
return float(value)
except (TypeError, ValueError):
return 0.0
def tier_rows(pages):
"""{(model, tier): {hour: {"requests", "input", "output"}}}. Pure.
Hours with no row for a pairing are simply absent. Filling them with zeros
here would destroy the difference between "served nothing" and "was not
grouped this way", which is the whole subject.
"""
out = {}
for page in pages or []:
for bucket in ((page or {}).get("data") or []):
hour = int(num((bucket or {}).get("start_time")))
for result in ((bucket or {}).get("results") or []):
result = result or {}
key = (str(result.get("model") or "(all models)"),
str(result.get("service_tier") or "(untiered)"))
row = out.setdefault(key, {}).setdefault(
hour, {"requests": 0.0, "input": 0.0, "output": 0.0})
row["requests"] += num(result.get("num_model_requests"))
row["input"] += num(result.get("input_tokens"))
row["output"] += num(result.get("output_tokens"))
return out
def totals_by_tier(rows):
"""{tier: total requests}. Pure."""
out = {}
for (_model, tier), hours in (rows or {}).items():
out[tier] = out.get(tier, 0.0) + sum(h["requests"] for h in hours.values())
return out
def hours_active(rows):
"""{hour: requests served across every tier}. Pure.
The control. Without it a night when the job did not run reads exactly like
a night when flex refused every request.
"""
out = {}
for hours in (rows or {}).values():
for hour, counts in hours.items():
out[hour] = out.get(hour, 0.0) + counts["requests"]
return out
def median(values):
"""The median of a list. Pure. 0.0 when empty.
Median rather than mean on purpose: one enormous backfill hour drags a mean
high enough to swallow the very gaps this is looking for.
"""
ordered = sorted(float(v) for v in (values or []))
if not ordered:
return 0.0
mid = len(ordered) // 2
if len(ordered) % 2:
return ordered[mid]
return (ordered[mid - 1] + ordered[mid]) / 2.0
def flex_by_hour(rows, model):
"""{hour: flex requests} for one model. Pure."""
return {hour: counts["requests"]
for hour, counts in ((rows or {}).get((model, FLEX)) or {}).items()}
def tiers_for_model(rows, model):
"""{tier: requests} for one model across every tier. Pure."""
out = {}
for (candidate, tier), hours in (rows or {}).items():
if candidate != model:
continue
out[tier] = out.get(tier, 0.0) + sum(h["requests"] for h in hours.values())
return out
def flex_gaps(flex_hours, active, floor=0.5, min_served=MIN_SERVED_HOURS):
"""[(hour, flex_requests, other_requests, median)] where flex collapsed. Pure.
Three guards, all of them there to stop absence being over-read. The hour
must be at or below floor times the median served hour; some tier must have
served something in that same hour; and there must be at least min_served
hours of flex traffic to take a median from at all.
"""
served = [v for v in (flex_hours or {}).values() if v > 0]
if len(served) < min_served:
return []
mid = median(served)
if mid <= 0:
return []
out = []
for hour, total in sorted((active or {}).items()):
flex = float((flex_hours or {}).get(hour, 0.0))
other = float(total) - flex
if flex <= mid * floor and other > 0:
out.append((hour, flex, other, mid))
out.sort(key=lambda r: (r[1], r[0]))
return out
def never_served(rows, configured):
"""[(model, flex_requests, {tier: requests})] for models with no flex rows.
Pure. configured is the list of model ids your code sends flex for, because
nothing the API returns knows what you meant to ask for. Without it this
check cannot run, and guessing would report every deliberately standard
model as a fault.
"""
out = []
for model in sorted(set(str(m) for m in (configured or []) if m)):
tiers = tiers_for_model(rows, model)
if tiers.get(FLEX, 0.0) > 0:
continue
if sum(tiers.values()) <= 0:
continue
out.append((model, 0.0, tiers))
return out
def verdict(model, flex_hours, gaps, tiers, configured):
"""Classify one model. Pure. Returns (state, detail)."""
tiers = tiers or {}
flex_total = tiers.get(FLEX, 0.0)
other_total = sum(v for t, v in tiers.items() if t != FLEX)
if flex_total <= 0 and model in set(configured or []):
if other_total <= 0:
return ("no-usage", "no requests on any tier in this window")
return ("flex-never-served",
"%s flex request(s) in this window, and %s on other tiers. The "
"service_tier parameter is not reaching the API."
% (fmt(flex_total), fmt(other_total)))
if flex_total <= 0:
return ("no-flex-usage", "never served on flex in this window")
if gaps:
mid = gaps[0][3]
return ("flex-shortfall",
"%d hour(s) at or below half the median served hour (median %s "
"requests)" % (len(gaps), fmt(mid)))
served = len([v for v in (flex_hours or {}).values() if v > 0])
if served < MIN_SERVED_HOURS:
return ("too-little-history",
"%d hour(s) of flex traffic, which is not enough to take a "
"median from" % served)
return ("flex-served",
"%s flex request(s) across %d hour(s), no collapsed hours"
% (fmt(flex_total), served))
def fmt(value):
"""Thousands separators. Pure."""
return "{:,}".format(int(round(num(value))))
def stamp(hour):
"""An hour bucket's start as a readable UTC string. Pure."""
return dt.datetime.fromtimestamp(int(hour), dt.timezone.utc).strftime(
"%Y-%m-%dT%H:00Z")
def repair_lines(state):
"""The repair for one verdict. Pure. Printed, never performed."""
if state == "flex-never-served":
return ["the tier in this report is the tier that was served. Check for a "
"gateway that rewrites request bodies, an SDK wrapper with its "
"own defaults, or a code path that never set service_tier at all.",
"until it arrives you are paying standard rates for a workload "
"you believe is discounted, and nothing will raise about it."]
if state == "flex-shortfall":
return ["back off and retry on 429 Resource Unavailable, which means no "
"capacity right now rather than a limit you exceeded. Retrying "
"it genuinely helps, unlike the billing 429s.",
"raise the client timeout to at least 15 minutes. The official "
"SDK default is 10 and flex responses regularly exceed it, and "
"an aborted request can still be billed if the server finishes.",
"fall back to service_tier auto when completing the work matters "
"more than the discount, and keep flex off anything a person is "
"waiting for."]
if state == "too-little-history":
return ["not a clean bill of health, just too little to read. Re-run over "
"a longer window once the job has more served hours behind it."]
return []
def get(session, path, **params):
r = session.get(API + path, params=params, timeout=90)
if r.status_code in (401, 403):
raise SystemExit("%d from OpenAI: the organization usage endpoints need "
"an admin key" % r.status_code)
r.raise_for_status()
return r.json()
def pages(session, path, **params):
params = dict(params)
for _ in range(50):
page = get(session, path, **params)
yield page
if not page.get("has_more") or not page.get("next_page"):
return
params["page"] = page["next_page"]
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--flex-model", action="append", default=[],
help="a model id your code sends service_tier flex for "
"(repeatable)")
ap.add_argument("--days", type=float, default=7.0,
help="window in days (max 7 at hourly buckets)")
ap.add_argument("--floor", type=float, default=0.5,
help="share of the median served hour below which an hour "
"counts as collapsed")
args = ap.parse_args()
key = os.environ.get("OPENAI_ADMIN_KEY")
if not key:
log.error("set OPENAI_ADMIN_KEY to an admin key that can read the "
"organization usage endpoints")
return 2
days = max(0.5, min(7.0, args.days))
s = requests.Session()
s.headers.update({"Authorization": "Bearer " + key,
"User-Agent": "openai-flex-tier-served/1.0"})
now = dt.datetime.now(dt.timezone.utc)
start = int((now - dt.timedelta(days=days)).timestamp())
payloads = list(pages(s, "/organization/usage/completions",
start_time=start, bucket_width="1h", limit=168,
**{"group_by[]": ["service_tier", "model"]}))
rows = tier_rows(payloads)
totals = totals_by_tier(rows)
active = hours_active(rows)
buckets = sum(len(page.get("data") or []) for page in payloads)
log.info("%d hourly bucket(s), %d tier(s) observed: %s",
buckets, len(totals), ", ".join(sorted(totals)) or "none")
configured = [str(m) for m in args.flex_model if m]
if not configured:
log.info("no --flex-model given, so the never-served check is skipped: "
"nothing the API returns knows which models your code asks for "
"flex on")
findings = 0
models = sorted({model for model, _tier in rows} | set(configured))
for model in models:
flex_hours = flex_by_hour(rows, model)
gaps = flex_gaps(flex_hours, active, args.floor)
tiers = tiers_for_model(rows, model)
state, detail = verdict(model, flex_hours, gaps, tiers, configured)
if state in ("no-flex-usage", "no-usage") and model not in configured:
continue
emit = log.warning if state in FINDINGS else log.info
emit("%-21s %s: %s", state, model, detail)
for hour, flex, other, _mid in gaps[:5]:
emit(" %s %s requests, other tiers served %s that hour",
stamp(hour), fmt(flex), fmt(other))
if state == "flex-shortfall":
emit(" note: a 429 Resource Unavailable is not charged and never "
"reaches this report, so these hours are absence rather than "
"error counts.")
for line in repair_lines(state):
emit(" repair: %s", line)
if state in FINDINGS:
findings += 1
for model, _flex, tiers in never_served(rows, configured):
if model in models:
continue
log.warning("%-21s %s: served only on %s", "flex-never-served", model,
", ".join("%s (%s)" % (t, fmt(v))
for t, v in sorted(tiers.items())))
findings += 1
log.info("%d finding(s)", findings)
return 1 if findings else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Find flex tier work that was never served, and flex you never actually asked for.
*
* Read only. One paged GET of the completions usage report, grouped by
* service_tier and model. The tier on each result is the tier the request was
* actually served on, which is what makes this readable at all.
*
* A 429 Resource Unavailable is explicitly not charged, so it never appears in
* any usage report: the evidence is a hole. The gap test is deliberately
* conservative because absence is one inference further from the data than
* everything else in this section.
*
* The cost report cannot substitute: its group_by accepts project_id, line_item
* and api_key_id and has no service tier dimension at all.
*/
const API = 'https://api.openai.com/v1';
const FLEX = 'flex';
export const MIN_SERVED_HOURS = 6;
const FINDINGS = new Set(['flex-never-served', 'flex-shortfall']);
/** A number, or 0. Pure. */
export function num(value) {
if (value === null || value === undefined || typeof value === 'boolean') return 0;
const n = Number(value);
return Number.isFinite(n) ? n : 0;
}
const key = (model, tier) => `${model}\u0000${tier}`;
/** {"model\0tier": {hour: {requests, input, output}}}. Pure. */
export function tierRows(pages) {
const out = {};
for (const page of pages ?? []) {
for (const bucket of page?.data ?? []) {
const hour = Math.trunc(num(bucket?.start_time));
for (const result of bucket?.results ?? []) {
const k = key(String(result?.model ?? '(all models)'),
String(result?.service_tier ?? '(untiered)'));
const hours = (out[k] ??= {});
const row = (hours[hour] ??= { requests: 0, input: 0, output: 0 });
row.requests += num(result?.num_model_requests);
row.input += num(result?.input_tokens);
row.output += num(result?.output_tokens);
}
}
}
return out;
}
/** {tier: total requests}. Pure. */
export function totalsByTier(rows) {
const out = {};
for (const [k, hours] of Object.entries(rows ?? {})) {
const tier = k.split('\u0000')[1];
out[tier] = (out[tier] ?? 0)
+ Object.values(hours).reduce((n, h) => n + h.requests, 0);
}
return out;
}
/** {hour: requests served across every tier}. Pure. The control. */
export function hoursActive(rows) {
const out = {};
for (const hours of Object.values(rows ?? {})) {
for (const [hour, counts] of Object.entries(hours)) {
out[hour] = (out[hour] ?? 0) + counts.requests;
}
}
return out;
}
/** The median of a list. Pure. 0 when empty. */
export function median(values) {
const ordered = [...(values ?? [])].map(Number).sort((a, b) => a - b);
if (ordered.length === 0) return 0;
const mid = Math.floor(ordered.length / 2);
return ordered.length % 2 ? ordered[mid] : (ordered[mid - 1] + ordered[mid]) / 2;
}
/** {hour: flex requests} for one model. Pure. */
export function flexByHour(rows, model) {
const hours = (rows ?? {})[key(model, FLEX)] ?? {};
return Object.fromEntries(Object.entries(hours).map(([h, c]) => [h, c.requests]));
}
/** {tier: requests} for one model across every tier. Pure. */
export function tiersForModel(rows, model) {
const out = {};
for (const [k, hours] of Object.entries(rows ?? {})) {
const [candidate, tier] = k.split('\u0000');
if (candidate !== model) continue;
out[tier] = (out[tier] ?? 0)
+ Object.values(hours).reduce((n, h) => n + h.requests, 0);
}
return out;
}
/** [[hour, flexRequests, otherRequests, median]] where flex collapsed. Pure. */
export function flexGaps(flexHours, active, floor = 0.5, minServed = MIN_SERVED_HOURS) {
const served = Object.values(flexHours ?? {}).filter((v) => v > 0);
if (served.length < minServed) return [];
const mid = median(served);
if (mid <= 0) return [];
const out = [];
for (const hour of Object.keys(active ?? {}).sort((a, b) => Number(a) - Number(b))) {
const flex = Number((flexHours ?? {})[hour] ?? 0);
const other = Number(active[hour]) - flex;
if (flex <= mid * floor && other > 0) out.push([Number(hour), flex, other, mid]);
}
out.sort((a, b) => (a[1] - b[1]) || (a[0] - b[0]));
return out;
}
/** [[model, 0, {tier: requests}]] for configured models with no flex rows. Pure. */
export function neverServed(rows, configured) {
const out = [];
const models = [...new Set((configured ?? []).filter(Boolean).map(String))].sort();
for (const model of models) {
const tiers = tiersForModel(rows, model);
if ((tiers[FLEX] ?? 0) > 0) continue;
if (Object.values(tiers).reduce((n, v) => n + v, 0) <= 0) continue;
out.push([model, 0, tiers]);
}
return out;
}
/** Thousands separators. Pure. */
export function fmt(value) {
return Math.round(num(value)).toLocaleString('en-US');
}
/** Classify one model. Pure. Returns [state, detail]. */
export function verdict(model, flexHours, gaps, tiers, configured) {
const byTier = tiers ?? {};
const flexTotal = byTier[FLEX] ?? 0;
const otherTotal = Object.entries(byTier)
.filter(([t]) => t !== FLEX).reduce((n, [, v]) => n + v, 0);
if (flexTotal <= 0 && (configured ?? []).includes(model)) {
if (otherTotal <= 0) return ['no-usage', 'no requests on any tier in this window'];
return ['flex-never-served',
`${fmt(flexTotal)} flex request(s) in this window, and ${fmt(otherTotal)} `
+ 'on other tiers. The service_tier parameter is not reaching the API.'];
}
if (flexTotal <= 0) return ['no-flex-usage', 'never served on flex in this window'];
if ((gaps ?? []).length) {
return ['flex-shortfall',
`${gaps.length} hour(s) at or below half the median served hour `
+ `(median ${fmt(gaps[0][3])} requests)`];
}
const served = Object.values(flexHours ?? {}).filter((v) => v > 0).length;
if (served < MIN_SERVED_HOURS) {
return ['too-little-history',
`${served} hour(s) of flex traffic, which is not enough to take a median from`];
}
return ['flex-served',
`${fmt(flexTotal)} flex request(s) across ${served} hour(s), no collapsed hours`];
}
/** An hour bucket's start as a readable UTC string. Pure. */
export function stamp(hour) {
return `${new Date(Number(hour) * 1000).toISOString().slice(0, 13)}:00Z`;
}
/** The repair for one verdict. Pure. Printed, never performed. */
export function repairLines(state) {
if (state === 'flex-never-served') {
return ['the tier in this report is the tier that was served. Check for a '
+ 'gateway that rewrites request bodies, an SDK wrapper with its own '
+ 'defaults, or a code path that never set service_tier at all.',
'until it arrives you are paying standard rates for a workload you believe '
+ 'is discounted, and nothing will raise about it.'];
}
if (state === 'flex-shortfall') {
return ['back off and retry on 429 Resource Unavailable, which means no '
+ 'capacity right now rather than a limit you exceeded. Retrying it '
+ 'genuinely helps, unlike the billing 429s.',
'raise the client timeout to at least 15 minutes. The official SDK default '
+ 'is 10 and flex responses regularly exceed it, and an aborted request can '
+ 'still be billed if the server finishes.',
'fall back to service_tier auto when completing the work matters more than '
+ 'the discount, and keep flex off anything a person is waiting for.'];
}
if (state === 'too-little-history') {
return ['not a clean bill of health, just too little to read. Re-run over a '
+ 'longer window once the job has more served hours behind it.'];
}
return [];
}
async function read(apiKey, path, params) {
const url = new URL(API + path);
for (const [k, v] of Object.entries(params ?? {})) {
for (const one of Array.isArray(v) ? v : [v]) url.searchParams.append(k, String(one));
}
const r = await fetch(url, { headers: { Authorization: `Bearer ${apiKey}`,
'User-Agent': 'openai-flex-tier-served/1.0' } });
if (r.status === 401 || r.status === 403) {
throw new Error(`${r.status} from OpenAI: the organization usage endpoints need `
+ 'an admin key');
}
if (!r.ok) throw new Error(`${r.status} ${await r.text()}`);
return r.json();
}
async function main() {
const apiKey = process.env.OPENAI_ADMIN_KEY;
if (!apiKey) {
console.error('set OPENAI_ADMIN_KEY to an admin key that can read the '
+ 'organization usage endpoints');
process.exitCode = 2;
return;
}
const days = Math.max(0.5, Math.min(7, Number(process.env.DAYS ?? 7)));
const floor = Number(process.env.FLOOR ?? 0.5);
const configured = (process.env.FLEX_MODELS ?? '').split(/[,\s]+/).filter(Boolean);
const start = Math.floor(Date.now() / 1000 - days * 86400);
const payloads = [];
const params = { start_time: start, bucket_width: '1h', limit: 168,
'group_by[]': ['service_tier', 'model'] };
for (let i = 0; i < 50; i += 1) {
const page = await read(apiKey, '/organization/usage/completions', params);
payloads.push(page);
if (!page.has_more || !page.next_page) break;
params.page = page.next_page;
}
const rows = tierRows(payloads);
const totals = totalsByTier(rows);
const active = hoursActive(rows);
const buckets = payloads.reduce((n, p) => n + (p.data ?? []).length, 0);
console.log(`${buckets} hourly bucket(s), ${Object.keys(totals).length} tier(s) `
+ `observed: ${Object.keys(totals).sort().join(', ') || 'none'}`);
if (configured.length === 0) {
console.log('no FLEX_MODELS given, so the never-served check is skipped: nothing '
+ 'the API returns knows which models your code asks for flex on');
}
let findings = 0;
const models = [...new Set([...Object.keys(rows).map((k) => k.split('\u0000')[0]),
...configured])].sort();
for (const model of models) {
const flexHours = flexByHour(rows, model);
const gaps = flexGaps(flexHours, active, floor);
const tiers = tiersForModel(rows, model);
const [state, detail] = verdict(model, flexHours, gaps, tiers, configured);
if (['no-flex-usage', 'no-usage'].includes(state) && !configured.includes(model)) {
continue;
}
console.log(`${state.padEnd(21)} ${model}: ${detail}`);
for (const [hour, flex, other] of gaps.slice(0, 5)) {
console.log(` ${stamp(hour)} ${fmt(flex)} requests, other tiers served `
+ `${fmt(other)} that hour`);
}
if (state === 'flex-shortfall') {
console.log(' note: a 429 Resource Unavailable is not charged and never reaches '
+ 'this report, so these hours are absence rather than error counts.');
}
for (const line of repairLines(state)) console.log(` repair: ${line}`);
if (FINDINGS.has(state)) findings += 1;
}
console.log(`${findings} finding(s)`);
process.exitCode = findings ? 1 : 0;
}
if (import.meta.url === `file://${process.argv[1]}`) await main();
Add a test
The first test builds a week where flex is served every hour except three, with other tiers busy throughout, and asserts those three come back as gaps with the median printed. The second is the control that keeps the note honest: the same three empty hours with no traffic on any tier are a quiet night and must produce nothing. Then the never-served case, which only fires for a model passed in as configured, because the API cannot know what your code asks for; the minimum history rule, which must decline rather than grade two data points; the median, which has to be a median and not a mean or one backfill hour hides everything; and the tier fold, which must keep hours with no row absent rather than inventing zeros.
from openai_flex_tier_served import (flex_by_hour, flex_gaps, hours_active,
median, never_served, stamp, repair_lines,
tier_rows, tiers_for_model, totals_by_tier,
verdict)
HOUR = 3600
BASE = 1787000000 // HOUR * HOUR
def result(model, tier, requests, out=0):
return {"object": "organization.usage.completions.result",
"input_tokens": requests * 800, "output_tokens": out,
"num_model_requests": requests, "project_id": None,
"model": model, "batch": False, "service_tier": tier}
def week(flex_per_hour, other_per_hour, hours=24, model="gpt-5.6"):
data = []
for i in range(hours):
results = []
flex = flex_per_hour(i)
other = other_per_hour(i)
if flex:
results.append(result(model, "flex", flex))
if other:
results.append(result(model, "default", other))
data.append({"object": "bucket", "start_time": BASE + i * HOUR,
"end_time": BASE + (i + 1) * HOUR, "results": results})
return [{"object": "page", "data": data, "has_more": False, "next_page": None}]
def test_hours_where_flex_collapsed_while_other_tiers_kept_serving():
# The note. Flex runs at about 2,000 an hour except for three hours where
# it is refused, and default keeps going throughout, which is what makes
# those three hours a capacity signal rather than a quiet night.
dead = {5, 11, 19}
pages = week(lambda i: 0 if i in dead else 2_000, lambda i: 8_000)
rows = tier_rows(pages)
gaps = flex_gaps(flex_by_hour(rows, "gpt-5.6"), hours_active(rows))
assert [g[0] for g in gaps] == [BASE + h * HOUR for h in sorted(dead)]
assert gaps[0][1] == 0.0 and gaps[0][2] == 8_000.0 and gaps[0][3] == 2_000.0
state, detail = verdict("gpt-5.6", flex_by_hour(rows, "gpt-5.6"), gaps,
tiers_for_model(rows, "gpt-5.6"), ["gpt-5.6"])
assert state == "flex-shortfall"
assert "3 hour(s)" in detail
lines = repair_lines(state)
assert any("Resource Unavailable" in line for line in lines)
assert any("15 minutes" in line for line in lines)
assert stamp(BASE).endswith(":00Z")
def test_a_quiet_night_is_not_a_capacity_failure():
# The control. Same three empty flex hours, but nothing else ran either.
dead = {5, 11, 19}
pages = week(lambda i: 0 if i in dead else 2_000,
lambda i: 0 if i in dead else 8_000)
rows = tier_rows(pages)
assert flex_gaps(flex_by_hour(rows, "gpt-5.6"), hours_active(rows)) == []
state, _ = verdict("gpt-5.6", flex_by_hour(rows, "gpt-5.6"), [],
tiers_for_model(rows, "gpt-5.6"), ["gpt-5.6"])
assert state == "flex-served"
def test_a_model_configured_for_flex_that_never_gets_it():
pages = week(lambda i: 0, lambda i: 1_717)
rows = tier_rows(pages)
tiers = tiers_for_model(rows, "gpt-5.6")
assert tiers == {"default": 41_208.0}
state, detail = verdict("gpt-5.6", {}, [], tiers, ["gpt-5.6"])
assert state == "flex-never-served"
assert "41,208 on other tiers" in detail
assert any("rewrites request bodies" in line for line in repair_lines(state))
# And the same model, not declared as configured, is nobody's business.
assert verdict("gpt-5.6", {}, [], tiers, [])[0] == "no-flex-usage"
assert never_served(rows, ["gpt-5.6"])[0][0] == "gpt-5.6"
assert never_served(rows, []) == []
assert never_served(rows, ["never-called-model"]) == []
def test_too_little_flex_history_declines_rather_than_grading():
pages = week(lambda i: 2_000 if i < 4 else 0, lambda i: 8_000)
rows = tier_rows(pages)
flex_hours = flex_by_hour(rows, "gpt-5.6")
assert flex_gaps(flex_hours, hours_active(rows)) == []
state, detail = verdict("gpt-5.6", flex_hours, [],
tiers_for_model(rows, "gpt-5.6"), ["gpt-5.6"])
assert state == "too-little-history"
assert "not enough to take a median" in detail
assert any("not a clean bill of health" in line for line in repair_lines(state))
def test_the_median_is_a_median_and_not_a_mean():
# One enormous backfill hour. The mean of this is over 12,000, which would
# put every ordinary hour under half of it and report the whole week.
values = [2_000, 2_000, 2_000, 2_000, 2_000, 100_000]
assert median(values) == 2_000.0
assert sum(values) / len(values) > 12_000
assert median([]) == 0.0
assert median([5]) == 5.0
assert median([1, 3]) == 2.0
def test_the_fold_keeps_absent_hours_absent():
pages = week(lambda i: 0 if i % 2 else 100, lambda i: 50)
rows = tier_rows(pages)
flex_hours = flex_by_hour(rows, "gpt-5.6")
# Twelve flex hours out of twenty four, and the other twelve are missing
# rather than present with a zero.
assert len(flex_hours) == 12
assert all(v == 100.0 for v in flex_hours.values())
assert len(hours_active(rows)) == 24
assert totals_by_tier(rows) == {"flex": 1_200.0, "default": 1_200.0}
assert tier_rows(None) == {} and totals_by_tier(None) == {}
assert hours_active(None) == {} and flex_by_hour(None, "x") == {}
assert verdict("x", {}, [], {}, [])[0] == "no-flex-usage"
assert repair_lines("flex-served") == []
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { flexByHour, flexGaps, hoursActive, median, neverServed, repairLines,
stamp, tierRows, tiersForModel, totalsByTier, verdict }
from './openai-flex-tier-served.mjs';
const HOUR = 3600;
const BASE = Math.floor(1787000000 / HOUR) * HOUR;
const result = (model, tier, requests) => ({
object: 'organization.usage.completions.result',
input_tokens: requests * 800, output_tokens: 0,
num_model_requests: requests, project_id: null,
model, batch: false, service_tier: tier,
});
const week = (flexPerHour, otherPerHour, hours = 24, model = 'gpt-5.6') => {
const data = [];
for (let i = 0; i < hours; i += 1) {
const results = [];
const flex = flexPerHour(i);
const other = otherPerHour(i);
if (flex) results.push(result(model, 'flex', flex));
if (other) results.push(result(model, 'default', other));
data.push({ object: 'bucket', start_time: BASE + i * HOUR,
end_time: BASE + (i + 1) * HOUR, results });
}
return [{ object: 'page', data, has_more: false, next_page: null }];
};
test('hours where flex collapsed while other tiers kept serving', () => {
const dead = new Set([5, 11, 19]);
const rows = tierRows(week((i) => (dead.has(i) ? 0 : 2000), () => 8000));
const gaps = flexGaps(flexByHour(rows, 'gpt-5.6'), hoursActive(rows));
assert.deepEqual(gaps.map((g) => g[0]),
[...dead].sort((a, b) => a - b).map((h) => BASE + h * HOUR));
assert.equal(gaps[0][1], 0);
assert.equal(gaps[0][2], 8000);
assert.equal(gaps[0][3], 2000);
const [state, detail] = verdict('gpt-5.6', flexByHour(rows, 'gpt-5.6'), gaps,
tiersForModel(rows, 'gpt-5.6'), ['gpt-5.6']);
assert.equal(state, 'flex-shortfall');
assert.match(detail, /3 hour\(s\)/);
const lines = repairLines(state);
assert.ok(lines.some((l) => l.includes('Resource Unavailable')));
assert.ok(lines.some((l) => l.includes('15 minutes')));
assert.ok(stamp(BASE).endsWith(':00Z'));
});
test('a quiet night is not a capacity failure', () => {
const dead = new Set([5, 11, 19]);
const rows = tierRows(week((i) => (dead.has(i) ? 0 : 2000),
(i) => (dead.has(i) ? 0 : 8000)));
assert.deepEqual(flexGaps(flexByHour(rows, 'gpt-5.6'), hoursActive(rows)), []);
const [state] = verdict('gpt-5.6', flexByHour(rows, 'gpt-5.6'), [],
tiersForModel(rows, 'gpt-5.6'), ['gpt-5.6']);
assert.equal(state, 'flex-served');
});
test('a model configured for flex that never gets it', () => {
const rows = tierRows(week(() => 0, () => 1717));
const tiers = tiersForModel(rows, 'gpt-5.6');
assert.deepEqual(tiers, { default: 41208 });
const [state, detail] = verdict('gpt-5.6', {}, [], tiers, ['gpt-5.6']);
assert.equal(state, 'flex-never-served');
assert.match(detail, /41,208 on other tiers/);
assert.ok(repairLines(state).some((l) => l.includes('rewrites request bodies')));
assert.equal(verdict('gpt-5.6', {}, [], tiers, [])[0], 'no-flex-usage');
assert.equal(neverServed(rows, ['gpt-5.6'])[0][0], 'gpt-5.6');
assert.deepEqual(neverServed(rows, []), []);
assert.deepEqual(neverServed(rows, ['never-called-model']), []);
});
test('too little flex history declines rather than grading', () => {
const rows = tierRows(week((i) => (i < 4 ? 2000 : 0), () => 8000));
const flexHours = flexByHour(rows, 'gpt-5.6');
assert.deepEqual(flexGaps(flexHours, hoursActive(rows)), []);
const [state, detail] = verdict('gpt-5.6', flexHours, [],
tiersForModel(rows, 'gpt-5.6'), ['gpt-5.6']);
assert.equal(state, 'too-little-history');
assert.match(detail, /not enough to take a median/);
assert.ok(repairLines(state).some((l) => l.includes('not a clean bill of health')));
});
test('the median is a median and not a mean', () => {
const values = [2000, 2000, 2000, 2000, 2000, 100000];
assert.equal(median(values), 2000);
assert.ok(values.reduce((a, b) => a + b, 0) / values.length > 12000);
assert.equal(median([]), 0);
assert.equal(median([5]), 5);
assert.equal(median([1, 3]), 2);
});
test('the fold keeps absent hours absent', () => {
const rows = tierRows(week((i) => (i % 2 ? 0 : 100), () => 50));
const flexHours = flexByHour(rows, 'gpt-5.6');
assert.equal(Object.keys(flexHours).length, 12);
assert.ok(Object.values(flexHours).every((v) => v === 100));
assert.equal(Object.keys(hoursActive(rows)).length, 24);
assert.deepEqual(totalsByTier(rows), { flex: 1200, default: 1200 });
assert.deepEqual(tierRows(null), {});
assert.deepEqual(totalsByTier(null), {});
assert.deepEqual(hoursActive(null), {});
assert.deepEqual(flexByHour(null, 'x'), {});
assert.equal(verdict('x', {}, [], {}, [])[0], 'no-flex-usage');
assert.deepEqual(repairLines('flex-served'), []);
});
FAQ
Why not retry a flex request and see whether it gets served?
Because that is a generation. Every script in this section holds a key that can spend money on inference and none of them send one, and a note about capacity has no business creating traffic against the tier it is complaining about. The reading here is entirely a GET of the usage report. The consequence is that the script cannot tell you how many 429 Resource Unavailable responses you got, only that flex volume collapsed in hours when the rest of the organization was busy.
Why not read the cost report? Flex is cheaper, so the spend should show it.
It should, and the cost report cannot show it. Its group_by supports project_id, line_item and api_key_id and has no service tier dimension at all, so flex spend and standard spend land in the same line item for the same model with no way to separate them. That is why this note reads one endpoint rather than reconciling two: the usage report is the only place the tiers are distinguished, and the money has to be worked out from the request and token counts it returns.
Does Anthropic have a flex tier this script should be checking too?
Not in any form you can ask for. Its service tier documentation describes Priority, Standard and Batch, and the service_tier request parameter on the Messages API accepts auto and standard_only. Its usage report does enumerate flex and flex_discount among the values its service_tiers filter accepts, which is worth knowing if you are writing a report parser, but there is no documented way to request that tier, so there is nothing for a script to detect and this note stays on OpenAI.
How is this different from the priority tier note?
That one is about a capacity commitment on Anthropic that a particular model may have no coverage for, and its finding is that a tier never appears for a model at all. This one is about a tier that appears, works, and then intermittently is not served, which is a different shape in a different provider's report: hours rather than models, and a collapse relative to a median rather than an absence relative to a purchase. They also fail in opposite directions financially. Priority is capacity you have paid for and may not be using; flex is a discount you may not be getting.
What about the timeouts in the title? Can the script see those?
No, and it says so rather than implying otherwise. Client configuration lives in your source tree and neither API returns anything that describes it. What the script prints is the floor: raise the client timeout to at least fifteen minutes, because the official SDK default is ten and flex responses regularly exceed it. The part worth internalising is that a client-side abort does not necessarily cancel the work, so an aborted request can still be billed for an answer nobody collected, which is the opposite of the unbilled 429 and lands in the same report.
Related field notes
- The other tier you pay for and may not be getting
- The other ten minute clock that kills a request mid flight
- Whether the wait instruction reaches your client at all
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Flex processing -- OpenAI platform docs
- Usage and costs -- OpenAI API reference
- openai-openapi -- the published OpenAPI specification
- Service tiers -- Claude platform docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.