Diagnostic LLM APIs
A service account key that has never been rotated
This one is the reward for having done it right. Somebody read the guidance, created service accounts, moved production onto them and deleted the personal keys, and every audit since has come back clean because every audit since has been looking for personal keys. The service account was created in February two years ago. Its key was minted the same afternoon. There has never been a second key, so there has never been a moment when two keys were valid at once, so rotation has never been a deploy with a rollback: it has always been a hard cutover with an outage on the other side of it. Which is why it keeps being scheduled for next quarter.
With an organization admin key, per project: GET /v1/organization/projects/{project_id}/service_accounts?limit=100 for the roster, then GET /v1/organization/projects/{project_id}/api_keys?limit=100&owner_project_access=any filtered to owner.type == "service_account". Group the keys by owner.service_account.id and take the newest created_at in each group. Anything past 180 days has not been rotated in six months.
Then read how many keys that account has, because the count changes the finding. One stale key is worse than two. With a single key there is no overlap window: swapping it means the old value stops working the instant the new one starts, which is a hard cutover with no rollback, and that is the actual reason the rotation keeps getting deferred rather than laziness.
Corroborate with GET /v1/organization/audit_logs?event_types[]=api_key.created&effective_at[gte]={now-180d}&limit=100. No api_key.created event for a project in the window means nothing has been minted there in six months. Read that carefully: the event carries project.id and actor and does not name a service account, so an absence is a fact about the project and never about one account. And if audit logging is not enabled for your organization, the endpoint's silence is not evidence of anything at all.
The problem in plain words
Service accounts fix ownership and nothing else. They detach the credential from a person's employment, which is the whole reason to use them, and in doing so they remove the last event that ever forced anybody to think about the key again. A personal key at least has an owner who eventually changes teams. A service account has nobody, no expiry, and no prompt.
So the key ages, and nothing about it degrades. It authenticates as well on day 900 as on day one. There is no error, no warning, no field that turns amber, and no read-only endpoint anywhere that reports when a credential was last rotated, because no such field exists on either provider. The only evidence available is created_at, and created_at looks identical on a key minted last month as part of a disciplined quarterly rotation and on a key minted last month because the service is new.
The single-key case is the one that explains the whole pattern. Rotation is safe when two keys can be valid at once: mint the second, deploy it, watch the first go quiet, remove it. A service account with exactly one key has never been in that shape, so every rotation attempt is a synchronised cutover across every deployment that holds the value. That is a change with a real chance of an outage and no obvious rollback, which is why it is always the thing being planned rather than the thing being done.
Why it happens
This clock is the one the other key notes ignore, and it points at the healthiest credential in the organization. The idleness note finds keys with no traffic. This one finds the opposite: a key used every second of every day, owned exactly as the guidance says it should be, and two years old. Nothing in a usage-based sweep can surface it, because on every usage measure it is the best-behaved key you have.
Key count is part of the verdict, not a detail in the output. One stale key means no overlap window has ever existed and the rotation is a cutover. Two keys where both are stale means overlap is possible and simply was not used. One fresh key beside one stale one is a rotation that started and never finished, which is the state that most deserves a nudge because somebody already did the risky part and left the old credential live. Three different findings, three different sentences, one created_at field.
The audit log confirms an absence at the project level and nothing finer. The api_key.created event carries id, data.scopes, effective_at, project.id and actor. It does not carry the service account the key belongs to. So "no creation events in this project for 180 days" is a sound conclusion about the project, and "therefore this particular service account was not rotated" is an inference the script declines to print as a fact. The per-account claim rests on created_at in the key list, which is where it belongs.
An unreachable or empty audit log is not a clean one. Audit logging is gated to organizations that have it enabled, so an empty response can mean nothing happened or can mean nothing is being recorded. The script reports audit-unavailable and explicitly downgrades its own confidence rather than upgrading a silence into corroboration. It also backs off on a 429, because this is the one administration endpoint that documents its own rate limit with a Retry-After.
Nothing here self-corrects, and one endpoint shows what it would look like if it did. Project API keys have no expires_at at all. Admin keys do, on GET /v1/organization/admin_api_keys, which is worth reading precisely because it demonstrates the field that would make this note unnecessary. Where you cannot set an expiry, the schedule has to live outside the platform, and a scheduled run of this script is the cheapest version of that.
The fix, as a flow
This chain is the reward for having done it right, which is why nothing in it fails. Service accounts fix ownership and in doing so remove the last event that would ever have forced anybody to think about the key again. There is no rotated_at field on either provider, so created_at on the newest key is the only clock there is.
How to fix it
Use an admin key with read scopes
Projects, service accounts, project keys and audit logs all live under /v1/organization/*, which rejects project keys. An sk-admin- key provisioned read-only reads all four.
Build the service-account roster per project
GET /v1/organization/projects/{project_id}/service_accounts?limit=100 returns id, name, role and created_at. Keep it even when a service account turns out to have no keys at all: that is its own small finding, and it usually means a half-finished migration.
Group the keys by owner.service_account.id and take the newest
GET /v1/organization/projects/{project_id}/api_keys?limit=100&owner_project_access=any, filtered to owner.type == "service_account". The newest created_at in each group is the rotation clock. Using the oldest instead reports a service account that rotated last week as stale, because the key it replaced is still sitting there.
Count the keys, because the count is the finding
One key past the threshold is a cutover with no rollback. Two stale keys mean the overlap was available and unused. One fresh and one stale is a rotation that stopped halfway. Print the right sentence for each rather than one generic age warning.
Ask the audit log, and grade its answer honestly
GET /v1/organization/audit_logs?event_types[]=api_key.created&effective_at[gte]={now-180d}&limit=100, paginating on after. Zero events for a project corroborates at the project level. An empty log across the whole organization means audit logging may not be enabled, and the script says audit-unavailable rather than treating silence as agreement.
How to check it worked
Re-run after a rotation. The account you rotated should show two keys with the newest one days old, then a single fresh key once the old one is revoked.
python3 openai_key_rotation_clock.py --stale-after 180
# 6 project(s), 11 service account(s), 14 service-account key(s)
# audit log: 3 api_key.created event(s) in 180 day(s) across 2 project(s)
# single-stale-key proj_prod ingest-worker newest key 731 day(s) old, and it is the only one
# repair: mint a second key first. One key means every rotation is a hard cutover with no rollback.
# stale-key proj_prod billing-sync newest key 402 day(s) old across 2 key(s)
# unfinished-rotation proj_api search-indexer newest key 12 day(s) old, oldest 588 day(s) and still live
# repair: the new key is deployed. Confirm the old one has gone quiet, then revoke it.
# corroboration: proj_prod has no api_key.created events in 180 day(s) (project level, not per account)
# 3 finding(s)
The full code
Four GETs and one clock. newest_and_oldest is where the note is won or lost: taking the newest created_at per account is what stops a freshly-rotated account being reported as stale because its retired key is still in the list. rotation_verdict takes the key count as a first-class input rather than a footnote, so a single stale key and two stale keys come back as different findings. corroboration is the honest one: it distinguishes a project with no creation events from an organization whose audit log is empty or unreachable, and it labels its own conclusion project-level because the api_key.created event does not name a service account. rotation_plan prints the overlap sequence with the revocation last.
"""Find service account keys that have never been rotated.
Read only. Four GETs against the OpenAI Administration API with an admin key:
projects, service accounts, project keys and the audit log. Nothing is created,
changed or removed, and no key value is printed.
The clock is created_at on the newest key belonging to each service account.
There is no rotated_at field anywhere on either provider, so age since minting
is the only evidence available, and the key count decides which of three
findings it is.
The audit log corroborates an absence at the PROJECT level only: the
api_key.created event carries project.id and actor and does not name a service
account. An empty or unreachable audit log is reported as unavailable rather
than as agreement, because audit logging is gated to organizations that have it
enabled and silence from it means nothing either way.
"""
import argparse
import datetime as dt
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("openai_key_rotation_clock")
API = "https://api.openai.com/v1"
SINGLE_STALE = "single-stale-key"
STALE = "stale-key"
UNFINISHED = "unfinished-rotation"
NO_KEYS = "service-account-with-no-keys"
ROTATING = "rotating"
TOO_NEW = "too-new"
FINDINGS = (SINGLE_STALE, STALE, UNFINISHED, NO_KEYS)
AUDIT_CONFIRMED = "confirmed-at-project-level"
AUDIT_ACTIVITY = "creation-activity-in-window"
AUDIT_UNAVAILABLE = "audit-unavailable"
def age_days(stamp, now):
"""Whole days between a unix timestamp and now. Pure. None when unreadable."""
if stamp is None or stamp == "" or isinstance(stamp, bool):
return None
try:
when = dt.datetime.fromtimestamp(float(stamp), dt.timezone.utc)
except (TypeError, ValueError, OSError, OverflowError):
return None
return int((now - when).total_seconds() // 86400)
def service_account_id(key):
"""The service account a key belongs to, or None. Pure."""
owner = (key or {}).get("owner")
if not isinstance(owner, dict):
return None
if str(owner.get("type") or "").strip().lower() != "service_account":
return None
account = owner.get("service_account")
if not isinstance(account, dict):
return None
return str(account.get("id") or "") or None
def group_by_account(keys):
"""Group service-account keys by owner.service_account.id. Pure.
Keys owned by a user are dropped here rather than counted as unattributed,
because a personal key in a project is a finding for a different note and
counting it towards a service account's key total would make a single-key
account look like it has an overlap window it does not have.
"""
out = {}
for key in keys or []:
account = service_account_id(key)
if account:
out.setdefault(account, []).append(key)
return out
def newest_and_oldest(keys, now):
"""(newest_age, oldest_age) in days across a key group. Pure.
The newest is the rotation clock. Using the oldest instead reports a
service account that rotated last week as stale, because the key it
replaced is still in the list until somebody revokes it.
"""
ages = [a for a in (age_days((k or {}).get("created_at"), now) for k in keys or [])
if a is not None]
if not ages:
return (None, None)
return (min(ages), max(ages))
def rotation_verdict(account, keys, now, stale_after=180, min_age=30):
"""Classify one service account's rotation state. Pure. (state, detail).
The key count is an input rather than a detail, because one stale key and
two stale keys are different problems. With one key there has never been a
moment when two credentials were valid at once, so every rotation is a
synchronised cutover with no rollback, and that is the reason it keeps
being deferred.
"""
name = str((account or {}).get("name") or (account or {}).get("id") or "(unnamed)")
rows = list(keys or [])
if not rows:
created = age_days((account or {}).get("created_at"), now)
return (NO_KEYS,
"service account %s has no keys at all%s"
% (name, "" if created is None else
", and was created %d day(s) ago" % created))
newest, oldest = newest_and_oldest(rows, now)
if newest is None:
return (TOO_NEW, "no readable created_at on any of its %d key(s)" % len(rows))
if newest < min_age and len(rows) == 1:
return (TOO_NEW, "its only key is %d day(s) old" % newest)
if newest < stale_after:
if oldest >= stale_after and len(rows) > 1:
return (UNFINISHED,
"newest key %d day(s) old, oldest %d day(s) and still live"
% (newest, oldest))
return (ROTATING, "newest key %d day(s) old" % newest)
if len(rows) == 1:
return (SINGLE_STALE,
"newest key %d day(s) old, and it is the only one" % newest)
return (STALE,
"newest key %d day(s) old across %d key(s)" % (newest, len(rows)))
def corroboration(events, project_id, audit_reachable=True, days=180):
"""What the audit log can and cannot confirm. Pure. (state, detail).
Three outcomes and only one of them is corroboration. The api_key.created
event names a project and an actor and never a service account, so the
strongest available statement is about the project. An empty or unreachable
log is reported as unavailable, because audit logging is gated and its
silence is not evidence.
"""
if not audit_reachable:
return (AUDIT_UNAVAILABLE,
"the audit log could not be read, so nothing here is "
"corroborated. Audit logging is gated to organizations that "
"have it enabled and its silence is not evidence.")
rows = list(events or [])
if not rows:
return (AUDIT_UNAVAILABLE,
"the audit log returned no events of any kind in %d day(s), "
"which can mean nothing was minted or can mean nothing is "
"being recorded. Treated as unavailable rather than clean."
% days)
here = [e for e in rows
if str(((e or {}).get("project") or {}).get("id") or "") == str(project_id)]
if here:
return (AUDIT_ACTIVITY,
"%d api_key.created event(s) in this project in %d day(s), so "
"something was minted here. The event does not name a service "
"account, so it neither confirms nor clears any one of them."
% (len(here), days))
return (AUDIT_CONFIRMED,
"no api_key.created events in this project in %d day(s). That is a "
"project-level fact: the event carries project.id and actor and "
"not the service account, so the per-account age above remains the "
"evidence for any single account." % days)
def rotation_plan(project_id, account_name, single_key):
"""The overlap rotation, printed and never performed. Pure."""
steps = []
if single_key:
steps.append("mint a second key first. One key means every rotation is "
"a hard cutover with no rollback, which is the actual "
"reason this has not happened yet.")
steps.extend([
"mint the replacement with an admin POST to /v1/organization/projects/"
"%s/service_accounts/{service_account_id}/api_keys for %s. The value "
"is returned exactly once." % (project_id, account_name),
"deploy the new value everywhere the old one is held, then watch the "
"old key: its last_used_at should stop advancing within one traffic "
"cycle. Do not skip this; it is the only rollback you get.",
"revoke the old key with a DELETE on /v1/organization/projects/%s/"
"api_keys/{api_key_id}, and diary the next rotation at 90 days. "
"Project keys have no expires_at, so nothing will remind you."
% project_id,
])
return steps
def get(session, path, params):
r = session.get(API + path, params=params, timeout=60)
if r.status_code in (401, 403):
raise SystemExit("%d from OpenAI: /v1/organization/* needs an admin "
"key (sk-admin-), not a project key" % r.status_code)
r.raise_for_status()
return r.json()
def paged(session, path, params, limit_pages=20):
"""Walk an administration listing on has_more / last_id."""
params = dict(params)
for _ in range(limit_pages):
page = get(session, path, params)
yield page
if not page.get("has_more") or not page.get("last_id"):
return
params["after"] = page["last_id"]
def collect(session, path, params):
rows = []
for page in paged(session, path, params):
rows.extend(page.get("data") or [])
return rows
def read_audit_log(session, days):
"""Read api_key.created events, tolerating an organization without them.
Returns (events, reachable). A 4xx here is not fatal: audit logging is not
enabled everywhere, and a script that dies on it would report nothing about
the key ages it already has in hand.
"""
since = int((dt.datetime.now(dt.timezone.utc)
- dt.timedelta(days=days)).timestamp())
try:
return (collect(session, "/organization/audit_logs",
{"limit": 100, "event_types[]": ["api_key.created"],
"effective_at[gte]": since}), True)
except requests.HTTPError as err:
status = getattr(getattr(err, "response", None), "status_code", None)
log.warning("audit log unreadable (%s): rotation ages below stand on "
"created_at alone", status)
return ([], False)
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--stale-after", type=int, default=180,
help="days since the newest key was minted (default 180)")
ap.add_argument("--min-age", type=int, default=30,
help="days before a new service account is graded at all")
args = ap.parse_args()
admin = os.environ.get("OPENAI_ADMIN_KEY")
if not admin:
log.error("set OPENAI_ADMIN_KEY to an admin key (sk-admin-) with read "
"scopes; a project key cannot read /v1/organization/*")
return 2
s = requests.Session()
s.headers.update({"Authorization": "Bearer " + admin})
now = dt.datetime.now(dt.timezone.utc)
events, reachable = read_audit_log(s, args.stale_after)
if reachable:
projects_seen = {str(((e or {}).get("project") or {}).get("id") or "")
for e in events}
log.info("audit log: %d api_key.created event(s) in %d day(s) across "
"%d project(s)", len(events), args.stale_after,
len([p for p in projects_seen if p]))
projects = collect(s, "/organization/projects",
{"limit": 100, "include_archived": "true"})
accounts_seen = 0
keys_seen = 0
findings = 0
for project in projects:
pid = project.get("id")
if not pid:
continue
name = project.get("name") or pid
accounts = collect(s, "/organization/projects/%s/service_accounts" % pid,
{"limit": 100})
keys = collect(s, "/organization/projects/%s/api_keys" % pid,
{"limit": 100, "owner_project_access": "any"})
grouped = group_by_account(keys)
accounts_seen += len(accounts)
keys_seen += sum(len(v) for v in grouped.values())
project_findings = 0
for account in accounts:
rows = grouped.get(str(account.get("id") or ""), [])
state, detail = rotation_verdict(account, rows, now,
args.stale_after, args.min_age)
if state not in FINDINGS:
continue
findings += 1
project_findings += 1
log.warning("%-19s %-11s %-15s %s", state, name,
account.get("name") or account.get("id") or "(unnamed)",
detail)
if state in (SINGLE_STALE, STALE, UNFINISHED):
for step in rotation_plan(pid,
account.get("name") or "(unnamed)",
state == SINGLE_STALE):
log.warning(" repair: %s", step)
if project_findings:
audit_state, audit_detail = corroboration(events, pid, reachable,
args.stale_after)
log.info("corroboration for %s: %s: %s", name, audit_state,
audit_detail)
log.info("%d project(s), %d service account(s), %d service-account key(s)",
len(projects), accounts_seen, keys_seen)
log.info("%d finding(s)", findings)
log.info("there is no rotated_at field on either provider: created_at on "
"the newest key is the only clock available")
return 1 if findings else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Find service account keys that have never been rotated.
*
* Read only. Four GETs against the OpenAI Administration API with an admin
* key. Nothing is created, changed or removed, and no key value is printed.
*
* The clock is created_at on the newest key belonging to each service account,
* because no rotated_at field exists on either provider. The key count decides
* which of three findings it is.
*
* The audit log corroborates an absence at the PROJECT level only: the
* api_key.created event carries project.id and actor and does not name a
* service account. An empty or unreachable log is reported as unavailable.
*/
const API = 'https://api.openai.com/v1';
export const SINGLE_STALE = 'single-stale-key';
export const STALE = 'stale-key';
export const UNFINISHED = 'unfinished-rotation';
export const NO_KEYS = 'service-account-with-no-keys';
export const ROTATING = 'rotating';
export const TOO_NEW = 'too-new';
const FINDINGS = new Set([SINGLE_STALE, STALE, UNFINISHED, NO_KEYS]);
export const AUDIT_CONFIRMED = 'confirmed-at-project-level';
export const AUDIT_ACTIVITY = 'creation-activity-in-window';
export const AUDIT_UNAVAILABLE = 'audit-unavailable';
/** Whole days between a unix timestamp and now. Pure. null when unreadable. */
export function ageDays(stamp, now) {
if (stamp === null || stamp === undefined || stamp === '' ||
typeof stamp === 'boolean') return null;
const seconds = Number(stamp);
if (!Number.isFinite(seconds)) return null;
return Math.floor((now.getTime() - seconds * 1000) / 86400000);
}
/** The service account a key belongs to, or null. Pure. */
export function serviceAccountId(key) {
const owner = (key ?? {}).owner;
if (!owner || typeof owner !== 'object') return null;
if (String(owner.type ?? '').trim().toLowerCase() !== 'service_account') return null;
const account = owner.service_account;
if (!account || typeof account !== 'object') return null;
return String(account.id ?? '') || null;
}
/** Group service-account keys by owner.service_account.id. Pure. */
export function groupByAccount(keys) {
const out = {};
for (const key of keys ?? []) {
const account = serviceAccountId(key);
if (!account) continue;
out[account] = out[account] ?? [];
out[account].push(key);
}
return out;
}
/** [newestAge, oldestAge] in days across a key group. Pure. */
export function newestAndOldest(keys, now) {
const ages = (keys ?? [])
.map((k) => ageDays((k ?? {}).created_at, now))
.filter((a) => a !== null);
if (!ages.length) return [null, null];
return [Math.min(...ages), Math.max(...ages)];
}
/** Classify one service account's rotation state. Pure. [state, detail]. */
export function rotationVerdict(account, keys, now, staleAfter = 180, minAge = 30) {
const name = String((account ?? {}).name ?? (account ?? {}).id ?? '(unnamed)');
const rows = [...(keys ?? [])];
if (!rows.length) {
const created = ageDays((account ?? {}).created_at, now);
return [NO_KEYS,
`service account ${name} has no keys at all` +
(created === null ? '' : `, and was created ${created} day(s) ago`)];
}
const [newest, oldest] = newestAndOldest(rows, now);
if (newest === null) {
return [TOO_NEW, `no readable created_at on any of its ${rows.length} key(s)`];
}
if (newest < minAge && rows.length === 1) {
return [TOO_NEW, `its only key is ${newest} day(s) old`];
}
if (newest < staleAfter) {
if (oldest >= staleAfter && rows.length > 1) {
return [UNFINISHED,
`newest key ${newest} day(s) old, oldest ${oldest} day(s) and still live`];
}
return [ROTATING, `newest key ${newest} day(s) old`];
}
if (rows.length === 1) {
return [SINGLE_STALE, `newest key ${newest} day(s) old, and it is the only one`];
}
return [STALE, `newest key ${newest} day(s) old across ${rows.length} key(s)`];
}
/** What the audit log can and cannot confirm. Pure. [state, detail]. */
export function corroboration(events, projectId, auditReachable = true, days = 180) {
if (!auditReachable) {
return [AUDIT_UNAVAILABLE,
'the audit log could not be read, so nothing here is corroborated. ' +
'Audit logging is gated to organizations that have it enabled and its ' +
'silence is not evidence.'];
}
const rows = [...(events ?? [])];
if (!rows.length) {
return [AUDIT_UNAVAILABLE,
`the audit log returned no events of any kind in ${days} day(s), which ` +
'can mean nothing was minted or can mean nothing is being recorded. ' +
'Treated as unavailable rather than clean.'];
}
const here = rows.filter(
(e) => String((e?.project ?? {}).id ?? '') === String(projectId));
if (here.length) {
return [AUDIT_ACTIVITY,
`${here.length} api_key.created event(s) in this project in ${days} ` +
'day(s), so something was minted here. The event does not name a ' +
'service account, so it neither confirms nor clears any one of them.'];
}
return [AUDIT_CONFIRMED,
`no api_key.created events in this project in ${days} day(s). That is a ` +
'project-level fact: the event carries project.id and actor and not the ' +
'service account, so the per-account age above remains the evidence for ' +
'any single account.'];
}
/** The overlap rotation, printed and never performed. Pure. */
export function rotationPlan(projectId, accountName, singleKey) {
const steps = [];
if (singleKey) {
steps.push('mint a second key first. One key means every rotation is a ' +
'hard cutover with no rollback, which is the actual reason ' +
'this has not happened yet.');
}
steps.push(
'mint the replacement with an admin POST to /v1/organization/projects/' +
`${projectId}/service_accounts/{service_account_id}/api_keys for ` +
`${accountName}. The value is returned exactly once.`,
'deploy the new value everywhere the old one is held, then watch the old ' +
'key: its last_used_at should stop advancing within one traffic cycle. ' +
'Do not skip this; it is the only rollback you get.',
`revoke the old key with a DELETE on /v1/organization/projects/${projectId}` +
'/api_keys/{api_key_id}, and diary the next rotation at 90 days. Project ' +
'keys have no expires_at, so nothing will remind you.');
return steps;
}
async function getJson(key, path, params) {
const url = new URL(API + path);
for (const [k, v] of params) url.searchParams.append(k, String(v));
const res = await fetch(url, { headers: { Authorization: `Bearer ${key}` } });
if (res.status === 401 || res.status === 403) {
throw new Error(`${res.status} from OpenAI: /v1/organization/* needs an ` +
'admin key (sk-admin-), not a project key');
}
if (!res.ok) {
const err = new Error(`${res.status} from ${path}`);
err.status = res.status;
throw err;
}
return res.json();
}
async function collect(key, path, params) {
const rows = [];
let after = null;
for (let page = 0; page < 20; page += 1) {
const query = after ? [...params, ['after', after]] : params;
const body = await getJson(key, path, query);
rows.push(...(body.data ?? []));
if (!body.has_more || !body.last_id) return rows;
after = body.last_id;
}
return rows;
}
async function main() {
const admin = process.env.OPENAI_ADMIN_KEY;
if (!admin) {
console.error('set OPENAI_ADMIN_KEY to an admin key (sk-admin-) with read ' +
'scopes; a project key cannot read /v1/organization/*');
process.exitCode = 2;
return;
}
const staleAfter = Number(process.env.STALE_AFTER ?? 180);
const minAge = Number(process.env.MIN_AGE ?? 30);
const now = new Date();
const since = Math.floor(now.getTime() / 1000) - staleAfter * 86400;
let events = [];
let reachable = true;
try {
events = await collect(admin, '/organization/audit_logs', [
['limit', 100], ['event_types[]', 'api_key.created'],
['effective_at[gte]', since]]);
console.log(`audit log: ${events.length} api_key.created event(s) in ` +
`${staleAfter} day(s)`);
} catch (err) {
reachable = false;
console.warn(`audit log unreadable (${err.status ?? err.message}): ` +
'rotation ages below stand on created_at alone');
}
const projects = await collect(admin, '/organization/projects',
[['limit', 100], ['include_archived', 'true']]);
let accountsSeen = 0;
let keysSeen = 0;
let findings = 0;
for (const project of projects) {
if (!project.id) continue;
const name = project.name ?? project.id;
const accounts = await collect(
admin, `/organization/projects/${project.id}/service_accounts`, [['limit', 100]]);
const keys = await collect(
admin, `/organization/projects/${project.id}/api_keys`,
[['limit', 100], ['owner_project_access', 'any']]);
const grouped = groupByAccount(keys);
accountsSeen += accounts.length;
keysSeen += Object.values(grouped).reduce((n, v) => n + v.length, 0);
let projectFindings = 0;
for (const account of accounts) {
const rows = grouped[String(account.id ?? '')] ?? [];
const [state, detail] = rotationVerdict(account, rows, now, staleAfter, minAge);
if (!FINDINGS.has(state)) continue;
findings += 1;
projectFindings += 1;
console.warn(`${state.padEnd(19)} ${String(name).padEnd(11)} ` +
`${String(account.name ?? account.id ?? '(unnamed)').padEnd(15)} ${detail}`);
if (state === SINGLE_STALE || state === STALE || state === UNFINISHED) {
for (const step of rotationPlan(project.id, account.name ?? '(unnamed)',
state === SINGLE_STALE)) {
console.warn(` repair: ${step}`);
}
}
}
if (projectFindings) {
const [auditState, auditDetail] =
corroboration(events, project.id, reachable, staleAfter);
console.log(`corroboration for ${name}: ${auditState}: ${auditDetail}`);
}
}
console.log(`${projects.length} project(s), ${accountsSeen} service account(s), ` +
`${keysSeen} service-account key(s)`);
console.log(`${findings} finding(s)`);
console.log('there is no rotated_at field on either provider: created_at on ' +
'the newest key is the only clock available');
process.exitCode = findings ? 1 : 0;
}
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The first test is the mistake that would make this note useless: an account that rotated last week still has its retired key in the list, so a reader that takes the oldest created_at reports it as two years stale. Reading the newest has to return unfinished-rotation, and once the retired key is revoked, nothing at all. The second is the count rule, three ways: one stale key, two stale keys and one fresh key beside one stale one are three findings with three different sentences, and only the first is told to mint a second key before touching anything. Then the honest half. corroboration has to return audit-unavailable for an empty log and for an unreachable one, and it must never claim that a project-level absence clears a specific service account. Finally: a key owned by a person must not be counted towards a service account's key total, or a single-key account looks like it has an overlap window it has never had.
import datetime as dt
from openai_key_rotation_clock import (age_days, corroboration,
group_by_account, newest_and_oldest,
rotation_plan, rotation_verdict,
service_account_id)
NOW = dt.datetime(2026, 8, 31, 12, 0, 0, tzinfo=dt.timezone.utc)
ACCOUNT = {"id": "svc_1", "name": "ingest-worker", "created_at": 0}
def unix(days_ago):
return int((NOW - dt.timedelta(days=days_ago)).timestamp())
def sa_key(key_id, days_ago, account="svc_1"):
return {"id": key_id, "created_at": unix(days_ago),
"owner": {"type": "service_account",
"service_account": {"id": account, "name": "ingest-worker"}}}
def test_the_newest_key_is_the_clock_and_the_oldest_would_lie():
# An account rotated last week still holds the retired key until somebody
# revokes it. A reader that takes the oldest created_at calls that account
# two years stale, which is the single mistake that would make this note
# useless. Reading the newest calls it a rotation that has not been
# finished, which is what it is.
rotated = [sa_key("key_new", 45), sa_key("key_old", 731)]
newest, oldest = newest_and_oldest(rotated, NOW)
assert (newest, oldest) == (45, 731)
state, detail = rotation_verdict(ACCOUNT, rotated, NOW)
assert state != "single-stale-key" and state != "stale-key"
assert state == "unfinished-rotation"
assert "newest key 45 day(s) old" in detail
# And once the retired key is revoked, there is nothing left to report.
finished = rotation_verdict(ACCOUNT, [sa_key("key_new", 45)], NOW)
assert finished[0] == "rotating"
assert "45 day(s) old" in finished[1]
def test_the_key_count_produces_three_different_findings():
single = rotation_verdict(ACCOUNT, [sa_key("key_a", 731)], NOW)
assert single[0] == "single-stale-key"
assert "it is the only one" in single[1]
assert any("mint a second key first" in step
for step in rotation_plan("proj_1", "ingest-worker", True))
both_old = rotation_verdict(ACCOUNT, [sa_key("key_a", 402),
sa_key("key_b", 500)], NOW)
assert both_old[0] == "stale-key"
assert "across 2 key(s)" in both_old[1]
assert not any("mint a second key first" in step
for step in rotation_plan("proj_1", "ingest-worker", False))
halfway = rotation_verdict(ACCOUNT, [sa_key("key_a", 12),
sa_key("key_b", 588)], NOW)
assert halfway[0] == "unfinished-rotation"
assert "still live" in halfway[1]
def test_an_empty_or_unreachable_audit_log_is_never_corroboration():
unreachable = corroboration([], "proj_1", audit_reachable=False)
assert unreachable[0] == "audit-unavailable"
assert "silence is not evidence" in unreachable[1]
empty = corroboration([], "proj_1", audit_reachable=True)
assert empty[0] == "audit-unavailable"
assert "nothing is being recorded" in empty[1]
def test_the_audit_log_confirms_a_project_and_never_an_account():
elsewhere = [{"type": "api_key.created", "project": {"id": "proj_other"}}]
state, detail = corroboration(elsewhere, "proj_1", True, 180)
assert state == "confirmed-at-project-level"
assert "project-level fact" in detail
assert "does not name" in detail or "not the service account" in detail
here = [{"type": "api_key.created", "project": {"id": "proj_1"}},
{"type": "api_key.created", "project": {"id": "proj_1"}}]
state, detail = corroboration(here, "proj_1", True, 180)
assert state == "creation-activity-in-window"
assert "neither confirms nor clears" in detail
def test_a_personal_key_is_not_counted_towards_a_service_account():
keys = [sa_key("key_a", 731),
{"id": "key_user", "created_at": unix(2),
"owner": {"type": "user", "user": {"email": "dev@example.test"}}},
{"id": "key_odd", "created_at": unix(2), "owner": None}]
grouped = group_by_account(keys)
assert list(grouped) == ["svc_1"]
assert len(grouped["svc_1"]) == 1
# Counting the personal key here would turn a single-key account into a
# two-key one and hide the fact that no overlap window has ever existed.
assert rotation_verdict(ACCOUNT, grouped["svc_1"], NOW)[0] == "single-stale-key"
assert service_account_id({"owner": {"type": "service_account"}}) is None
assert service_account_id(None) is None
assert group_by_account([]) == {}
def test_a_service_account_with_no_keys_and_one_too_new_to_judge():
empty = rotation_verdict({"id": "svc_2", "name": "search-indexer",
"created_at": unix(300)}, [], NOW)
assert empty[0] == "service-account-with-no-keys"
assert "300 day(s) ago" in empty[1]
fresh = rotation_verdict(ACCOUNT, [sa_key("key_a", 4)], NOW)
assert fresh[0] == "too-new"
unreadable = rotation_verdict(ACCOUNT, [{"id": "key_a", "created_at": None,
"owner": None}], NOW)
assert unreadable[0] == "too-new"
def test_ages_are_read_from_unix_seconds_only():
assert age_days(unix(180), NOW) == 180
assert age_days(None, NOW) is None
assert age_days("", NOW) is None
assert age_days(True, NOW) is None
assert age_days("not a number", NOW) is None
def test_the_rotation_plan_revokes_last_and_names_the_missing_field():
steps = rotation_plan("proj_prod", "ingest-worker", False)
assert len(steps) == 3
assert "returned exactly once" in steps[0]
assert "last_used_at should stop advancing" in steps[1]
assert steps[2].startswith("revoke the old key")
assert "no expires_at" in steps[2]
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { ageDays, corroboration, groupByAccount, newestAndOldest, rotationPlan,
rotationVerdict, serviceAccountId }
from './openai-key-rotation-clock.mjs';
const NOW = new Date('2026-08-31T12:00:00Z');
const ACCOUNT = { id: 'svc_1', name: 'ingest-worker', created_at: 0 };
const unix = (daysAgo) => Math.floor(NOW.getTime() / 1000) - daysAgo * 86400;
const saKey = (id, daysAgo, account = 'svc_1') => ({
id, created_at: unix(daysAgo),
owner: { type: 'service_account',
service_account: { id: account, name: 'ingest-worker' } },
});
test('the newest key is the clock and the oldest would lie', () => {
const rotated = [saKey('key_new', 45), saKey('key_old', 731)];
assert.deepEqual(newestAndOldest(rotated, NOW), [45, 731]);
const [state, detail] = rotationVerdict(ACCOUNT, rotated, NOW);
assert.ok(state !== 'single-stale-key' && state !== 'stale-key');
assert.equal(state, 'unfinished-rotation');
assert.match(detail, /newest key 45 day\(s\) old/);
const finished = rotationVerdict(ACCOUNT, [saKey('key_new', 45)], NOW);
assert.equal(finished[0], 'rotating');
});
test('the key count produces three different findings', () => {
const single = rotationVerdict(ACCOUNT, [saKey('key_a', 731)], NOW);
assert.equal(single[0], 'single-stale-key');
assert.match(single[1], /it is the only one/);
assert.ok(rotationPlan('proj_1', 'ingest-worker', true)
.some((s) => s.includes('mint a second key first')));
const bothOld = rotationVerdict(ACCOUNT, [saKey('key_a', 402), saKey('key_b', 500)], NOW);
assert.equal(bothOld[0], 'stale-key');
assert.match(bothOld[1], /across 2 key\(s\)/);
assert.ok(!rotationPlan('proj_1', 'ingest-worker', false)
.some((s) => s.includes('mint a second key first')));
const halfway = rotationVerdict(ACCOUNT, [saKey('key_a', 12), saKey('key_b', 588)], NOW);
assert.equal(halfway[0], 'unfinished-rotation');
assert.match(halfway[1], /still live/);
});
test('an empty or unreachable audit log is never corroboration', () => {
const unreachable = corroboration([], 'proj_1', false);
assert.equal(unreachable[0], 'audit-unavailable');
assert.match(unreachable[1], /silence is not evidence/);
const empty = corroboration([], 'proj_1', true);
assert.equal(empty[0], 'audit-unavailable');
assert.match(empty[1], /nothing is being recorded/);
});
test('the audit log confirms a project and never an account', () => {
const [state, detail] = corroboration(
[{ type: 'api_key.created', project: { id: 'proj_other' } }], 'proj_1', true, 180);
assert.equal(state, 'confirmed-at-project-level');
assert.match(detail, /project-level fact/);
assert.match(detail, /not the service account/);
const [state2, detail2] = corroboration([
{ type: 'api_key.created', project: { id: 'proj_1' } },
{ type: 'api_key.created', project: { id: 'proj_1' } }], 'proj_1', true, 180);
assert.equal(state2, 'creation-activity-in-window');
assert.match(detail2, /neither confirms nor clears/);
});
test('a personal key is not counted towards a service account', () => {
const grouped = groupByAccount([
saKey('key_a', 731),
{ id: 'key_user', created_at: unix(2),
owner: { type: 'user', user: { email: 'dev@example.test' } } },
{ id: 'key_odd', created_at: unix(2), owner: null },
]);
assert.deepEqual(Object.keys(grouped), ['svc_1']);
assert.equal(grouped.svc_1.length, 1);
assert.equal(rotationVerdict(ACCOUNT, grouped.svc_1, NOW)[0], 'single-stale-key');
assert.equal(serviceAccountId({ owner: { type: 'service_account' } }), null);
assert.equal(serviceAccountId(null), null);
assert.deepEqual(groupByAccount([]), {});
});
test('a service account with no keys and one too new to judge', () => {
const empty = rotationVerdict(
{ id: 'svc_2', name: 'search-indexer', created_at: unix(300) }, [], NOW);
assert.equal(empty[0], 'service-account-with-no-keys');
assert.match(empty[1], /300 day\(s\) ago/);
assert.equal(rotationVerdict(ACCOUNT, [saKey('key_a', 4)], NOW)[0], 'too-new');
assert.equal(rotationVerdict(
ACCOUNT, [{ id: 'key_a', created_at: null, owner: null }], NOW)[0], 'too-new');
});
test('ages are read from unix seconds only', () => {
assert.equal(ageDays(unix(180), NOW), 180);
assert.equal(ageDays(null, NOW), null);
assert.equal(ageDays('', NOW), null);
assert.equal(ageDays(true, NOW), null);
assert.equal(ageDays('not a number', NOW), null);
});
test('the rotation plan revokes last and names the missing field', () => {
const steps = rotationPlan('proj_prod', 'ingest-worker', false);
assert.equal(steps.length, 3);
assert.match(steps[0], /returned exactly once/);
assert.match(steps[1], /last_used_at should stop advancing/);
assert.ok(steps[2].startsWith('revoke the old key'));
assert.match(steps[2], /no expires_at/);
});
FAQ
How is an unrotated key different from an unused one?
They are opposite readings of the same roster. The idleness note looks for credentials with no traffic behind them, and its finding is a key nothing depends on. This one looks at creation dates, and its subject is the key everything depends on: used constantly, owned by a service account exactly as the guidance says, and minted two years ago. On every usage measure it is the best-behaved credential in the organization, which is precisely why no traffic-based sweep will ever mention it.
Why does having only one key make it worse rather than simpler?
Because rotation is only safe when two credentials can be valid at once. The safe sequence is mint the second, deploy it, watch the first go quiet, then revoke. A service account with exactly one key has never been in that shape, so any rotation is a synchronised cutover across every deployment holding the value, with no rollback if something is missed. That is a genuinely risky change, and it is the real reason the rotation keeps moving to next quarter. The script tells single-key accounts to mint the second key first and stop there.
Can the audit log tell me a specific service account was never rotated?
No, and the script refuses to imply otherwise. The api_key.created event carries id, data.scopes, effective_at, project.id and actor. It does not carry the service account the key belongs to. So the strongest sound conclusion from an absence is that nothing was minted in that project during the window, which is reported as a project-level fact. The claim about any single account rests on created_at in the key list, which is the field that actually knows.
The audit log came back empty. Is that good news?
It is no news. Audit logging is gated to organizations that have it enabled, so an empty response can mean nothing was minted or can mean nothing is being recorded, and those are not the same. The script reports audit-unavailable in both cases and explicitly says the key ages stand on created_at alone. It also treats a 4xx from that endpoint as non-fatal, because dying there would throw away the rotation ages it already has in hand.
Is there any field that would make this check unnecessary?
An expiry, and one endpoint has it. Admin API keys carry an optional expires_at, so an admin key can be made to lapse on a schedule of its own. Project API keys have no such field at all, and neither provider records anything like a rotated_at. So for the credentials that actually run production, the schedule has to live outside the platform, and a scheduled run of this script is the cheapest version of that until an expiry exists to set.
Related field notes
- The same roster read for traffic instead of ownership
- The events behind every change to that roster, and who made them
- The key whose owner lost access, which is a fact about a person rather than about traffic
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Projects — OpenAI API reference
- Project API keys — OpenAI API reference
- Audit logs — OpenAI API reference
- Administration APIs — OpenAI developer docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.