Diagnostic LLM APIs
Almost everyone in the organization holds the owner role
Nobody decided this. Someone needed to create a project in week three and the fastest unblock was to make them an owner; someone needed to raise a rate limit in month five and the same thing happened; the contractor needed a key for a fortnight in March. Every one of those was the right call at the time and none of them was ever undone, because demotion is a visible act with a social cost and nothing in the platform ever asks. Two years later the roster has fourteen names on it and thirteen can change the billing settings.
GET /v1/organization/users?limit=100 with an organization admin key, paginated on after. Each organization.user carries role ("owner" or "reader"), is_service_account, is_scim_managed, added_at and api_key_last_used_at. Compute the owner share with service accounts removed, because they are returned in the same list and are frequently owners by construction; counting them produces a confident finding about robots.
The role model is the reason this happens. There are two org roles and reader is genuinely restrictive, so the first time anyone needs to do anything at all — create a project, mint a key, change a limit — the only move available is promotion to owner. An owner can create and archive projects, provision admin keys, change rate limits, alter billing, invite further owners, and remove other members.
Then go one level down. GET /v1/organization/projects/{project_id}/users?limit=100 per project returns a per-project role of "member" or "owner". An organization that has already granted project owner to everyone will not change behaviour when you demote at the org level, so the two readings belong in one report.
And read GET /v1/organization/admin_api_keys, which returns each key's owner. That is the list of people whose demotion actually breaks something, and it is much shorter than the roster.
The problem in plain words
The coarseness is the mechanism. With two roles and no middle, "can do their job" and "can change the billing settings" are the same grant, so every unblock in the organization's history is recorded as a promotion. There is no forcing function in the other direction: nothing expires, nothing reviews, and demoting a colleague is a small act of distrust that nobody wants to perform on a Tuesday.
What the organization loses is not really security in the dramatic sense. It is accountability. When fourteen people can change a rate limit, a rate limit that changed has no owner, and the question "who did this and why" has fourteen candidate answers and no shorter list. The same is true of a project that was archived, a spend limit that was raised, and an admin key that appeared.
The sharper edge is that an owner can provision an admin API key, and an admin key reads the whole organization's usage, costs, users, projects and audit log. So the owner count is not a count of people who can administer the platform through a console session; it is a count of people who can mint a long-lived credential that does the same thing from anywhere.
Why it happens
Service accounts are in the same list, and including them is the most common way to get this wrong. GET /v1/organization/users returns service accounts alongside people, marked with is_service_account, and a service account created to run a job is often an owner because that is what it needed to be. Counting them inflates the ratio, sometimes past the threshold on its own, and produces a report recommending that you demote a cron job. The script filters first and says how many it removed.
A null api_key_last_used_at is not evidence of an unused privilege. It means this principal has not authenticated an API request, which is exactly what you would expect of somebody who administers the platform through the console and never writes code. The field is genuinely useful — an owner with no API usage is a candidate for demotion — but the script reports it as a question rather than a verdict, because the console leaves no trace in this field and treating silence as absence would recommend demoting the person who actually runs your billing.
SCIM-managed members cannot be fixed here, and changing them through the API is worse than doing nothing. is_scim_managed means the membership and role are projected from your identity provider. A role changed through the API will be reverted at the next sync, leaving an audit-log entry that says somebody tried and a roster that is unchanged. Those names need a different ticket, pointed at a different system, and the script separates them for exactly that reason.
The org roster is half the picture, because project roles are granted separately. Reading only /v1/organization/users can show a tidy organization whose every project grants owner to every member, which is the same problem one level down and is invisible from the top. The script tallies per-project roles too, and the repair it prints for a real demotion is to grant the person a project role instead, so they keep the access they actually use.
The fix, as a flow
Two roles, no middle, and one direction of travel. Every unblock in the history of the organization is recorded as a promotion, and nothing anywhere asks for a demotion, so the distinction erodes quietly until a configuration change has fourteen candidate authors. The reading has to remove service accounts first, because they are on the same roster and are frequently owners by design.
How to fix it
Use an organization admin key, provisioned read-only
All four reads are under /v1/organization/*, which rejects a project key. Read scopes are sufficient; nothing here changes a role, and role changes should not be automated anyway.
List the roster and remove the service accounts
GET /v1/organization/users?limit=100, paginated on after. Filter on is_service_account before computing anything. The endpoint also accepts an emails[] filter if you want to check a specific person rather than audit the whole list.
Compute the owner share, and hold the floor
Two owners in a three-person company is not a governance finding, it is a company. The script refuses to grade an organization below a small member floor, because a ratio computed over four people is a statement about the founders rather than about access control.
Separate the SCIM-managed names
is_scim_managed members are projected from your identity provider. Their roles are not repairable through this API, so they go in their own list with their own instruction: change the group mapping in the IdP, because a change made here reverts at the next sync.
Read the admin keys and the project roles, then print the demotion list
GET /v1/organization/admin_api_keys names the owners who hold a long-lived org-wide credential; those are the demotions with consequences. GET /v1/organization/projects/{project_id}/users shows whether the same over-granting has already happened one level down. The repair is reader at the org level plus a project role where the person actually works — printed, never performed.
How to check it worked
Re-run after a demotion round. The ratio moves, and the more useful number is the second one: the count of owners with no API usage should fall to the people who genuinely administer billing and keys. If an owner reappears after a sync, that name was SCIM-managed and the change belonged in the identity provider.
python3 openai_owner_ratio_audit.py --days 180
# 14 member(s), 3 service account(s) excluded, 2 SCIM-managed
# owner-majority 11 of 14 human member(s) hold the owner role (79%)
# a***@example.com owner added 2024-03-11 no API key use on record
# m***@example.com owner added 2024-06-02 last key use 214 day(s) ago
# p***@example.com owner added 2025-01-19 holds 1 admin API key
# project roles: 3 of 4 project(s) also grant owner to every member
# repair: demote to reader anyone who does not administer billing, keys or projects
# repair: grant a project role instead, so people keep the access they use
# repair: 2 owner(s) are SCIM-managed; change the group mapping, not the API
# repair: 1 owner holds an admin API key; revoke the key before the role
# 1 finding(s)
The full code
Three paged GETs plus one per project, and eight pure functions. The service-account filter, which runs before any arithmetic; the role tally, which files an unrecognised role under other rather than assuming it is harmless; the ratio; the email mask, so the output is safe to paste into a channel; the unused-privilege test, which is deliberately reported as a question; the admin-key owner index, which reads names and ids and never a key value; the project-level tally; and the verdict, which holds a member floor because a ratio over four people is not a finding about access control.
"""Find an OpenAI organization where the owner role is the default.
Read only. Paged GETs against /v1/organization/users, /admin_api_keys,
/projects and each project's /users, with an organization admin key. Every
request is a GET and no request body is constructed.
No key value is read or printed. The admin key listing is used for its `owner`
block only, and email addresses are masked, because this report is a list of
named colleagues.
"""
import argparse
import logging
import os
import sys
import time
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("openai_owner_ratio_audit")
API = "https://api.openai.com/v1"
DAY = 86400
OWNER = "owner"
READER = "reader"
OTHER = "other"
FINDINGS = ("everyone-is-owner", "owner-majority", "owner-count-high")
def humans(users):
"""The roster with service accounts removed. Pure.
Service accounts are returned by this endpoint alongside people and are
frequently owners because that is what the job needed. Counting them
inflates the ratio and produces a report that recommends demoting a cron.
"""
return [u for u in (users or []) if not (u or {}).get("is_service_account")]
def role_of(user):
"""Normalise one member's org role. Pure.
An unrecognised role is filed under "other" rather than folded into reader:
a future role this script has never heard of should show up in the output
as unknown, not be silently counted as restricted.
"""
raw = str((user or {}).get("role") or "").strip().lower()
return raw if raw in (OWNER, READER) else OTHER
def role_counts(people):
"""{role: count} over a roster. Pure."""
counts = {OWNER: 0, READER: 0, OTHER: 0}
for person in people or []:
counts[role_of(person)] += 1
return counts
def owner_ratio(counts):
"""Owners as a share of the roster. Pure. 0.0 when the roster is empty."""
data = counts or {}
total = sum(int(data.get(r) or 0) for r in (OWNER, READER, OTHER))
if total <= 0:
return 0.0
return int(data.get(OWNER) or 0) / float(total)
def mask(email):
"""Hide the local part of an email address. Pure. Non-emails pass through.
Every row of this report names a colleague and their privileges. Masking by
default costs nothing and makes the output safe to paste into a channel.
"""
text = str(email or "").strip()
if "@" not in text:
return text or "unknown"
local, _, domain = text.partition("@")
if not local:
return text
return local[0] + "***@" + domain
def unused_privilege(user, now, days=180):
"""Has this member authenticated an API request recently? Pure.
Reported as a question, never as a verdict. A null api_key_last_used_at
means no API request, which is exactly what an administrator who works
through the console looks like.
"""
stamp = (user or {}).get("api_key_last_used_at")
if not stamp:
return (True, "no API key use on record")
try:
age = (int(now) - int(stamp)) // DAY
except (TypeError, ValueError):
return (True, "unreadable api_key_last_used_at")
if age >= days:
return (True, "last key use %d day(s) ago" % age)
return (False, "last key use %d day(s) ago" % age)
def admin_key_owners(keys):
"""{owner_id: owner_name} from the admin key listing. Pure.
Reads the owner block and nothing else. The key value is not returned by
this endpoint and is not wanted.
"""
out = {}
for key in keys or []:
owner = (key or {}).get("owner") or {}
oid = owner.get("id") or (owner.get("user") or {}).get("id")
if not oid:
continue
name = owner.get("name") or (owner.get("user") or {}).get("email") or "unnamed"
out[str(oid)] = str(name)
return out
def project_owner_share(members):
"""(owners, total, ratio) for one project's member list. Pure."""
rows = [m for m in (members or []) if not (m or {}).get("is_service_account")]
owners = sum(1 for m in rows
if str((m or {}).get("role") or "").strip().lower() == OWNER)
total = len(rows)
return (owners, total, (owners / float(total)) if total else 0.0)
def verdict(counts, min_members=3, ratio_max=0.50, count_max=5):
"""Classify the roster. Pure. Returns (state, detail).
The member floor comes first. Two owners in a three-person company is a
company, not a governance finding, and grading it produces a report that
nobody can act on and everybody learns to ignore.
"""
data = counts or {}
owners = int(data.get(OWNER) or 0)
total = sum(int(data.get(r) or 0) for r in (OWNER, READER, OTHER))
ratio = owner_ratio(data)
if total < min_members:
return ("too-few-members",
"%d human member(s) in the organization, too few for a role "
"distribution to mean anything" % total)
if ratio >= 0.90 and owners >= 3:
return ("everyone-is-owner",
"%d of %d human member(s) hold the owner role (%.0f%%). The "
"distinction between owner and reader has stopped existing here."
% (owners, total, ratio * 100))
if ratio > ratio_max:
return ("owner-majority",
"%d of %d human member(s) hold the owner role (%.0f%%)"
% (owners, total, ratio * 100))
if owners > count_max:
return ("owner-count-high",
"%d of %d human member(s) hold the owner role. The share is "
"fine and the absolute count is past the %d this audit treats "
"as a working ceiling, which is a convention rather than a "
"platform rule." % (owners, total, count_max))
return ("scoped",
"%d of %d human member(s) hold the owner role (%.0f%%)"
% (owners, total, ratio * 100))
def repair_lines(state, scim_owners=0, key_holders=0, loose_projects=0):
"""The repair for one roster verdict. Pure. Printed, never performed."""
if state not in FINDINGS:
return []
lines = [
"demote to reader anyone who does not administer billing, keys or "
"projects, with POST /v1/organization/users/{user_id} and role reader.",
"grant a project role instead, so people keep the access they actually "
"use: POST /v1/organization/projects/{project_id}/users with member.",
]
if scim_owners:
lines.append("%d owner(s) are SCIM-managed. Change the group mapping in "
"the identity provider; a role changed through this API is "
"reverted at the next sync." % scim_owners)
if key_holders:
lines.append("%d owner(s) hold an admin API key. Revoke the key before "
"the role, or the credential outlives the demotion."
% key_holders)
if loose_projects:
lines.append("%d project(s) also grant owner to every member, so an "
"org-level demotion alone will not change what anybody can "
"do there." % loose_projects)
return lines
def get(session, path, **params):
r = session.get(API + path, params=params, timeout=60)
if r.status_code in (401, 403):
raise SystemExit("%d from OpenAI: /v1/organization/* needs an "
"organization admin key, not a project key"
% r.status_code)
r.raise_for_status()
return r.json()
def paged(session, path, **params):
"""Walk an after/last_id cursor listing."""
params = dict(params)
while True:
page = get(session, path, **params)
data = page.get("data") or []
for item in data:
yield item
if not page.get("has_more") or not data:
return
params["after"] = page.get("last_id") or (data[-1] or {}).get("id")
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--days", type=int, default=180,
help="days of API silence before privilege is questioned")
ap.add_argument("--ratio", type=float, default=0.50,
help="owner share above which the roster is flagged")
ap.add_argument("--max-owners", type=int, default=5,
help="absolute owner count treated as a working ceiling")
args = ap.parse_args()
admin = os.environ.get("OPENAI_ADMIN_KEY")
if not admin:
log.error("set OPENAI_ADMIN_KEY to an organization admin key; a "
"project key cannot read /v1/organization/*")
return 2
s = requests.Session()
s.headers.update({"Authorization": "Bearer " + admin})
now = int(time.time())
users = list(paged(s, "/organization/users", limit=100))
people = humans(users)
counts = role_counts(people)
owners = [p for p in people if role_of(p) == OWNER]
scim = [p for p in owners if p.get("is_scim_managed")]
holders = admin_key_owners(paged(s, "/organization/admin_api_keys", limit=100))
loose = 0
projects = [p for p in paged(s, "/organization/projects", limit=100)
if str(p.get("status") or "").lower() != "archived"]
for project in projects:
members = list(paged(s, "/organization/projects/%s/users" % project.get("id"),
limit=100))
got, total, ratio = project_owner_share(members)
if total and ratio >= 0.90:
loose += 1
log.info("%d member(s), %d service account(s) excluded, %d SCIM-managed",
len(people), len(users) - len(people), len(scim))
state, detail = verdict(counts, ratio_max=args.ratio, count_max=args.max_owners)
if state not in FINDINGS:
log.info("%-18s %s", state, detail)
return 0
log.warning("%-18s %s", state, detail)
for person in sorted(owners, key=lambda p: int(p.get("added_at") or 0)):
_, note = unused_privilege(person, now, args.days)
extra = " holds an admin API key" if str(person.get("id")) in holders else ""
log.warning(" %-24s owner added %s %s%s", mask(person.get("email")),
time.strftime("%Y-%m-%d",
time.gmtime(int(person.get("added_at") or 0))),
note, extra)
if loose:
log.warning(" project roles: %d of %d project(s) also grant owner to "
"every member", loose, len(projects))
key_holders = sum(1 for p in owners if str(p.get("id")) in holders)
for line in repair_lines(state, len(scim), key_holders, loose):
log.warning(" repair: %s", line)
log.info("1 finding(s)")
return 1
if __name__ == "__main__":
sys.exit(main())
/**
* Find an OpenAI organization where the owner role is the default.
*
* Read only. Paged GETs against /v1/organization/users, /admin_api_keys,
* /projects and each project's /users. No request body is constructed and no
* key value is read or printed; email addresses are masked.
*/
const API = 'https://api.openai.com/v1';
const DAY = 86400;
const OWNER = 'owner';
const READER = 'reader';
const OTHER = 'other';
const FINDINGS = new Set(['everyone-is-owner', 'owner-majority', 'owner-count-high']);
/** The roster with service accounts removed. Pure. */
export function humans(users) {
return (users ?? []).filter((u) => !(u ?? {}).is_service_account);
}
/** Normalise one member's org role. Pure. Unknown roles are "other". */
export function roleOf(user) {
const raw = String(user?.role ?? '').trim().toLowerCase();
return raw === OWNER || raw === READER ? raw : OTHER;
}
/** {role: count} over a roster. Pure. */
export function roleCounts(people) {
const counts = { [OWNER]: 0, [READER]: 0, [OTHER]: 0 };
for (const person of people ?? []) counts[roleOf(person)] += 1;
return counts;
}
/** Owners as a share of the roster. Pure. */
export function ownerRatio(counts) {
const data = counts ?? {};
const total = [OWNER, READER, OTHER].reduce((a, r) => a + (data[r] ?? 0), 0);
if (total <= 0) return 0;
return (data[OWNER] ?? 0) / total;
}
/** Hide the local part of an email address. Pure. Non-emails pass through. */
export function mask(email) {
const text = String(email ?? '').trim();
if (!text.includes('@')) return text || 'unknown';
const at = text.indexOf('@');
const local = text.slice(0, at);
if (!local) return text;
return `${local[0]}***${text.slice(at)}`;
}
/** Has this member authenticated an API request recently? Pure. A question, not a verdict. */
export function unusedPrivilege(user, now, days = 180) {
const stamp = user?.api_key_last_used_at;
if (!stamp) return [true, 'no API key use on record'];
const value = Number(stamp);
if (!Number.isFinite(value)) return [true, 'unreadable api_key_last_used_at'];
const age = Math.floor((Number(now) - value) / DAY);
return [age >= days, `last key use ${age} day(s) ago`];
}
/** {owner_id: owner_name} from the admin key listing. Pure. Owner block only. */
export function adminKeyOwners(keys) {
const out = {};
for (const key of keys ?? []) {
const owner = key?.owner ?? {};
const id = owner.id ?? owner.user?.id;
if (!id) continue;
out[String(id)] = String(owner.name ?? owner.user?.email ?? 'unnamed');
}
return out;
}
/** [owners, total, ratio] for one project's member list. Pure. */
export function projectOwnerShare(members) {
const rows = (members ?? []).filter((m) => !(m ?? {}).is_service_account);
const owners = rows.filter(
(m) => String(m?.role ?? '').trim().toLowerCase() === OWNER).length;
const total = rows.length;
return [owners, total, total ? owners / total : 0];
}
/** Classify the roster. Pure. Returns [state, detail]. The member floor comes first. */
export function verdict(counts, minMembers = 3, ratioMax = 0.50, countMax = 5) {
const data = counts ?? {};
const owners = data[OWNER] ?? 0;
const total = [OWNER, READER, OTHER].reduce((a, r) => a + (data[r] ?? 0), 0);
const ratio = ownerRatio(data);
if (total < minMembers) {
return ['too-few-members',
`${total} human member(s) in the organization, too few for a role `
+ 'distribution to mean anything'];
}
if (ratio >= 0.90 && owners >= 3) {
return ['everyone-is-owner',
`${owners} of ${total} human member(s) hold the owner role `
+ `(${(ratio * 100).toFixed(0)}%). The distinction between owner and `
+ 'reader has stopped existing here.'];
}
if (ratio > ratioMax) {
return ['owner-majority',
`${owners} of ${total} human member(s) hold the owner role `
+ `(${(ratio * 100).toFixed(0)}%)`];
}
if (owners > countMax) {
return ['owner-count-high',
`${owners} of ${total} human member(s) hold the owner role. The share `
+ `is fine and the absolute count is past the ${countMax} this audit `
+ 'treats as a working ceiling, which is a convention rather than a '
+ 'platform rule.'];
}
return ['scoped',
`${owners} of ${total} human member(s) hold the owner role `
+ `(${(ratio * 100).toFixed(0)}%)`];
}
/** The repair for one roster verdict. Pure. Printed, never performed. */
export function repairLines(state, scimOwners = 0, keyHolders = 0, looseProjects = 0) {
if (!FINDINGS.has(state)) return [];
const lines = [
'demote to reader anyone who does not administer billing, keys or projects, '
+ 'with POST /v1/organization/users/{user_id} and role reader.',
'grant a project role instead, so people keep the access they actually use: '
+ 'POST /v1/organization/projects/{project_id}/users with member.',
];
if (scimOwners) {
lines.push(`${scimOwners} owner(s) are SCIM-managed. Change the group mapping in `
+ 'the identity provider; a role changed through this API is reverted at the '
+ 'next sync.');
}
if (keyHolders) {
lines.push(`${keyHolders} owner(s) hold an admin API key. Revoke the key before `
+ 'the role, or the credential outlives the demotion.');
}
if (looseProjects) {
lines.push(`${looseProjects} project(s) also grant owner to every member, so an `
+ 'org-level demotion alone will not change what anybody can do there.');
}
return lines;
}
async function read(key, path, params) {
const url = new URL(API + path);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, String(v));
const r = await fetch(url, { headers: { Authorization: `Bearer ${key}` } });
if (r.status === 401 || r.status === 403) {
throw new Error(`${r.status} from OpenAI: /v1/organization/* needs an `
+ 'organization admin key, not a project key');
}
if (!r.ok) throw new Error(`${r.status} ${await r.text()}`);
return r.json();
}
async function paged(key, path, params) {
const out = [];
const q = { ...params };
for (;;) {
const page = await read(key, path, q);
const data = page.data ?? [];
out.push(...data);
if (!page.has_more || data.length === 0) return out;
q.after = page.last_id ?? data[data.length - 1]?.id;
}
}
async function main() {
const admin = process.env.OPENAI_ADMIN_KEY;
if (!admin) {
console.error('set OPENAI_ADMIN_KEY to an organization admin key; a project '
+ 'key cannot read /v1/organization/*');
process.exitCode = 2;
return;
}
const days = Number(process.env.DAYS ?? 180);
const now = Math.floor(Date.now() / 1000);
const users = await paged(admin, '/organization/users', { limit: 100 });
const people = humans(users);
const counts = roleCounts(people);
const owners = people.filter((p) => roleOf(p) === OWNER);
const scim = owners.filter((p) => p.is_scim_managed);
const holders = adminKeyOwners(
await paged(admin, '/organization/admin_api_keys', { limit: 100 }));
const projects = (await paged(admin, '/organization/projects', { limit: 100 }))
.filter((p) => String(p.status ?? '').toLowerCase() !== 'archived');
let loose = 0;
for (const project of projects) {
const members = await paged(admin, `/organization/projects/${project.id}/users`,
{ limit: 100 });
const [, total, ratio] = projectOwnerShare(members);
if (total && ratio >= 0.90) loose += 1;
}
console.log(`${people.length} member(s), ${users.length - people.length} service `
+ `account(s) excluded, ${scim.length} SCIM-managed`);
const [state, detail] = verdict(counts);
console.log(`${state.padEnd(18)} ${detail}`);
if (!FINDINGS.has(state)) return;
for (const person of [...owners].sort((a, b) =>
Number(a.added_at ?? 0) - Number(b.added_at ?? 0))) {
const [, note] = unusedPrivilege(person, now, days);
const extra = holders[String(person.id)] ? ' holds an admin API key' : '';
const added = new Date(Number(person.added_at ?? 0) * 1000)
.toISOString().slice(0, 10);
console.log(` ${mask(person.email).padEnd(24)} owner added ${added} ${note}${extra}`);
}
if (loose) {
console.log(` project roles: ${loose} of ${projects.length} project(s) also `
+ 'grant owner to every member');
}
const keyHolders = owners.filter((p) => holders[String(p.id)]).length;
for (const line of repairLines(state, scim.length, keyHolders, loose)) {
console.log(` repair: ${line}`);
}
process.exitCode = 1;
}
if (import.meta.url === `file://${process.argv[1]}`) await main();
Add a test
The first test is the one that decides whether this note is worth anything: a roster of ten where three are service accounts and all three are owners, so the raw list says eighty per cent and the roster of people says seventy-one, and only one of those numbers is about access control. Under it: the member floor, because two owners in a three-person company is a company; the absolute count that flags at a low share and says out loud that the ceiling is a convention; the null api_key_last_used_at that must be reported as a question rather than as an unused privilege; SCIM-managed owners, whose repair points at the identity provider because an API change reverts at the next sync; the admin-key index, which reads the owner block and never a key value; and the project level, where the same over-granting hides from a roster that looks fine.
from openai_owner_ratio_audit import (admin_key_owners, humans, mask,
owner_ratio, project_owner_share,
repair_lines, role_counts, role_of,
unused_privilege, verdict)
NOW = 1_780_000_000
def user(uid, email, role="reader", service=False, scim=False, last_used=NOW,
added=1_700_000_000):
return {"id": uid, "email": email, "role": role,
"is_service_account": service, "is_scim_managed": scim,
"api_key_last_used_at": last_used, "added_at": added}
ROSTER = [
user("u_1", "ada@example.com", "owner", last_used=None),
user("u_2", "mel@example.com", "owner", last_used=NOW - 214 * 86400),
user("u_3", "pat@example.com", "owner"),
user("u_4", "sam@example.com", "owner", scim=True),
user("u_5", "kim@example.com", "owner", scim=True),
user("u_6", "rob@example.com", "reader"),
user("u_7", "jo@example.com", "reader"),
user("sa_1", "ingest@svc", "owner", service=True),
user("sa_2", "batch@svc", "owner", service=True),
user("sa_3", "evals@svc", "owner", service=True),
]
def test_service_accounts_never_count_toward_the_owner_ratio():
# The trap in one assertion. The raw list is 8 owners of 10; the roster of
# people is 5 of 7. Only the second is a statement about access control,
# and the first recommends demoting a cron job.
assert round(owner_ratio(role_counts(ROSTER)), 2) == 0.80
people = humans(ROSTER)
assert len(people) == 7
counts = role_counts(people)
assert counts == {"owner": 5, "reader": 2, "other": 0}
state, detail = verdict(counts)
assert state == "owner-majority"
assert "5 of 7" in detail
def test_a_small_organization_is_never_graded():
two = [user("u_1", "a@x.com", "owner"), user("u_2", "b@x.com", "owner")]
state, detail = verdict(role_counts(humans(two)))
assert state == "too-few-members"
assert "too few" in detail
assert repair_lines(state) == []
def test_everyone_being_an_owner_is_its_own_state():
roster = [user("u_%d" % i, "p%d@x.com" % i, "owner") for i in range(6)]
state, detail = verdict(role_counts(humans(roster)))
assert state == "everyone-is-owner"
assert "stopped existing" in detail
def test_a_high_count_at_a_low_share_says_the_ceiling_is_a_convention():
roster = ([user("o_%d" % i, "o%d@x.com" % i, "owner") for i in range(6)]
+ [user("r_%d" % i, "r%d@x.com" % i, "reader") for i in range(34)])
state, detail = verdict(role_counts(humans(roster)))
assert state == "owner-count-high"
assert "convention rather than a platform rule" in detail
def test_no_recorded_key_use_is_a_question_and_not_a_verdict():
quiet, note = unused_privilege(ROSTER[0], NOW)
assert quiet is True and note == "no API key use on record"
old, note = unused_privilege(ROSTER[1], NOW)
assert old is True and "214 day(s) ago" in note
fresh, note = unused_privilege(ROSTER[2], NOW)
assert fresh is False
assert unused_privilege({"api_key_last_used_at": "yesterday"}, NOW)[0] is True
def test_scim_managed_owners_get_a_repair_pointed_somewhere_else():
owners = [p for p in humans(ROSTER) if role_of(p) == "owner"]
scim = [p for p in owners if p["is_scim_managed"]]
assert len(scim) == 2
lines = repair_lines("owner-majority", len(scim), 1, 3)
assert any("identity provider" in line and "reverted at the next sync" in line
for line in lines)
assert any("Revoke the key before" in line for line in lines)
assert any("org-level demotion alone" in line for line in lines)
def test_the_admin_key_index_reads_the_owner_block_and_nothing_else():
keys = [{"id": "key_admin_1", "name": "ci",
"owner": {"id": "u_3", "name": "Pat", "type": "user"}},
{"id": "key_admin_2", "owner": {"user": {"id": "u_9",
"email": "x@y.com"}}},
{"id": "key_admin_3", "owner": {}}]
index = admin_key_owners(keys)
assert index == {"u_3": "Pat", "u_9": "x@y.com"}
assert admin_key_owners(None) == {}
def test_project_roles_are_the_second_level():
members = [{"id": "u_1", "role": "owner"}, {"id": "u_2", "role": "owner"},
{"id": "u_3", "role": "owner"},
{"id": "sa_1", "role": "owner", "is_service_account": True}]
assert project_owner_share(members) == (3, 3, 1.0)
mixed = [{"id": "u_1", "role": "owner"}, {"id": "u_2", "role": "member"},
{"id": "u_3", "role": "member"}]
owners, total, ratio = project_owner_share(mixed)
assert (owners, total) == (1, 3) and round(ratio, 2) == 0.33
assert project_owner_share([]) == (0, 0, 0.0)
def test_unknown_roles_are_never_counted_as_restricted():
assert role_of({"role": "OWNER"}) == "owner"
assert role_of({"role": "reader"}) == "reader"
assert role_of({"role": "billing"}) == "other"
assert role_of({}) == "other"
assert owner_ratio({}) == 0.0
def test_emails_are_masked():
assert mask("ada@example.com") == "a***@example.com"
assert mask("service-account") == "service-account"
assert mask(None) == "unknown"
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { adminKeyOwners, humans, mask, ownerRatio, projectOwnerShare, repairLines,
roleCounts, roleOf, unusedPrivilege, verdict }
from './openai-owner-ratio-audit.mjs';
const NOW = 1780000000;
const user = (id, email, role = 'reader', {
service = false, scim = false, lastUsed = NOW, added = 1700000000 } = {}) =>
({ id, email, role, is_service_account: service, is_scim_managed: scim,
api_key_last_used_at: lastUsed, added_at: added });
const ROSTER = [
user('u_1', 'ada@example.com', 'owner', { lastUsed: null }),
user('u_2', 'mel@example.com', 'owner', { lastUsed: NOW - 214 * 86400 }),
user('u_3', 'pat@example.com', 'owner'),
user('u_4', 'sam@example.com', 'owner', { scim: true }),
user('u_5', 'kim@example.com', 'owner', { scim: true }),
user('u_6', 'rob@example.com', 'reader'),
user('u_7', 'jo@example.com', 'reader'),
user('sa_1', 'ingest@svc', 'owner', { service: true }),
user('sa_2', 'batch@svc', 'owner', { service: true }),
user('sa_3', 'evals@svc', 'owner', { service: true }),
];
test('service accounts never count toward the owner ratio', () => {
assert.equal(Math.round(ownerRatio(roleCounts(ROSTER)) * 100) / 100, 0.8);
const people = humans(ROSTER);
assert.equal(people.length, 7);
const counts = roleCounts(people);
assert.deepEqual(counts, { owner: 5, reader: 2, other: 0 });
const [state, detail] = verdict(counts);
assert.equal(state, 'owner-majority');
assert.match(detail, /5 of 7/);
});
test('a small organization is never graded', () => {
const two = [user('u_1', 'a@x.com', 'owner'), user('u_2', 'b@x.com', 'owner')];
const [state, detail] = verdict(roleCounts(humans(two)));
assert.equal(state, 'too-few-members');
assert.match(detail, /too few/);
assert.deepEqual(repairLines(state), []);
});
test('everyone being an owner is its own state', () => {
const roster = Array.from({ length: 6 },
(_, i) => user(`u_${i}`, `p${i}@x.com`, 'owner'));
const [state, detail] = verdict(roleCounts(humans(roster)));
assert.equal(state, 'everyone-is-owner');
assert.match(detail, /stopped existing/);
});
test('a high count at a low share says the ceiling is a convention', () => {
const roster = [
...Array.from({ length: 6 }, (_, i) => user(`o_${i}`, `o${i}@x.com`, 'owner')),
...Array.from({ length: 34 }, (_, i) => user(`r_${i}`, `r${i}@x.com`, 'reader')),
];
const [state, detail] = verdict(roleCounts(humans(roster)));
assert.equal(state, 'owner-count-high');
assert.match(detail, /convention rather than a platform rule/);
});
test('no recorded key use is a question and not a verdict', () => {
assert.deepEqual(unusedPrivilege(ROSTER[0], NOW), [true, 'no API key use on record']);
const [old, note] = unusedPrivilege(ROSTER[1], NOW);
assert.equal(old, true);
assert.match(note, /214 day\(s\) ago/);
assert.equal(unusedPrivilege(ROSTER[2], NOW)[0], false);
assert.equal(unusedPrivilege({ api_key_last_used_at: 'yesterday' }, NOW)[0], true);
});
test('scim-managed owners get a repair pointed somewhere else', () => {
const owners = humans(ROSTER).filter((p) => roleOf(p) === 'owner');
const scim = owners.filter((p) => p.is_scim_managed);
assert.equal(scim.length, 2);
const lines = repairLines('owner-majority', scim.length, 1, 3);
assert.ok(lines.some((l) => l.includes('identity provider')
&& l.includes('reverted at the next sync')));
assert.ok(lines.some((l) => l.includes('Revoke the key before')));
assert.ok(lines.some((l) => l.includes('org-level demotion alone')));
});
test('the admin key index reads the owner block and nothing else', () => {
const keys = [
{ id: 'key_admin_1', name: 'ci', owner: { id: 'u_3', name: 'Pat', type: 'user' } },
{ id: 'key_admin_2', owner: { user: { id: 'u_9', email: 'x@y.com' } } },
{ id: 'key_admin_3', owner: {} },
];
assert.deepEqual(adminKeyOwners(keys), { u_3: 'Pat', u_9: 'x@y.com' });
assert.deepEqual(adminKeyOwners(null), {});
});
test('project roles are the second level', () => {
const members = [{ id: 'u_1', role: 'owner' }, { id: 'u_2', role: 'owner' },
{ id: 'u_3', role: 'owner' },
{ id: 'sa_1', role: 'owner', is_service_account: true }];
assert.deepEqual(projectOwnerShare(members), [3, 3, 1]);
const [owners, total, ratio] = projectOwnerShare(
[{ id: 'u_1', role: 'owner' }, { id: 'u_2', role: 'member' },
{ id: 'u_3', role: 'member' }]);
assert.deepEqual([owners, total], [1, 3]);
assert.equal(Math.round(ratio * 100) / 100, 0.33);
assert.deepEqual(projectOwnerShare([]), [0, 0, 0]);
});
test('unknown roles are never counted as restricted', () => {
assert.equal(roleOf({ role: 'OWNER' }), 'owner');
assert.equal(roleOf({ role: 'reader' }), 'reader');
assert.equal(roleOf({ role: 'billing' }), 'other');
assert.equal(roleOf({}), 'other');
assert.equal(ownerRatio({}), 0);
});
test('emails are masked', () => {
assert.equal(mask('ada@example.com'), 'a***@example.com');
assert.equal(mask('service-account'), 'service-account');
assert.equal(mask(null), 'unknown');
});
FAQ
Is a lot of owners really a problem, or is this just security theatre?
It is an accountability problem before it is a security one. When eleven people can change a rate limit, a rate limit that changed has eleven candidate authors and no shorter list, and the same is true of an archived project, a raised spend limit and a new admin key. The sharper edge is that an owner can provision an admin API key, which reads the whole organization's usage, costs, users, projects and audit log from anywhere. So the owner count is not a count of people who can administer from a console session, it is a count of people who can mint a long-lived credential that does the same thing.
Why does the script exclude service accounts from the ratio?
Because they are in the same list and they are frequently owners by construction. A service account created to run a nightly job may well have needed owner, and it is not a governance failure that it has one. Including them inflates the ratio, sometimes past the threshold on its own, and produces a report whose top recommendation is to demote a cron job. The script filters on is_service_account before it computes anything and prints how many it removed, so the number you are reading is always about people.
One of our owners shows no API key use at all. Should they be demoted?
Maybe, but not on that field alone. A null api_key_last_used_at means the person has not authenticated an API request, which is exactly what an administrator who works through the console looks like — including the person who actually runs your billing. The script reports it as a question next to the name rather than as a finding, because treating silence as absence is how an audit ends up recommending the removal of the one owner you cannot do without.
We manage membership through SCIM. Does any of this apply?
The reading does; the repair does not. Members with is_scim_managed set have their membership and role projected from your identity provider, so a role changed through this API is reverted at the next sync, leaving an audit-log entry saying somebody tried and a roster that has not moved. The script counts those owners separately and prints a repair aimed at the group mapping in the IdP. Everyone else is repairable here.
Does Anthropic have the same problem?
It has the same shape with different names and one extra consequence. Console organizations use user, developer, billing, admin and claude_code_user rather than owner and reader, and developer is the correct default for people building against the API. The role that matters is admin, because only an admin can provision an Admin API key, and an org:admin token grants access to the whole organization regardless of which workspace the underlying profile is bound to. The equivalent reads are the org user list and, one level down, each workspace's member list.
Related field notes
- A credential that outlives the access it was minted under
- A container that stops appearing in the listing and keeps its keys
- The grants that were offered and never collected
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Administration — OpenAI API reference
- Admin APIs — OpenAI platform docs
- Projects — OpenAI API reference
- openai-python admin API surface
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.