Diagnostic LLM APIs
Zero data retention is claimed and no project resolves to it
The security questionnaire came back with one line highlighted. Prompts and completions are not retained by the model provider. Somebody wrote that eighteen months ago and it was true of the arrangement negotiated at the time, and nobody has looked since, because there is nothing to look at: no header comes back on a completion saying what the retention mode was, no field in the response, no warning in the logs. Meanwhile four projects have been created since that contract was signed, and each one started life on whatever the organization default happened to be that morning.
Two levels, both read with an organization admin key. GET /v1/organization/data_retention returns an organization.data_retention object with a type drawn from zero_data_retention, modified_abuse_monitoring, enhanced_zero_data_retention and enhanced_modified_abuse_monitoring. Then, for every project from GET /v1/organization/projects?limit=100&include_archived=true, GET /v1/organization/projects/{project_id}/data_retention returns a project.data_retention whose type is drawn from a wider set that adds organization_default and none.
The finding is the resolution, not either level on its own. A project reading organization_default is not configured, it is inherited: it resolves to whatever the org says today and it moves the day the org default moves. A project reading none has no retention control at all regardless of what the org negotiated. Both look identical in a console list of projects, and neither is visible from any request you make.
Read residency off each project object while you are there. It is a separate axis with its own values — GLOBAL, US_STORAGE_PROCESSING, EU_STORAGE_PROCESSING and a list of country storage options — and a project can satisfy a retention claim while sitting in the wrong jurisdiction, or the reverse.
The problem in plain words
Retention posture is configuration, and configuration set at two levels drifts. The organization has a default; a project can pin its own value or inherit. Nothing forces a choice at project creation, so a project created a year after the contract was signed takes whatever the default was that morning, and a change to the org default silently re-points every inheriting project underneath it.
What makes it invisible rather than merely unfixed is that the inference path never mentions it. There is no response header carrying the retention mode, nothing in the usage report, no warning anywhere. The only way to know is to ask the admin endpoint, and the reason nobody asks is that the answer was established once, in a contract negotiation, by people who were not going to be the ones creating projects two years later.
So the failure is not a setting turned off. It is a claim made in one system — a DPA, a questionnaire, a page on the trust site — that was true of the organization at a moment in time and was never a property of the project the regulated workload actually runs in.
Why it happens
The script refuses to rank the six type values, and that refusal is deliberate. It would be easy to write a severity ladder that puts enhanced_zero_data_retention above zero_data_retention above enhanced_modified_abuse_monitoring and grade against it. That ladder would be an invention. Whether modified abuse monitoring satisfies a particular commitment is a question about the commitment, not about the API, and a script that answers it confidently is producing compliance theatre. So the posture you claim is a parameter: the script groups the types into families, takes the family you say you claim, and reports which projects do not reach it.
Inherited and compliant is a third state, not a pass. A project sitting at organization_default while the org default is a ZDR variant is compliant right now and pinned to nothing. It is reported separately from both the compliant projects and the failing ones, because the sentence you need is different: not "fix this" and not "this is fine", but "this resolves correctly today and nothing on the project holds it there". That is the state that turns into an incident when somebody changes the org default for an unrelated reason.
none is not a weaker retention mode, it is the absence of one, and it must not be mistaken for an inherit. The two values sit in the same enum and read similarly at a glance. A project at none gets no retention control whatever the organization says, which is the opposite of inheriting, and the script gives it its own state and its own line so it can never be summarised into the same row as an inheriting project.
The request field and the response field have different names, and the printed repair says so. Reading gives you type. Writing takes retention_type. A repair line copied from the read shape produces a 400, and the person copying it usually concludes the endpoint is broken. The script prints the correct body, and adds the caveat that matters more: ZDR and the enhanced variants generally have to be enabled on the account by OpenAI rather than being self-serve, so the action is to request it, not to set it. The audit script never executes a retention change under any flag.
The fix, as a flow
The claim and the configuration are made in different systems by different people two years apart, and nothing connects them. A project created after the contract takes whatever the organization default happens to be that morning, and no header, field or log line on the inference path ever mentions retention, so the only way the drift surfaces is if somebody goes and asks the admin endpoint.
How to fix it
Use an organization admin key, provisioned read-only
Both retention endpoints and the project listing reject a project key. The script only ever issues GETs; there is no flag that makes it write.
Read the organization default first
GET /v1/organization/data_retention. Its type is the value every inheriting project resolves to, so nothing below it can be graded until you have it.
List every project, archived ones included
GET /v1/organization/projects?limit=100&include_archived=true, paginated on after. An archived project cannot take new traffic and its retained data is still retained, so it is graded and labelled rather than skipped.
Resolve each project against the default
GET /v1/organization/projects/{project_id}/data_retention. Map organization_default onto the org's value and keep the fact that it was inherited; treat none as its own state; treat an unreadable or unrecognised value as unreadable rather than as safe.
Grade residency as a separate axis and print the repair
residency on the project object answers a different question from retention and is reported on its own line. The repair is POST /v1/organization/projects/{project_id}/data_retention with a retention_type body — printed, never performed, and annotated with the fact that the enhanced variants are requested rather than set.
How to check it worked
Pin the projects that should be pinned, then re-run. Every row that read inherited-not-pinned should read compliant with the value now set on the project itself, and the count of projects resolving through the org default should be the ones you deliberately left inheriting. Change the org default afterwards on purpose, in a test organization if you have one, and confirm the pinned rows do not move.
python3 openai_data_retention_audit.py --require zdr --residency EU_STORAGE_PROCESSING
# organization default: modified_abuse_monitoring
# 9 project(s), 4 finding(s)
# no-retention-control proj_ingest Ingest pipeline
# type is none: no retention control at all, whatever the organization default says
# repair: POST /v1/organization/projects/proj_ingest/data_retention
# {"retention_type": "zero_data_retention"}
# repair: the request field is retention_type; the response field is type. A body
# copied from the read shape 400s.
# weaker-than-claimed proj_web Customer web app
# resolves to modified_abuse_monitoring (inherited from the organization), and
# zero data retention was claimed
# inherited-not-pinned proj_eu EU tenant
# resolves to enhanced_zero_data_retention only because the organization default
# says so. Nothing on the project pins it.
# residency proj_eu residency is US_STORAGE_PROCESSING, and
# EU_STORAGE_PROCESSING was claimedThe full code
One GET for the org, one paged GET for the projects, one GET per project, and six pure functions. The family map, which groups the six type values into zdr, modified-abuse-monitoring and none without ranking them against each other; the resolver, which returns both the effective type and whether it was inherited, because those are two different facts; the classifier, which tests unreadable before none before family before inheritance; the residency check, which treats a missing value as unset rather than as GLOBAL; the archived label; and the repair lines, which print a retention_type body and say out loud that the enhanced variants are requested rather than set.
"""Find OpenAI projects whose retention posture is not the one you claim.
Read only. One GET for the organization default, one paged GET for the project
list, and one GET per project. Every request is a GET and no request body is
constructed anywhere, including for the repair, which is printed as text.
The two levels disagree more often than anyone expects, and the interesting
answer is the resolution: what a project actually gets, and whether anything on
the project holds it there.
"""
import argparse
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("openai_data_retention_audit")
API = "https://api.openai.com/v1"
ZDR = ("zero_data_retention", "enhanced_zero_data_retention")
MAM = ("modified_abuse_monitoring", "enhanced_modified_abuse_monitoring")
INHERIT = "organization_default"
NO_CONTROL = "none"
# Families, not a ranking. Whether modified abuse monitoring satisfies a given
# commitment is a question about the commitment; the script will not answer it.
FAMILY_LABEL = {"zdr": "zero data retention",
"modified-abuse-monitoring": "modified abuse monitoring"}
# What to write when a project has to be brought up to the claimed family. The
# enhanced variants are requested from OpenAI rather than set, and the printed
# repair says so.
TARGET = {"zdr": "zero_data_retention",
"modified-abuse-monitoring": "modified_abuse_monitoring"}
FINDINGS = ("retention-unreadable", "no-retention-control", "weaker-than-claimed",
"inherited-not-pinned")
SEVERITY = {"no-retention-control": 0, "weaker-than-claimed": 1,
"retention-unreadable": 2, "inherited-not-pinned": 3}
def family(retention_type):
"""Group one type value into a family. Pure. Never ranks families."""
t = str(retention_type or "").strip().lower()
if not t:
return "unreadable"
if t in ZDR:
return "zdr"
if t in MAM:
return "modified-abuse-monitoring"
if t == NO_CONTROL:
return "none"
return "unrecognised"
def effective(org_type, project_type):
"""(type, inherited) for one project. Pure.
organization_default is the whole reason this function exists: the project
reports a word rather than a posture, and the posture is one level up.
"""
t = str(project_type or "").strip().lower()
if not t:
return (None, False)
if t == INHERIT:
return (str(org_type or "").strip().lower() or None, True)
return (t, False)
def archived(project):
"""Is this project archived? Pure. Both signals, because they disagree."""
row = project or {}
return bool(row.get("archived_at")) or str(row.get("status") or "") == "archived"
def classify(project, org_type, project_type, require="zdr"):
"""Classify one project's retention. Pure. Returns (state, detail).
Order matters: unreadable, then none, then family, then inheritance. An
unrecognised value is never graded as safe, and none is never summarised
into the same row as an inherit even though they sit in the same enum.
"""
eff, inherited = effective(org_type, project_type)
fam = family(eff)
tail = " (archived, and its retained data is still retained)" if archived(project) else ""
want = FAMILY_LABEL.get(require, require)
if fam in ("unreadable", "unrecognised"):
return ("retention-unreadable",
"the project reports %s, which this audit will not grade as safe%s"
% (repr(str(project_type)) if project_type else "nothing", tail))
if fam == "none":
return ("no-retention-control",
"type is none: no retention control at all, whatever the "
"organization default says%s" % tail)
if fam != require:
return ("weaker-than-claimed",
"resolves to %s (%s)%s, and %s was claimed"
% (eff,
"inherited from the organization" if inherited
else "set on the project", tail, want))
if inherited:
return ("inherited-not-pinned",
"resolves to %s only because the organization default says so. "
"Nothing on the project pins it%s" % (eff, tail))
return ("compliant", "pinned on the project at %s%s" % (eff, tail))
def residency_note(project, want):
"""(ok, detail) on the residency axis. Pure. Absent is unset, not GLOBAL."""
if not want:
return (True, None)
got = (project or {}).get("residency")
if got is None:
return (False, "residency is unset on this project, which is neither "
"GLOBAL nor %s" % want)
if str(got) != str(want):
return (False, "residency is %s, and %s was claimed" % (got, want))
return (True, None)
def repair_lines(state, project, require="zdr"):
"""The repair for one project. Pure. Printed, never performed."""
pid = str((project or {}).get("id") or "unknown")
lines = []
if state not in FINDINGS:
return lines
if state == "inherited-not-pinned":
lines.append("this resolves correctly today and moves the day somebody "
"changes the organization default. Pin it on the project if "
"the commitment is about this workload.")
elif state == "retention-unreadable":
lines.append("the endpoint returned a value this audit does not "
"recognise. Read it by hand before assuming anything.")
target = TARGET.get(require)
if target:
lines.append("POST /v1/organization/projects/%s/data_retention with a body "
'of {"retention_type": "%s"}' % (pid, target))
lines.append("the request field is retention_type; the response field is "
"type. A body copied from the read shape 400s.")
lines.append("zero data retention and the enhanced variants are generally "
"enabled on the account by OpenAI rather than being "
"self-serve. Request it; do not assume the call will take.")
return lines
def get(session, path, params=None):
r = session.get(API + path, params=params or {}, timeout=60)
if r.status_code in (401, 403):
raise SystemExit("%d from OpenAI: /v1/organization/* needs an organization "
"admin key, not a project key" % r.status_code)
if r.status_code == 404:
return None
r.raise_for_status()
return r.json()
def paged(session, path, **params):
params = dict(params)
while True:
page = get(session, path, params) or {}
data = page.get("data") or []
for item in data:
yield item
if not page.get("has_more") or not data:
return
params["after"] = page.get("last_id") or (data[-1] or {}).get("id")
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--require", default="zdr",
choices=sorted(FAMILY_LABEL),
help="the retention family your commitments claim")
ap.add_argument("--residency", default=None,
help="the project residency your commitments claim, "
"e.g. EU_STORAGE_PROCESSING")
args = ap.parse_args()
admin = os.environ.get("OPENAI_ADMIN_KEY")
if not admin:
log.error("set OPENAI_ADMIN_KEY to an organization admin key; a project "
"key cannot read /v1/organization/data_retention")
return 2
s = requests.Session()
s.headers.update({"Authorization": "Bearer " + admin})
org = get(s, "/organization/data_retention") or {}
org_type = org.get("type")
log.info("organization default: %s", org_type or "unreadable")
projects = list(paged(s, "/organization/projects", limit=100,
include_archived="true"))
findings = []
for project in projects:
pid = str(project.get("id") or "")
block = get(s, "/organization/projects/%s/data_retention" % pid) or {}
state, detail = classify(project, org_type, block.get("type"), args.require)
if state in FINDINGS:
findings.append((project, state, detail))
residency_bad = []
if args.residency:
for project in projects:
ok, detail = residency_note(project, args.residency)
if not ok:
residency_bad.append((project, detail))
log.info("%d project(s), %d retention finding(s), %d residency finding(s)",
len(projects), len(findings), len(residency_bad))
findings.sort(key=lambda r: (SEVERITY.get(r[1], 9), str(r[0].get("name") or "")))
for project, state, detail in findings:
log.warning("%-22s %-14s %s", state, project.get("id"),
project.get("name") or "(unnamed)")
log.warning(" %s", detail)
for line in repair_lines(state, project, args.require):
log.warning(" repair: %s", line)
for project, detail in residency_bad:
log.warning("%-22s %-14s %s", "residency", project.get("id"), detail)
return 1 if (findings or residency_bad) else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Find OpenAI projects whose retention posture is not the one you claim.
*
* Read only. One GET for the organization default, one paged GET for the
* project list, one GET per project. No request body is constructed anywhere,
* including for the repair, which is printed as text.
*/
const API = 'https://api.openai.com/v1';
const ZDR = new Set(['zero_data_retention', 'enhanced_zero_data_retention']);
const MAM = new Set(['modified_abuse_monitoring', 'enhanced_modified_abuse_monitoring']);
const INHERIT = 'organization_default';
const NO_CONTROL = 'none';
const FAMILY_LABEL = { zdr: 'zero data retention',
'modified-abuse-monitoring': 'modified abuse monitoring' };
const TARGET = { zdr: 'zero_data_retention',
'modified-abuse-monitoring': 'modified_abuse_monitoring' };
const FINDINGS = new Set(['retention-unreadable', 'no-retention-control',
'weaker-than-claimed', 'inherited-not-pinned']);
const SEVERITY = { 'no-retention-control': 0, 'weaker-than-claimed': 1,
'retention-unreadable': 2, 'inherited-not-pinned': 3 };
/** Group one type value into a family. Pure. Never ranks families. */
export function family(retentionType) {
const t = String(retentionType ?? '').trim().toLowerCase();
if (!t) return 'unreadable';
if (ZDR.has(t)) return 'zdr';
if (MAM.has(t)) return 'modified-abuse-monitoring';
if (t === NO_CONTROL) return 'none';
return 'unrecognised';
}
/** [type, inherited] for one project. Pure. */
export function effective(orgType, projectType) {
const t = String(projectType ?? '').trim().toLowerCase();
if (!t) return [null, false];
if (t === INHERIT) return [String(orgType ?? '').trim().toLowerCase() || null, true];
return [t, false];
}
/** Is this project archived? Pure. Both signals, because they disagree. */
export function archived(project) {
return Boolean(project?.archived_at) || String(project?.status ?? '') === 'archived';
}
/** Classify one project's retention. Pure. Returns [state, detail]. */
export function classify(project, orgType, projectType, require = 'zdr') {
const [eff, inherited] = effective(orgType, projectType);
const fam = family(eff);
const tail = archived(project)
? ' (archived, and its retained data is still retained)' : '';
const want = FAMILY_LABEL[require] ?? require;
if (fam === 'unreadable' || fam === 'unrecognised') {
return ['retention-unreadable',
`the project reports ${projectType ? `'${projectType}'` : 'nothing'}, `
+ `which this audit will not grade as safe${tail}`];
}
if (fam === 'none') {
return ['no-retention-control',
'type is none: no retention control at all, whatever the organization '
+ `default says${tail}`];
}
if (fam !== require) {
return ['weaker-than-claimed',
`resolves to ${eff} (${inherited ? 'inherited from the organization'
: 'set on the project'})${tail}, and ${want} was claimed`];
}
if (inherited) {
return ['inherited-not-pinned',
`resolves to ${eff} only because the organization default says so. `
+ `Nothing on the project pins it${tail}`];
}
return ['compliant', `pinned on the project at ${eff}${tail}`];
}
/** [ok, detail] on the residency axis. Pure. Absent is unset, not GLOBAL. */
export function residencyNote(project, want) {
if (!want) return [true, null];
const got = project?.residency ?? null;
if (got === null) {
return [false, 'residency is unset on this project, which is neither GLOBAL '
+ `nor ${want}`];
}
if (String(got) !== String(want)) {
return [false, `residency is ${got}, and ${want} was claimed`];
}
return [true, null];
}
/** The repair for one project. Pure. Printed, never performed. */
export function repairLines(state, project, require = 'zdr') {
const pid = String(project?.id ?? 'unknown');
const lines = [];
if (!FINDINGS.has(state)) return lines;
if (state === 'inherited-not-pinned') {
lines.push('this resolves correctly today and moves the day somebody changes '
+ 'the organization default. Pin it on the project if the commitment is '
+ 'about this workload.');
} else if (state === 'retention-unreadable') {
lines.push('the endpoint returned a value this audit does not recognise. Read '
+ 'it by hand before assuming anything.');
}
const target = TARGET[require];
if (target) {
lines.push(`POST /v1/organization/projects/${pid}/data_retention with a body of `
+ `{"retention_type": "${target}"}`);
lines.push('the request field is retention_type; the response field is type. A '
+ 'body copied from the read shape 400s.');
lines.push('zero data retention and the enhanced variants are generally enabled '
+ 'on the account by OpenAI rather than being self-serve. Request it; do not '
+ 'assume the call will take.');
}
return lines;
}
async function read(key, path, params = {}) {
const url = new URL(API + path);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, String(v));
const r = await fetch(url, { headers: { Authorization: `Bearer ${key}` } });
if (r.status === 401 || r.status === 403) {
throw new Error(`${r.status} from OpenAI: /v1/organization/* needs an `
+ 'organization admin key, not a project key');
}
if (r.status === 404) return null;
if (!r.ok) throw new Error(`${r.status} ${await r.text()}`);
return r.json();
}
async function paged(key, path, params) {
const out = [];
const q = { ...params };
for (;;) {
const page = (await read(key, path, q)) ?? {};
const data = page.data ?? [];
out.push(...data);
if (!page.has_more || data.length === 0) return out;
q.after = page.last_id ?? data[data.length - 1]?.id;
}
}
async function main() {
const admin = process.env.OPENAI_ADMIN_KEY;
if (!admin) {
console.error('set OPENAI_ADMIN_KEY to an organization admin key; a project '
+ 'key cannot read /v1/organization/data_retention');
process.exitCode = 2;
return;
}
const require_ = process.env.REQUIRE ?? 'zdr';
const wantResidency = process.env.RESIDENCY ?? null;
const org = (await read(admin, '/organization/data_retention')) ?? {};
console.log(`organization default: ${org.type ?? 'unreadable'}`);
const projects = await paged(admin, '/organization/projects',
{ limit: 100, include_archived: 'true' });
const findings = [];
for (const project of projects) {
const block = (await read(admin,
`/organization/projects/${String(project.id ?? '')}/data_retention`)) ?? {};
const [state, detail] = classify(project, org.type, block.type, require_);
if (FINDINGS.has(state)) findings.push([project, state, detail]);
}
const residencyBad = [];
if (wantResidency) {
for (const project of projects) {
const [ok, detail] = residencyNote(project, wantResidency);
if (!ok) residencyBad.push([project, detail]);
}
}
console.log(`${projects.length} project(s), ${findings.length} retention `
+ `finding(s), ${residencyBad.length} residency finding(s)`);
findings.sort(([pa, sa], [pb, sb]) =>
(SEVERITY[sa] ?? 9) - (SEVERITY[sb] ?? 9)
|| String(pa.name ?? '').localeCompare(String(pb.name ?? '')));
for (const [project, state, detail] of findings) {
console.warn(`${state.padEnd(22)} ${String(project.id).padEnd(14)} `
+ `${project.name ?? '(unnamed)'}`);
console.warn(` ${detail}`);
for (const line of repairLines(state, project, require_)) {
console.warn(` repair: ${line}`);
}
}
for (const [project, detail] of residencyBad) {
console.warn(`${'residency'.padEnd(22)} ${String(project.id).padEnd(14)} ${detail}`);
}
process.exitCode = (findings.length || residencyBad.length) ? 1 : 0;
}
if (import.meta.url === `file://${process.argv[1]}`) await main();
Add a test
The first test is the note: one project, one value, read twice under two different organization defaults. A project sitting at organization_default is inherited-not-pinned when the org is a ZDR variant and weaker-than-claimed when it is not, and nothing on the project changed between the two readings. The second test keeps none away from the inherit path, because both words live in the same enum and only one of them means “ask the level above”. Then: the family map, which groups the enhanced variants without ranking them; an unrecognised string, which must be unreadable rather than safe; residency, which is a separate axis where absent is unset and not GLOBAL; and the repair body, which has to say retention_type and has to say the enhanced variants are requested rather than set.
from openai_data_retention_audit import (archived, classify, effective,
family, repair_lines, residency_note)
EU = {"id": "proj_eu", "name": "EU tenant", "status": "active",
"residency": "EU_STORAGE_PROCESSING"}
def test_the_same_project_resolves_two_ways_under_two_org_defaults():
# The note. Nothing on proj_eu changes between these two readings.
state, detail = classify(EU, "enhanced_zero_data_retention",
"organization_default", "zdr")
assert state == "inherited-not-pinned"
assert "only because the organization default says so" in detail
assert any("moves the day somebody changes" in line
for line in repair_lines(state, EU, "zdr"))
state, detail = classify(EU, "modified_abuse_monitoring",
"organization_default", "zdr")
assert state == "weaker-than-claimed"
assert "inherited from the organization" in detail
assert "zero data retention was claimed" in detail
# And pinned on the project is a pass, not a finding.
assert classify(EU, "modified_abuse_monitoring",
"zero_data_retention", "zdr")[0] == "compliant"
def test_none_is_never_treated_as_an_inherit():
state, detail = classify({"id": "proj_ingest"}, "enhanced_zero_data_retention",
"none", "zdr")
assert state == "no-retention-control"
assert "whatever the organization default says" in detail
# The org default is a ZDR variant and the project still fails, which is the
# whole reason none has its own state.
assert effective("enhanced_zero_data_retention", "none") == ("none", False)
assert effective("enhanced_zero_data_retention", "organization_default") == \
("enhanced_zero_data_retention", True)
assert effective("zero_data_retention", None) == (None, False)
def test_the_family_map_groups_without_ranking():
assert family("zero_data_retention") == "zdr"
assert family("enhanced_zero_data_retention") == "zdr"
assert family("modified_abuse_monitoring") == "modified-abuse-monitoring"
assert family("enhanced_modified_abuse_monitoring") == "modified-abuse-monitoring"
assert family("none") == "none"
assert family(None) == "unreadable"
assert family("standard") == "unrecognised"
# Claiming the weaker family makes a MAM project pass, and that is a
# decision the caller makes rather than one the script makes for them.
project = {"id": "proj_a"}
assert classify(project, None, "modified_abuse_monitoring",
"modified-abuse-monitoring")[0] == "compliant"
assert classify(project, None, "modified_abuse_monitoring",
"zdr")[0] == "weaker-than-claimed"
def test_an_unrecognised_value_is_never_graded_as_safe():
state, detail = classify({"id": "proj_x"}, "zero_data_retention",
"legacy_mode", "zdr")
assert state == "retention-unreadable"
assert "will not grade as safe" in detail
assert any("Read it by hand" in line for line in repair_lines(state,
{"id": "proj_x"}))
# A project the endpoint would not answer for is unreadable too.
assert classify({"id": "proj_y"}, "zero_data_retention", None,
"zdr")[0] == "retention-unreadable"
def test_archived_projects_are_graded_and_labelled():
old = {"id": "proj_old", "archived_at": 1_700_000_000}
assert archived(old) is True
assert archived({"id": "p", "status": "archived"}) is True
assert archived({"id": "p", "status": "active"}) is False
state, detail = classify(old, "modified_abuse_monitoring", "none", "zdr")
assert state == "no-retention-control"
assert "its retained data is still retained" in detail
def test_residency_is_a_separate_axis_and_absent_is_not_global():
ok, detail = residency_note(EU, "EU_STORAGE_PROCESSING")
assert ok is True and detail is None
ok, detail = residency_note({"id": "p", "residency": "US_STORAGE_PROCESSING"},
"EU_STORAGE_PROCESSING")
assert ok is False
assert "residency is US_STORAGE_PROCESSING" in detail
ok, detail = residency_note({"id": "p"}, "EU_STORAGE_PROCESSING")
assert ok is False
assert "neither GLOBAL nor" in detail
# No claim, no finding.
assert residency_note({"id": "p"}, None) == (True, None)
def test_the_repair_body_uses_retention_type_and_says_it_is_a_request():
lines = repair_lines("no-retention-control", {"id": "proj_ingest"}, "zdr")
assert any('{"retention_type": "zero_data_retention"}' in line for line in lines)
assert any("the response field is type" in line for line in lines)
assert any("Request it" in line for line in lines)
assert repair_lines("compliant", {"id": "proj_ingest"}, "zdr") == []
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { archived, classify, effective, family, repairLines, residencyNote }
from './openai-data-retention-audit.mjs';
const EU = { id: 'proj_eu', name: 'EU tenant', status: 'active',
residency: 'EU_STORAGE_PROCESSING' };
test('the same project resolves two ways under two org defaults', () => {
let [state, detail] = classify(EU, 'enhanced_zero_data_retention',
'organization_default', 'zdr');
assert.equal(state, 'inherited-not-pinned');
assert.ok(detail.includes('only because the organization default says so'));
assert.ok(repairLines(state, EU, 'zdr')
.some((l) => l.includes('moves the day somebody changes')));
[state, detail] = classify(EU, 'modified_abuse_monitoring',
'organization_default', 'zdr');
assert.equal(state, 'weaker-than-claimed');
assert.ok(detail.includes('inherited from the organization'));
assert.ok(detail.includes('zero data retention was claimed'));
assert.equal(classify(EU, 'modified_abuse_monitoring',
'zero_data_retention', 'zdr')[0], 'compliant');
});
test('none is never treated as an inherit', () => {
const [state, detail] = classify({ id: 'proj_ingest' },
'enhanced_zero_data_retention', 'none', 'zdr');
assert.equal(state, 'no-retention-control');
assert.ok(detail.includes('whatever the organization default says'));
assert.deepEqual(effective('enhanced_zero_data_retention', 'none'), ['none', false]);
assert.deepEqual(effective('enhanced_zero_data_retention', 'organization_default'),
['enhanced_zero_data_retention', true]);
assert.deepEqual(effective('zero_data_retention', null), [null, false]);
});
test('the family map groups without ranking', () => {
assert.equal(family('zero_data_retention'), 'zdr');
assert.equal(family('enhanced_zero_data_retention'), 'zdr');
assert.equal(family('modified_abuse_monitoring'), 'modified-abuse-monitoring');
assert.equal(family('enhanced_modified_abuse_monitoring'),
'modified-abuse-monitoring');
assert.equal(family('none'), 'none');
assert.equal(family(null), 'unreadable');
assert.equal(family('standard'), 'unrecognised');
const project = { id: 'proj_a' };
assert.equal(classify(project, null, 'modified_abuse_monitoring',
'modified-abuse-monitoring')[0], 'compliant');
assert.equal(classify(project, null, 'modified_abuse_monitoring', 'zdr')[0],
'weaker-than-claimed');
});
test('an unrecognised value is never graded as safe', () => {
const [state, detail] = classify({ id: 'proj_x' }, 'zero_data_retention',
'legacy_mode', 'zdr');
assert.equal(state, 'retention-unreadable');
assert.ok(detail.includes('will not grade as safe'));
assert.ok(repairLines(state, { id: 'proj_x' })
.some((l) => l.includes('Read it by hand')));
assert.equal(classify({ id: 'proj_y' }, 'zero_data_retention', null, 'zdr')[0],
'retention-unreadable');
});
test('archived projects are graded and labelled', () => {
const old = { id: 'proj_old', archived_at: 1700000000 };
assert.equal(archived(old), true);
assert.equal(archived({ id: 'p', status: 'archived' }), true);
assert.equal(archived({ id: 'p', status: 'active' }), false);
const [state, detail] = classify(old, 'modified_abuse_monitoring', 'none', 'zdr');
assert.equal(state, 'no-retention-control');
assert.ok(detail.includes('its retained data is still retained'));
});
test('residency is a separate axis and absent is not GLOBAL', () => {
assert.deepEqual(residencyNote(EU, 'EU_STORAGE_PROCESSING'), [true, null]);
let [ok, detail] = residencyNote({ id: 'p', residency: 'US_STORAGE_PROCESSING' },
'EU_STORAGE_PROCESSING');
assert.equal(ok, false);
assert.ok(detail.includes('residency is US_STORAGE_PROCESSING'));
[ok, detail] = residencyNote({ id: 'p' }, 'EU_STORAGE_PROCESSING');
assert.equal(ok, false);
assert.ok(detail.includes('neither GLOBAL nor'));
assert.deepEqual(residencyNote({ id: 'p' }, null), [true, null]);
});
test('the repair body uses retention_type and says it is a request', () => {
const lines = repairLines('no-retention-control', { id: 'proj_ingest' }, 'zdr');
assert.ok(lines.some((l) => l.includes('{"retention_type": "zero_data_retention"}')));
assert.ok(lines.some((l) => l.includes('the response field is type')));
assert.ok(lines.some((l) => l.includes('Request it')));
assert.deepEqual(repairLines('compliant', { id: 'proj_ingest' }, 'zdr'), []);
});
FAQ
Why does the script make me tell it what posture I claim?
Because the API returns six values and no ordering, and inventing one would be the worst thing this script could do. It is genuinely unclear whether enhanced_modified_abuse_monitoring satisfies a commitment that a customer signed off as zero data retention; that is a question for whoever wrote the commitment, and the answer differs between contracts. So the script groups the six values into families it can define objectively, takes the family you say you claim, and reports which projects do not reach it. Everything it asserts is then a fact about your organization plus one input you supplied, rather than a compliance opinion it made up.
Our organization default is set correctly. Why is a project still flagged?
Two reasons, and they are different. If the project reads none, it has no retention control at all and the organization default does not reach it — none and organization_default sit in the same enum and mean opposite things. If the project reads organization_default and the org value satisfies your claim, the script reports inherited-not-pinned, which is not a failure today: it is a note that nothing on the project holds it there, so the posture of that workload changes the next time somebody edits the org default for an unrelated reason. Whether to pin it depends on whether the commitment is about the workload or about the account.
Can the script fix this for me?
No, and it would be the wrong shape of tool if it could. The corrective call is printed rather than executed, for the usual reason in this section, and for one specific to retention: zero data retention and the enhanced variants generally have to be enabled on the account by OpenAI rather than being self-serve, so a script that fires the POST would report success or failure that has nothing to do with whether your account is entitled to the setting. The repair line says request this rather than set this. There is also a field-name trap worth knowing: you read type and you write retention_type.
What does residency have to do with retention?
Nothing, which is exactly why it is on a separate line. Retention answers how long prompts and completions are held; residency answers where they are stored and processed. A project can hold zero data retention and still be sitting at GLOBAL when the contract said EU_STORAGE_PROCESSING, and a project can be correctly in the EU and retain everything. The script reads residency off the project object because it is free to read while you are already listing projects, grades it against a separate claim you pass in, and never blends the two into one verdict.
Does the audit log show me when this last changed?
Partly. GET /v1/organization/audit_logs carries organization.updated and project.updated events, and a retention change shows up as one of those, so you can establish roughly when the posture moved and who moved it. What the log does not do is give you a clean before-and-after of the retention type itself, and it does not reach back further than the log's own retention window. Use it to reconstruct a story once the current-state audit has told you there is a story; do not use it as the primary check.
Related field notes
- An encryption config the API itself describes as inert
- Whether any policy would have stopped the call
- A container that stops being listed and keeps working
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Your data — OpenAI platform docs
- Admin APIs — OpenAI platform docs
- Projects — OpenAI API reference
- openai-python admin API surface
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.