Diagnostic LLM APIs
A vector store with expires_after deletes itself on a clock
The retrieval demo was built in one good afternoon in March, shown twice, and then left alone while the team shipped something else. In May somebody asks to show it again, it comes up, and it answers every question out of the model's own head. The store id is unchanged and still in the config. The store still exists. Its status is expired, its file_counts are all zero, and the file objects it held were deleted on a schedule that was set at creation by a tool nobody remembers configuring.
One paged GET with a project key: GET /v1/vector_stores?limit=100. Every object carries status, expires_after, expires_at and last_active_at, and that is the whole reading. Flag three things: status == "expired", an expires_at inside your notice window, and any policy at all on a store the team treats as permanent.
expires_after is {"anchor": "last_active_at", "days": N}, and the anchor is not a choice: last_active_at is the only supported value. So the clock is an idle timer, which means it runs fastest during exactly the periods when nobody is watching the store — a holiday, a quarter spent on something else, the gap between a prototype and the decision to ship it.
When it fires, the store's status becomes expired and the contained vector_store.file objects are deleted. That is not recoverable from the store. Re-ingesting is the only repair, which is why the useful version of this check runs before the date rather than after it.
Read the expires_at the API returns rather than computing last_active_at + days yourself. The two can disagree, because which operations count as activity is not something the object or the reference states, and the script reports that disagreement as a number instead of resolving it. The API's own field is the countdown; your arithmetic is a guess about a definition you do not have.
The problem in plain words
An expiration policy is a reasonable feature aimed at a real problem — retained bytes bill forever — and it fails because it is set once, at creation, by whoever wrote the create call, and is then invisible to everybody who uses the store afterwards. Nothing on the retrieval path mentions it. The tool configuration holds an id and no metadata. The store keeps working, right up until it does not.
The anchor makes it worse in a specific way. A countdown from creation would at least be predictable: seven days is seven days. A countdown from last activity resets whenever the store is used, which sounds safer and behaves in the opposite direction: the store survives as long as it is busy and dies during the quiet stretch, which is precisely the stretch in which nobody will notice that it did.
The consequence is asymmetric with almost everything else in this section. Most findings here are a number that is wrong or a control that is missing, and the repair is to change a setting. This one deletes data. The vector_store.file objects that the store contained are gone when it expires, and the only way back is to attach the source files again — assuming the source files still exist, which for a corpus assembled by hand during a prototype is not a safe assumption.
The reverse case is quieter and belongs on the other side of the ledger. A store with no policy at all never expires, retains its bytes indefinitely and is billed for them by the hour. That is not a fault; it is a bill, and it is the cost note rather than this one.
Why it happens
The anchor is not a setting, so advice about choosing it is advice about a choice that does not exist. expires_after.anchor has exactly one supported value, last_active_at. Every expiration policy on the platform is an idle timer, and a report that suggests reviewing whether the anchor is the one you wanted will send somebody to look for a dropdown that is not there. What the script reports instead is the anchor arriving as anything other than last_active_at, which would be a change to the platform rather than a fault in your configuration, and is worth reading about before acting.
Trust the reported expires_at over a recomputed one, and report the difference rather than picking a winner. The obvious check is last_active_at + days * 86400, and it is the wrong one to act on, because what counts as activity is not stated anywhere you can read. Attaching a file plausibly counts; a metadata read plausibly does not; a retrieval query almost certainly does. The script computes both, uses the API's number for every decision, and prints the drift as a separate line so that a large gap is visible without ever being interpreted.
A short window is not the finding. A short window on a store you thought was permanent is. Genuinely temporary stores — per-session uploads, one-off evaluations, anything built to be thrown away — should have a policy, and grading them as failures buries the one store that matters. That is why the ids you consider permanent are an input: passing them turns a scheduled expiry into a finding, and passing nothing leaves the script reporting the schedule without claiming it is wrong.
The already-expired case has no repair and must not be printed as though it does. When status is expired, the contained file objects are already deleted and nothing on the API brings them back. Clearing the policy on a store in that state is a change that accomplishes nothing. The output says the files are gone, says re-ingesting is the only path, and puts the policy change on the new store rather than the dead one.
A store that never expires is a cost line, not a clean bill of health. The absence of a policy is reported as its own state, because the same listing that answers this question also answers the opposite one, and a reader who sees only "no findings" will conclude the storage is free. It is billed by the hour on bytes retained, and that reading belongs to a different note.
The fix, as a flow
The only note in the batch whose finding is in the future. An expiration policy is a countdown anchored to the last time the store was active, so it runs fastest exactly when nobody is looking at the store, and it takes the contained file objects with it when it fires. Read the expiry the API reports rather than recomputing it: which operations count as activity is not something the API states.
How to fix it
List the stores with a project key
GET /v1/vector_stores?limit=100, paged on after with has_more and last_id. One call answers this note entirely; there is no need to read a single child object.
Read status before anything else
status == "expired" is the past tense of this note. The contained vector_store.file objects are gone and the counts will all be zero, which is why an expired store and a never-ingested one look identical from the counts alone.
Read expires_after, and treat the anchor as fixed
{"anchor": "last_active_at", "days": N}. last_active_at is the only supported anchor, so every policy is an idle timer. An anchor with any other value is a platform change worth reading about, not a misconfiguration to correct.
Compare the API's expires_at against now, and against your own arithmetic
Use the returned expires_at for the decision. Compute last_active_at + days * 86400 as well and print the difference, because which operations reset the anchor is not documented and a large drift is worth seeing without being acted on.
Pass the ids you consider permanent, and print the repair
PERMANENT_VECTOR_STORE_IDS raises a scheduled expiry on those stores from a note to a finding. The repair for a live store is to clear the policy by updating it to null; for an expired one it is to re-ingest, because the files are not recoverable.
How to check it worked
Clear the policy on the store that should not have had one, then re-run. It should move to permanent, with expires_at absent rather than far away. Re-run again a week later: the value of this check is that it fires before the date, and the only way to know it will is to have it running on a schedule shorter than the shortest days value it reports.
PERMANENT_VECTOR_STORE_IDS=vs_a1,vs_d4 \
python3 openai_vector_store_expiry_audit.py --notice-days 7
# 6 store(s) visible to this key, 4 with an expiration policy
# expired vs_c3 march-demo: expired 84 day(s) ago. The contained
# file objects were deleted and are not recoverable.
# repair: re-ingest into a new store. Clearing the policy on this one changes
# nothing, because the files it held are already gone.
# policy-on-permanent vs_a1 handbook: 7 day idle timer on a store you listed as
# permanent, 2.1 day(s) left
# repair: clear it by updating expires_after to null on the store.
# drift: reported expires_at is 3h ahead of last_active_at + 7d
# expiring-soon vs_b2 policies: 4.8 day(s) left, idle for 25.2 day(s)
# scheduled vs_e5 session-uploads: 1 day idle timer, 0.6 day(s) left
# permanent vs_d4 pricing: no policy, 41.2 MiB retained and billed
# 2 finding(s)The full code
One paged GET and six pure functions. policy, which normalises expires_after into an anchor and an integer day count or None; expiry_at, which coerces the timestamp; idle_seconds, which measures the store against the clock; drift_seconds, which computes the difference between the API's countdown and your own arithmetic and is never used to override the former; anchor_note, which fires only if the anchor is ever anything but last_active_at; and expiry_state, which reads status first, because an expired store has no repair that a live one has.
"""Find OpenAI vector stores that will delete themselves, and ones that have.
Read only. One paged GET against /v1/vector_stores. No request body is
constructed and no file_search query is ever run.
expires_after is {"anchor": "last_active_at", "days": N} and the anchor is not
a choice: last_active_at is the only supported value, so every expiration
policy on the platform is an idle timer. When it fires the store's status
becomes "expired" and the vector_store.file objects it contained are deleted,
which no read call can undo.
Decisions are made on the expires_at the API returns. The obvious alternative,
last_active_at + days, is computed too and reported as a drift, because which
operations reset the anchor is not something the object or the reference
states. Printing the difference is honest; resolving it would be a guess.
"""
import argparse
import logging
import os
import re
import sys
import time
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("openai_vector_store_expiry_audit")
API = "https://api.openai.com/v1"
BETA = {"OpenAI-Beta": "assistants=v2"}
DAY = 86400
# The only anchor the API supports. Anything else is a platform change worth
# reading about rather than a misconfiguration worth correcting.
ANCHOR = "last_active_at"
FINDINGS = ("expired", "policy-on-permanent", "expiring-soon")
def id_set(*raw):
"""The store ids the team treats as permanent. Pure. Order irrelevant."""
out = set()
for chunk in raw:
if not chunk:
continue
items = chunk if isinstance(chunk, (list, tuple)) else [chunk]
for item in items:
for token in re.split(r"[,\s]+", str(item or "").strip()):
if token.strip():
out.add(token.strip())
return out
def policy(store):
"""(anchor, days) from expires_after, or None. Pure.
A policy with a missing or unparseable day count reads as no policy rather
than as a zero-day one, because a zero would grade every such store as
already expiring and the object never actually says that.
"""
raw = (store or {}).get("expires_after")
if not isinstance(raw, dict):
return None
try:
days = int(raw.get("days"))
except (TypeError, ValueError):
return None
if days <= 0:
return None
anchor = str(raw.get("anchor") or "").strip().lower() or ANCHOR
return (anchor, days)
def expiry_at(store):
"""expires_at as an integer, or None. Pure."""
try:
value = int((store or {}).get("expires_at") or 0)
except (TypeError, ValueError):
return None
return value or None
def idle_seconds(store, now):
"""Seconds since last_active_at, or None when the field is absent. Pure."""
try:
last = int((store or {}).get("last_active_at") or 0)
except (TypeError, ValueError):
return None
return (now - last) if last > 0 else None
def drift_seconds(store):
"""reported expires_at minus last_active_at + days. Pure. None if unknown.
Never used to override the reported value. It exists so that a large gap is
visible, because the definition of activity that would explain it is not
published anywhere a script can read.
"""
pol = policy(store)
reported = expiry_at(store)
if not pol or reported is None:
return None
try:
last = int((store or {}).get("last_active_at") or 0)
except (TypeError, ValueError):
return None
if last <= 0:
return None
return reported - (last + pol[1] * DAY)
def anchor_note(store):
"""A line about an unexpected anchor, or None. Pure."""
pol = policy(store)
if pol and pol[0] != ANCHOR:
return ("expires_after.anchor is %r and the only documented value is "
"%r. Read the reference before treating this as a "
"misconfiguration." % (pol[0], ANCHOR))
return None
def expiry_state(store, now, permanent=(), notice_days=7):
"""Classify one store's clock. Pure. Returns (state, detail).
status is read before anything else. An expired store has already lost the
files it held, so it does not share a repair with a store that is merely
close to the same fate.
"""
store = store or {}
sid = str(store.get("id") or "")
pol = policy(store)
reported = expiry_at(store)
idle = idle_seconds(store, now)
if str(store.get("status") or "").strip().lower() == "expired":
ago = ""
if reported:
ago = " %.0f day(s) ago" % max((now - reported) / DAY, 0)
return ("expired",
"expired%s. The contained file objects were deleted and are "
"not recoverable." % ago)
if not pol:
try:
size = int(store.get("usage_bytes") or 0)
except (TypeError, ValueError):
size = 0
return ("permanent",
"no policy, %.1f MiB retained and billed" % (size / 1048576.0))
left = ((reported - now) / DAY) if reported else None
left_text = ("%.1f day(s) left" % left) if left is not None else \
"no expires_at reported"
if sid in set(permanent or ()):
return ("policy-on-permanent",
"%d day idle timer on a store you listed as permanent, %s"
% (pol[1], left_text))
if left is not None and left <= notice_days:
idle_text = (", idle for %.1f day(s)" % (idle / DAY)) if idle else ""
return ("expiring-soon", "%s%s" % (left_text, idle_text))
return ("scheduled", "%d day idle timer, %s" % (pol[1], left_text))
def repair_lines(state, store=None):
"""The repair for one verdict. Pure. Printed, never performed."""
if state == "expired":
return ["re-ingest into a new store. Clearing the policy on this one "
"changes nothing, because the files it held are already gone.",
"set the policy you actually want on the new store at creation, "
"and put whatever produced the corpus into source control so "
"the next re-ingest is a command rather than an afternoon."]
if state == "policy-on-permanent":
return ["clear it by updating expires_after to null on the store. The "
"listing is a read; the clear is a write and is yours to run.",
"the anchor is last_active_at and cannot be changed, so a "
"permanent store cannot be expressed as a long policy. It has "
"to be no policy at all."]
if state == "expiring-soon":
return ["decide which this store is before the date. Temporary is "
"fine and needs no change; permanent means clearing the policy "
"now rather than after the files are deleted.",
"run this check on a schedule shorter than the smallest days "
"value it reports, or it will tell you about the deletion "
"afterwards."]
if state == "permanent":
return ["nothing expires here, which also means nothing is reclaimed. "
"Retained bytes are billed by the hour whether or not anything "
"queries them."]
return []
def get(session, path, **params):
r = session.get(API + path, params=params, timeout=90)
if r.status_code in (401, 403):
raise SystemExit("%d from OpenAI: /v1/vector_stores needs a project key"
% r.status_code)
r.raise_for_status()
return r.json()
def paged(session, path, max_pages=200, **params):
"""Walk an after/last_id cursor listing."""
params = dict(params)
for _ in range(max_pages):
page = get(session, path, **params)
data = page.get("data") or []
for item in data:
yield item
if not page.get("has_more") or not data:
return
params["after"] = page.get("last_id") or (data[-1] or {}).get("id")
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--notice-days", type=float, default=7.0,
help="how far ahead an expiry counts as soon")
ap.add_argument("--permanent", action="append", default=[],
help="a store id your team treats as permanent (repeatable)")
args = ap.parse_args()
key = os.environ.get("OPENAI_API_KEY")
if not key:
log.error("set OPENAI_API_KEY to a project key for the project that "
"owns the vector stores")
return 2
permanent = id_set(os.environ.get("PERMANENT_VECTOR_STORE_IDS"),
args.permanent)
s = requests.Session()
s.headers.update({"Authorization": "Bearer " + key, **BETA})
stores = list(paged(s, "/vector_stores", limit=100))
with_policy = [st for st in stores if policy(st)]
log.info("%d store(s) visible to this key, %d with an expiration policy",
len(stores), len(with_policy))
now = int(time.time())
findings = 0
for store in stores:
sid = (store or {}).get("id") or "?"
name = (store or {}).get("name") or "(unnamed)"
state, detail = expiry_state(store, now, permanent, args.notice_days)
emit = log.warning if state in FINDINGS else log.info
emit("%-20s %s %s: %s", state, sid, name, detail)
for line in repair_lines(state, store):
emit(" repair: %s", line)
note = anchor_note(store)
if note:
emit(" anchor: %s", note)
drift = drift_seconds(store)
if drift is not None and abs(drift) > 3600:
emit(" drift: reported expires_at is %.1fh %s last_active_at plus "
"the policy window", abs(drift) / 3600.0,
"ahead of" if drift > 0 else "behind")
if state in FINDINGS:
findings += 1
log.info("%d finding(s)", findings)
return 1 if findings else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Find OpenAI vector stores that will delete themselves, and ones that have.
*
* Read only. One paged GET against /v1/vector_stores. No request body, and no
* file_search query is ever run.
*
* expires_after is {anchor: "last_active_at", days: N} and the anchor is not a
* choice, so every policy is an idle timer. Decisions are made on the reported
* expires_at; last_active_at + days is computed only to be printed as a drift,
* because which operations reset the anchor is not documented.
*/
const API = 'https://api.openai.com/v1';
const BETA = { 'OpenAI-Beta': 'assistants=v2' };
const DAY = 86400;
/** The only anchor the API supports. */
export const ANCHOR = 'last_active_at';
const FINDINGS = new Set(['expired', 'policy-on-permanent', 'expiring-soon']);
/** The store ids the team treats as permanent. Pure. */
export function idSet(...raw) {
const out = new Set();
for (const chunk of raw) {
if (!chunk) continue;
const items = Array.isArray(chunk) ? chunk : [chunk];
for (const item of items) {
for (const token of String(item ?? '').trim().split(/[,\s]+/)) {
if (token) out.add(token);
}
}
}
return out;
}
/** [anchor, days] from expires_after, or null. Pure. */
export function policy(store) {
const raw = store?.expires_after;
if (!raw || typeof raw !== 'object') return null;
const days = Number(raw.days);
if (!Number.isFinite(days) || Math.trunc(days) <= 0) return null;
const anchor = String(raw.anchor ?? '').trim().toLowerCase() || ANCHOR;
return [anchor, Math.trunc(days)];
}
/** expires_at as an integer, or null. Pure. */
export function expiryAt(store) {
const n = Number(store?.expires_at ?? 0);
return Number.isFinite(n) && n > 0 ? Math.trunc(n) : null;
}
/** Seconds since last_active_at, or null. Pure. */
export function idleSeconds(store, now) {
const last = Number(store?.last_active_at ?? 0);
if (!Number.isFinite(last) || last <= 0) return null;
return now - Math.trunc(last);
}
/** reported expires_at minus last_active_at + days. Pure. Never overrides. */
export function driftSeconds(store) {
const pol = policy(store);
const reported = expiryAt(store);
if (!pol || reported === null) return null;
const last = Number(store?.last_active_at ?? 0);
if (!Number.isFinite(last) || last <= 0) return null;
return reported - (Math.trunc(last) + pol[1] * DAY);
}
/** A line about an unexpected anchor, or null. Pure. */
export function anchorNote(store) {
const pol = policy(store);
if (pol && pol[0] !== ANCHOR) {
return `expires_after.anchor is '${pol[0]}' and the only documented value is `
+ `'${ANCHOR}'. Read the reference before treating this as a misconfiguration.`;
}
return null;
}
/** Classify one store's clock. Pure. Returns [state, detail]. */
export function expiryState(store, now, permanent = new Set(), noticeDays = 7) {
const st = store ?? {};
const sid = String(st.id ?? '');
const pol = policy(st);
const reported = expiryAt(st);
const idle = idleSeconds(st, now);
const perm = permanent instanceof Set ? permanent : new Set(permanent ?? []);
if (String(st.status ?? '').trim().toLowerCase() === 'expired') {
const ago = reported ? ` ${Math.max((now - reported) / DAY, 0).toFixed(0)} day(s) ago` : '';
return ['expired',
`expired${ago}. The contained file objects were deleted and are not `
+ 'recoverable.'];
}
if (!pol) {
const size = Number(st.usage_bytes ?? 0);
return ['permanent',
`no policy, ${(Number.isFinite(size) ? size / 1048576 : 0).toFixed(1)} `
+ 'MiB retained and billed'];
}
const left = reported !== null ? (reported - now) / DAY : null;
const leftText = left !== null ? `${left.toFixed(1)} day(s) left`
: 'no expires_at reported';
if (perm.has(sid)) {
return ['policy-on-permanent',
`${pol[1]} day idle timer on a store you listed as permanent, ${leftText}`];
}
if (left !== null && left <= noticeDays) {
const idleText = idle ? `, idle for ${(idle / DAY).toFixed(1)} day(s)` : '';
return ['expiring-soon', `${leftText}${idleText}`];
}
return ['scheduled', `${pol[1]} day idle timer, ${leftText}`];
}
/** The repair for one verdict. Pure. Printed, never performed. */
export function repairLines(state) {
if (state === 'expired') {
return ['re-ingest into a new store. Clearing the policy on this one changes '
+ 'nothing, because the files it held are already gone.',
'set the policy you actually want on the new store at creation, and '
+ 'put whatever produced the corpus into source control so the next '
+ 're-ingest is a command rather than an afternoon.'];
}
if (state === 'policy-on-permanent') {
return ['clear it by updating expires_after to null on the store. The listing '
+ 'is a read; the clear is a write and is yours to run.',
'the anchor is last_active_at and cannot be changed, so a permanent '
+ 'store cannot be expressed as a long policy. It has to be no policy '
+ 'at all.'];
}
if (state === 'expiring-soon') {
return ['decide which this store is before the date. Temporary is fine and '
+ 'needs no change; permanent means clearing the policy now rather '
+ 'than after the files are deleted.',
'run this check on a schedule shorter than the smallest days value it '
+ 'reports, or it will tell you about the deletion afterwards.'];
}
if (state === 'permanent') {
return ['nothing expires here, which also means nothing is reclaimed. '
+ 'Retained bytes are billed by the hour whether or not anything '
+ 'queries them.'];
}
return [];
}
async function read(key, path, params) {
const url = new URL(API + path);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, String(v));
const r = await fetch(url, { headers: { Authorization: `Bearer ${key}`, ...BETA } });
if (r.status === 401 || r.status === 403) {
throw new Error(`${r.status} from OpenAI: /v1/vector_stores needs a project key`);
}
if (!r.ok) throw new Error(`${r.status} ${await r.text()}`);
return r.json();
}
async function* paged(key, path, params, maxPages = 200) {
const q = { ...params };
for (let i = 0; i < maxPages; i += 1) {
const page = await read(key, path, q);
const data = page.data ?? [];
for (const item of data) yield item;
if (!page.has_more || data.length === 0) return;
q.after = page.last_id ?? data[data.length - 1]?.id;
}
}
async function main() {
const key = process.env.OPENAI_API_KEY;
if (!key) {
console.error('set OPENAI_API_KEY to a project key for the project that owns '
+ 'the vector stores');
process.exitCode = 2;
return;
}
const permanent = idSet(process.env.PERMANENT_VECTOR_STORE_IDS);
const noticeDays = Number(process.env.NOTICE_DAYS ?? 7);
const stores = [];
for await (const st of paged(key, '/vector_stores', { limit: 100 })) stores.push(st);
const withPolicy = stores.filter((st) => policy(st));
console.log(`${stores.length} store(s) visible to this key, ${withPolicy.length} `
+ 'with an expiration policy');
const now = Math.floor(Date.now() / 1000);
let findings = 0;
for (const store of stores) {
const sid = store?.id ?? '?';
const name = store?.name ?? '(unnamed)';
const [state, detail] = expiryState(store, now, permanent, noticeDays);
console.log(`${state.padEnd(20)} ${sid} ${name}: ${detail}`);
for (const line of repairLines(state)) console.log(` repair: ${line}`);
const note = anchorNote(store);
if (note) console.log(` anchor: ${note}`);
const drift = driftSeconds(store);
if (drift !== null && Math.abs(drift) > 3600) {
console.log(` drift: reported expires_at is ${(Math.abs(drift) / 3600).toFixed(1)}h `
+ `${drift > 0 ? 'ahead of' : 'behind'} last_active_at plus the `
+ 'policy window');
}
if (FINDINGS.has(state)) findings += 1;
}
console.log(`${findings} finding(s)`);
process.exitCode = findings ? 1 : 0;
}
if (import.meta.url === `file://${process.argv[1]}`) await main();
Add a test
The first test is the state that has no repair: an expired store, whose output has to say the files are gone and has to refuse to suggest clearing a policy that no longer matters. The second is the one that does have a repair and only exists because you supplied a list — a seven-day timer on a store the team calls permanent, which must be a finding while the identical timer on a session-upload store is not. Then the drift, asserted to be reported and never applied; the anchor check, which must stay silent on the only value the API actually returns; and policy against an expires_after with no usable day count, which has to read as no policy rather than as a zero-day one.
from openai_vector_store_expiry_audit import (anchor_note, drift_seconds,
expiry_at, expiry_state, id_set,
idle_seconds, policy,
repair_lines)
NOW = 1_800_000_000
DAY = 86400
def store(sid="vs_a1", name="handbook", status="completed", days=None,
anchor="last_active_at", expires_at=None, last_active_at=None,
usage_bytes=41_000_000):
row = {"id": sid, "name": name, "status": status, "usage_bytes": usage_bytes,
"last_active_at": last_active_at, "expires_at": expires_at,
"file_counts": {"total": 9, "completed": 9, "failed": 0,
"in_progress": 0, "cancelled": 0}}
if days is not None:
row["expires_after"] = {"anchor": anchor, "days": days}
return row
def test_an_expired_store_has_no_repair_that_touches_the_policy():
# The one state in this note where nothing can be recovered. Saying "clear
# the policy" here would be a change that accomplishes nothing at all.
dead = store(status="expired", days=7, expires_at=NOW - 84 * DAY)
state, detail = expiry_state(dead, NOW)
assert state == "expired"
assert "84 day(s) ago" in detail
assert "not recoverable" in detail
lines = repair_lines(state)
assert any("re-ingest into a new store" in line for line in lines)
assert not any("clear it by updating" in line for line in lines)
def test_the_same_timer_is_a_finding_only_on_a_store_you_called_permanent():
live = store(sid="vs_a1", days=7, expires_at=NOW + 2 * DAY,
last_active_at=NOW - 5 * DAY)
temp = store(sid="vs_e5", name="session-uploads", days=7,
expires_at=NOW + 2 * DAY, last_active_at=NOW - 5 * DAY)
assert expiry_state(live, NOW, {"vs_a1"})[0] == "policy-on-permanent"
assert expiry_state(temp, NOW, {"vs_a1"})[0] == "expiring-soon"
assert any("has to be no policy at all" in line
for line in repair_lines("policy-on-permanent"))
def test_the_reported_expiry_wins_and_the_drift_is_only_printed():
# last_active_at + 7d would put this three hours earlier than the API says.
# The decision uses the API's number; the gap is reported, never resolved.
drifting = store(days=7, last_active_at=NOW - 5 * DAY,
expires_at=NOW + 2 * DAY + 3 * 3600)
assert drift_seconds(drifting) == 3 * 3600
left = (expiry_at(drifting) - NOW) / DAY
assert 2.1 < left < 2.2
assert expiry_state(drifting, NOW, set(), notice_days=7)[0] == "expiring-soon"
assert drift_seconds(store(days=7)) is None
assert drift_seconds(store(expires_at=NOW)) is None
def test_the_anchor_is_only_mentioned_when_it_is_not_the_documented_one():
assert anchor_note(store(days=7)) is None
assert anchor_note(store(days=7, anchor="last_active_at")) is None
assert anchor_note(store()) is None
note = anchor_note(store(days=7, anchor="created_at"))
assert "created_at" in note and "last_active_at" in note
def test_a_policy_with_no_usable_day_count_reads_as_no_policy():
assert policy(store(days=7)) == ("last_active_at", 7)
assert policy(store()) is None
assert policy({"expires_after": {"anchor": "last_active_at"}}) is None
assert policy({"expires_after": {"anchor": "last_active_at", "days": 0}}) is None
assert policy({"expires_after": "7 days"}) is None
assert policy(None) is None
def test_a_store_with_no_policy_is_reported_as_a_bill_not_a_pass():
state, detail = expiry_state(store(usage_bytes=43_200_512), NOW)
assert state == "permanent"
assert "41.2 MiB retained and billed" in detail
assert any("billed by the hour" in line for line in repair_lines(state))
def test_the_clock_helpers_tolerate_a_missing_field():
assert idle_seconds(store(last_active_at=NOW - 3 * DAY), NOW) == 3 * DAY
assert idle_seconds(store(), NOW) is None
assert expiry_at(store()) is None
assert expiry_at({"expires_at": "soon"}) is None
assert id_set("vs_a1, vs_b2", ["vs_a1"]) == {"vs_a1", "vs_b2"}
assert id_set(None) == set()
far = store(days=90, expires_at=NOW + 60 * DAY, last_active_at=NOW - 30 * DAY)
assert expiry_state(far, NOW)[0] == "scheduled"
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { anchorNote, driftSeconds, expiryAt, expiryState, idSet, idleSeconds,
policy, repairLines } from './openai-vector-store-expiry-audit.mjs';
const NOW = 1800000000;
const DAY = 86400;
const store = ({ id = 'vs_a1', name = 'handbook', status = 'completed',
days = null, anchor = 'last_active_at', expiresAt = null,
lastActiveAt = null, usageBytes = 41000000 } = {}) => {
const row = { id, name, status, usage_bytes: usageBytes,
last_active_at: lastActiveAt, expires_at: expiresAt,
file_counts: { total: 9, completed: 9, failed: 0,
in_progress: 0, cancelled: 0 } };
if (days !== null) row.expires_after = { anchor, days };
return row;
};
test('an expired store has no repair that touches the policy', () => {
const dead = store({ status: 'expired', days: 7, expiresAt: NOW - 84 * DAY });
const [state, detail] = expiryState(dead, NOW);
assert.equal(state, 'expired');
assert.match(detail, /84 day\(s\) ago/);
assert.match(detail, /not recoverable/);
const lines = repairLines(state);
assert.ok(lines.some((l) => l.includes('re-ingest into a new store')));
assert.ok(!lines.some((l) => l.includes('clear it by updating')));
});
test('the same timer is a finding only on a store you called permanent', () => {
const live = store({ id: 'vs_a1', days: 7, expiresAt: NOW + 2 * DAY,
lastActiveAt: NOW - 5 * DAY });
const temp = store({ id: 'vs_e5', name: 'session-uploads', days: 7,
expiresAt: NOW + 2 * DAY, lastActiveAt: NOW - 5 * DAY });
assert.equal(expiryState(live, NOW, new Set(['vs_a1']))[0], 'policy-on-permanent');
assert.equal(expiryState(temp, NOW, new Set(['vs_a1']))[0], 'expiring-soon');
assert.ok(repairLines('policy-on-permanent')
.some((l) => l.includes('has to be no policy at all')));
});
test('the reported expiry wins and the drift is only printed', () => {
const drifting = store({ days: 7, lastActiveAt: NOW - 5 * DAY,
expiresAt: NOW + 2 * DAY + 3 * 3600 });
assert.equal(driftSeconds(drifting), 3 * 3600);
const left = (expiryAt(drifting) - NOW) / DAY;
assert.ok(left > 2.1 && left < 2.2);
assert.equal(expiryState(drifting, NOW, new Set(), 7)[0], 'expiring-soon');
assert.equal(driftSeconds(store({ days: 7 })), null);
assert.equal(driftSeconds(store({ expiresAt: NOW })), null);
});
test('the anchor is only mentioned when it is not the documented one', () => {
assert.equal(anchorNote(store({ days: 7 })), null);
assert.equal(anchorNote(store()), null);
const note = anchorNote(store({ days: 7, anchor: 'created_at' }));
assert.match(note, /created_at/);
assert.match(note, /last_active_at/);
});
test('a policy with no usable day count reads as no policy', () => {
assert.deepEqual(policy(store({ days: 7 })), ['last_active_at', 7]);
assert.equal(policy(store()), null);
assert.equal(policy({ expires_after: { anchor: 'last_active_at' } }), null);
assert.equal(policy({ expires_after: { anchor: 'last_active_at', days: 0 } }), null);
assert.equal(policy({ expires_after: '7 days' }), null);
assert.equal(policy(null), null);
});
test('a store with no policy is reported as a bill not a pass', () => {
const [state, detail] = expiryState(store({ usageBytes: 43200512 }), NOW);
assert.equal(state, 'permanent');
assert.match(detail, /41\.2 MiB retained and billed/);
assert.ok(repairLines(state).some((l) => l.includes('billed by the hour')));
});
test('the clock helpers tolerate a missing field', () => {
assert.equal(idleSeconds(store({ lastActiveAt: NOW - 3 * DAY }), NOW), 3 * DAY);
assert.equal(idleSeconds(store(), NOW), null);
assert.equal(expiryAt(store()), null);
assert.equal(expiryAt({ expires_at: 'soon' }), null);
assert.deepEqual([...idSet('vs_a1, vs_b2', ['vs_a1'])].sort(), ['vs_a1', 'vs_b2']);
assert.equal(idSet(null).size, 0);
const far = store({ days: 90, expiresAt: NOW + 60 * DAY,
lastActiveAt: NOW - 30 * DAY });
assert.equal(expiryState(far, NOW)[0], 'scheduled');
});
FAQ
Can I change the anchor so the countdown runs from creation instead?
No. expires_after.anchor has exactly one supported value, last_active_at, so every expiration policy on the platform is an idle timer and there is no creation-anchored variant to switch to. This matters for the repair: a store you want to keep cannot be expressed as a very long policy that you top up, it has to have no policy at all. The script reports an anchor with any other value not as something for you to correct but as a change to the platform worth reading about before acting on.
Why not just compute the expiry from last_active_at plus the day count?
Because you do not know what activity means. The object gives you last_active_at and the reference does not enumerate which operations update it. A retrieval query almost certainly counts, attaching a file plausibly counts, reading the store's metadata plausibly does not, and betting on any of that puts a deletion date in your monitoring that the platform does not agree with. The script uses the expires_at the API returns for every decision and prints the difference between that and the naive sum as a drift line, so a large gap is visible without being interpreted.
The store's status is expired. Can I get the files back by clearing the policy?
No. When a store expires, the vector_store.file objects it contained are deleted, and that is not reversible through the API. Clearing the policy on an expired store leaves you with an empty store and no countdown, which is worse than it sounds because it looks fixed. The only repair is to attach the sources again, into a new store or the same one, and the thing worth fixing at the same time is whatever made the corpus hard to rebuild: an ingest you can re-run from source control turns this from an incident into a command.
Is an expired store the same thing as an empty one?
From the counts, yes, which is exactly the trap. An expired store reports zeroes across file_counts just like a store nothing was ever attached to, and only status tells them apart. The difference matters because the causes are different and one of them recurs: a never-ingested store needs an ingest, and an expired store needs an ingest plus a policy change, or it will be empty again on the same schedule. The empty-store note reads status for that reason and names expiry as the cause when it finds it.
Should every store have a policy then?
Every store you would be happy to lose, yes, and that is more of them than most teams assume: per-session uploads, evaluation corpora, anything a prototype produced. Retained bytes are billed by the hour whether or not anything queries them, so a store with no policy is a standing cost rather than a free safety margin. The script reports policy-free stores with their retained size for that reason, as a bill rather than a pass, and the cost note is where that reading is developed properly.
Related field notes
- The store that has nothing in it and is still named in your config
- What those retained bytes cost while nobody queries them
- The other object on this platform that expires on a clock
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Vector stores — OpenAI API reference
- File search — OpenAI platform docs
- openai-openapi — the published OpenAPI specification
- Files — OpenAI API reference
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.