Skip to content

Diagnostic LLM APIs

previous_response_id 404s once the parent has aged out

The support assistant remembers everything, which is the entire product. A customer opens a thread in March, adds to it in April, and comes back in June to ask about the thing they described the first time. This time the request 404s. Not the model, not the key, not the prompt: the id of the message before this one, which your code has been passing forward faithfully for three months. Server-side conversation state is a convenience with an expiry date on it, and nobody read the date.

Read-only key Python and Node.js Tests included
A pile of items sitting on top of a table
Photo by Growtika on Unsplash
The short answer

Walk the chain before a customer does. With a project key set to Read Only, take the response ids your application recorded and call GET /v1/responses/{response_id}. A 404 is a link that is already gone, and the next turn on that thread will fail. A 200 gives you created_at and previous_response_id, so the script hops upward and repeats until it reaches a root or a gap.

The clock is documented and it is short. Response objects are saved for 30 days by default. So a chain is only as durable as its oldest surviving link, and the script computes the runway from that link rather than from the newest one, which is the mistake the calendar invites you to make.

There is one documented exception and it is also the repair: a response attached to a conversation has its items persisted with no 30-day TTL. The script reads the conversation field off every link it resolves and reports a chain whose history lives in a conversation object separately, because that history survives whether or not the individual response objects do.

You cannot enumerate this. /v1/responses has no list endpoint, so the ids come from your own records and every verdict is bounded by the chains you handed over. If you are not recording response ids, that is the first repair, and it is a column.

The problem in plain words

previous_response_id looks like memory and behaves like a cache. It chains server-side state so that a turn only has to carry the newest message, which is genuinely useful: less bandwidth, less assembly code, no local history store. What it does not look like, anywhere in the call site, is a reference into storage with a retention policy.

Response objects are saved for 30 days by default. Past that the parent is gone, and the next request that names it comes back as a 404 that points at previous_response_id. Nothing warns at write time, because at write time everything is fine; the break happens on the turn after the gap, which may be weeks later and will be in front of whoever is using the product.

Storage is also conditional in the first place. A response only exists to be chained from if the call that created it was stored, which is the default and is not universal: a Zero Data Retention organization does not get stored responses at all, a call that opted out has nothing to chain from, and a cleanup sweep that deletes responses does the same damage as the clock does, only faster.

The failure shape is what makes this expensive. It is not uniform, so it does not look like an outage: the threads that break are the oldest ones, which correlate with your most engaged users, and they break one at a time. A dashboard sees a handful of 404s a day and a rounding error in the error rate. A customer sees an assistant that forgot a conversation they have been having for a month.

Thread startsin Marchone id chainedforwardTurns added forweekseach one worksParent passes30 daysretention, notdeletionNext turn 404sonprevious_response_idAssistantforgets userthe most engagedone first
The newest id resolves perfectly. The request fails on a parent that quietly aged out weeks ago.

Why it happens

A chain is exactly as durable as its oldest surviving link, and that is not the link you would think to check. The natural instinct is to test the most recent id, which is the one that is freshest and will always pass. The turn that fails is the one that walks back to a parent recorded a month ago. So the script hops upward until it reaches a root or a gap, and it computes the runway from the oldest link it found rather than from the leaf it started at.

A 200 is the proof of storage, because there is no field for it. The Response object does not carry a store flag to read back, so the honest test of "was this stored" is whether retrieving it works. The script says exactly that rather than reporting a field that does not exist, and it treats a 404 as one fact with two possible causes — it aged out, or it was never stored — which are separated by whether the chain has other links that resolve.

The conversation exception is a different object with a different rule, and it is the repair. Items attached to a conversation are persisted with no 30-day TTL, so a thread built on a conversation does not have this failure mode. That is why the script reads the conversation field on every link it resolves: a chain that is already conversation-backed has its history somewhere durable, whatever happens to the individual response objects, and it is not this note's problem.

This clock is not the batch clock. The section has notes about a batch abandoning unfinished rows after a 24-hour completion window, and about batch error files that expire before anyone reads them. Those are deadlines on a job you submitted. This is a retention policy on an object your application keeps pointing at, it fails forward one turn at a time rather than all at once, and its repair is a design change rather than a resubmission.

Every verdict here is bounded by the ids you supplied. There is no list endpoint for /v1/responses, so this cannot sweep the account; it can only grade the chains you know about. That makes the sample part of the finding, which is why the script prints how many chains it walked next to what it concluded, and why the first repair for an application that keeps no ids is to start keeping them.

The fix, as a flow

Server side conversation state looks like memory and behaves like a cache with a retention policy. Response objects are saved for 30 days by default, so a thread is exactly as durable as its oldest surviving link, and that is never the link you would think to test. The script walks each recorded chain upward, from the turn being served to the parent nobody has touched in a month.

Chain walked to a rootor to the first gapA parent does not resolvethe next turn will 404Oldest link near 30 daysdays of runway, not weeksStopped at the hop limitthe oldest link was never seenEvery link on a conversationitems persisted with no TTLRoot reached with runwaydurable for now, by the clock
The leaf always passes, so it is never the reading. Every outcome here is a statement about the far end of the chain.

How to fix it

Export the newest response id per thread

One id per line, from your own records. The newest id in each thread is the right starting point because the script walks upward from it; starting at a root tells you nothing about the chain hanging below it. Blank lines and # comments are ignored, so a file exported straight out of a query works.

Use a project key set to Read Only

Every call is GET /v1/responses/{response_id}. Nothing is created, nothing is deleted, and no completion is generated. An admin key is the wrong credential: stored responses are project data.

Walk each chain upward to a root or a gap

On a 200 the script reads created_at, previous_response_id and conversation, then follows the parent. It stops at a response with no parent, at the first 404, or at --max-hops, which exists so that a chain of a thousand turns does not become a thousand requests. A chain cut short by the hop limit is reported as unfinished rather than as healthy.

Read the runway off the oldest link, not the newest

The script reports the age of the oldest surviving link in each chain and the days remaining against the documented 30-day retention. --warn-days sets how close to the edge counts as a finding; five is a reasonable default because it is longer than a weekend.

Take the repair, which is a design change

Either move the thread onto a conversation object, whose items are persisted with no 30-day TTL, or keep the full message history in your own store and replay it. Before continuing an old thread, verify the parent resolves and fall back to replaying local history when it does not. The script prints this and changes nothing.

How to check it worked

Re-run with the same id file after the change. Chains that moved onto a conversation come back as conversation-backed; chains you rebuilt from local history come back as roots, because there is no longer a parent to walk to. The number that should keep falling on every run is the count of chains whose oldest link is inside the warning window, and that is the one worth putting on a schedule — weekly, against the threads that have not been touched in a fortnight.

python3 openai_response_chain_probe.py --ids thread-heads.txt --warn-days 5
# resp_c9  chain of 4, oldest resp_a1 at 2026-07-24T09:02:11Z, 38.2 days old
# chain-broken         resp_c9: the parent resp_a1 no longer resolves, so the
#                      next turn on this thread will 404
#   repair: fall back to replaying local history for this thread, and stop
#           chaining from an id you did not verify.
# resp_f2  chain of 3, oldest resp_d7 at 2026-08-05T14:40:03Z, 26.4 days old
# chain-expiring       resp_f2: the oldest link is 26.4 days old, so this chain
#                      has about 3.6 days of the documented 30 day retention left
# resp_k4  chain of 2, conversation conv_x1 on every link
# conversation-backed  resp_k4: items attached to a conversation are persisted
#                      with no 30 day TTL
# 3 chain(s) walked, 2 finding(s)

The full code

One GET per link and six pure functions. parse_ids, which reads a file exported by somebody in a hurry, ignoring blanks, comments and duplicates; link_row, which reduces a retrieved response to the four fields that matter and never invents a store flag, because the object does not carry one; age_days and runway_days, which take the clock as an argument so the whole retention calculation is testable without one; oldest_link, which is where the finding actually lives; and classify_chain, which grades a walked chain and treats a chain cut short by the hop limit as unfinished rather than as healthy.

Get this script on GitHub Follow @allanninal Python and Node.js, with tests. Read only, it never writes. One of 97 LLM API fixes, free and open source.
openai_response_chain_probe.py
"""Walk recorded previous_response_id chains and find the links already gone.

Read only. One GET of /v1/responses/{response_id} per link, and nothing else.
No completion is created, nothing is stored and nothing is deleted.

Response objects are saved for 30 days by default, so a chain is exactly as
durable as its oldest surviving link. This walks upward from the newest id you
recorded to a root or to a gap, and reports the runway from the oldest link
rather than from the newest.

The one documented exception is also the repair: a response attached to a
conversation has its items persisted with no 30 day TTL, so a conversation
backed chain keeps its history whatever happens to the response objects.

/v1/responses has no list endpoint, so the ids come from your own records and
every verdict is bounded by the chains you supply.
"""
import argparse
import logging
import os
import sys
import time

import requests

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("openai_response_chain_probe")

BASE_URL = "https://api.openai.com/v1/responses"

# The documented default: response objects are saved for 30 days.
RETENTION_DAYS = 30

FINDINGS = ("chain-broken", "chain-expiring", "chain-unreadable")


def parse_ids(text):
    """Response ids from a file. Pure. Blanks, comments and repeats dropped."""
    seen = []
    for line in str(text or "").splitlines():
        item = line.split("#", 1)[0].strip()
        if item and item not in seen:
            seen.append(item)
    return seen


def link_row(body):
    """One retrieved response, reduced. Pure. Four fields and no invention.

    There is deliberately no "stored" field here. The Response object does not
    carry a store flag, so the only honest evidence that a response was stored
    is that retrieving it worked, and that is recorded as the status code.
    """
    body = body if isinstance(body, dict) else {}
    conversation = body.get("conversation")
    if isinstance(conversation, dict):
        conversation = conversation.get("id")
    try:
        created = int(body.get("created_at") or 0)
    except (TypeError, ValueError):
        created = 0
    return {"id": str(body.get("id") or ""),
            "created_at": created,
            "previous_response_id": str(body.get("previous_response_id") or ""),
            "conversation": str(conversation or ""),
            "status": str(body.get("status") or "")}


def age_days(created_at, now):
    """Age of one link in days. Pure. The clock is an argument."""
    try:
        created = int(created_at)
        now = int(now)
    except (TypeError, ValueError):
        return None
    if created <= 0:
        return None
    return (now - created) / 86400.0


def oldest_link(chain):
    """The link that decides the chain. Pure. None for an empty chain."""
    usable = [row for row in chain or [] if int(row.get("created_at") or 0) > 0]
    if not usable:
        return None
    return min(usable, key=lambda row: int(row["created_at"]))


def runway_days(chain, now, retention=RETENTION_DAYS):
    """Days left on the oldest link. Pure. None when nothing is datable."""
    row = oldest_link(chain)
    if row is None:
        return None
    age = age_days(row["created_at"], now)
    if age is None:
        return None
    return retention - age


def classify_chain(head, chain, gap, unreadable, truncated, now, warn_days):
    """Grade one walked chain. Pure. Returns (state, detail)."""
    if unreadable:
        return ("chain-unreadable",
                "%s: %s, so nothing about this chain was established"
                % (head, unreadable))
    if gap:
        others = len(chain)
        if others:
            return ("chain-broken",
                    "%s: the parent %s no longer resolves, so the next turn on "
                    "this thread will 404" % (head, gap))
        return ("chain-broken",
                "%s: this id itself does not resolve. It has either aged out of "
                "the 30 day retention or was never stored" % head)
    if not chain:
        return ("nothing-walked", "%s: no links were read" % head)

    conversations = {row.get("conversation") for row in chain}
    if conversations and "" not in conversations:
        return ("conversation-backed",
                "%s: items attached to a conversation are persisted with no 30 "
                "day TTL" % head)

    left = runway_days(chain, now)
    if left is None:
        return ("undatable",
                "%s: no link carried a usable created_at, so the runway cannot "
                "be computed" % head)
    if left <= 0:
        return ("chain-broken",
                "%s: the oldest link is past the documented %d day retention "
                "and is only resolving on borrowed time"
                % (head, RETENTION_DAYS))
    if left <= warn_days:
        row = oldest_link(chain)
        return ("chain-expiring",
                "%s: the oldest link is %.1f days old, so this chain has about "
                "%.1f days of the documented %d day retention left"
                % (head, age_days(row["created_at"], now), left, RETENTION_DAYS))
    if truncated:
        return ("chain-unfinished",
                "%s: stopped at the hop limit before reaching a root, so the "
                "oldest link was never seen" % head)
    return ("chain-intact",
            "%s: walked to a root with %.1f days left on the oldest link"
            % (head, left))


def repair_lines(state):
    """The repair for one verdict. Pure. Printed, never performed."""
    move = ("move this thread onto a conversation object, whose items are "
            "persisted with no 30 day TTL, or keep the full message history in "
            "your own store and replay it.")
    if state == "chain-broken":
        return ["fall back to replaying local history for this thread, and stop "
                "chaining from an id you did not verify.", move]
    if state == "chain-expiring":
        return [move,
                "until then, verify the parent resolves before continuing an "
                "old thread rather than discovering it inside a user request."]
    if state == "chain-unreadable":
        return ["the key could not read this response. Check that it belongs to "
                "the project that created it before concluding anything about "
                "retention."]
    if state == "chain-unfinished":
        return ["raise --max-hops for this thread. A chain graded without "
                "reaching its oldest link has not been graded."]
    if state == "undatable":
        return ["the links resolved but carried no created_at, which is odd "
                "enough to read one of them by hand before trusting the rest."]
    return []


def retrieve(response_id, key, timeout=30):
    """One GET. Returns (status, body). A 404 is the answer, not an error."""
    try:
        r = requests.get("%s/%s" % (BASE_URL, response_id),
                         headers={"Authorization": "Bearer " + key},
                         timeout=timeout)
    except requests.RequestException as exc:
        log.debug("retrieve of %s failed: %s", response_id, exc)
        return (None, None)
    try:
        return (r.status_code, r.json())
    except ValueError:
        return (r.status_code, None)


def walk(head, key, max_hops):
    """Follow one chain upward. Returns (chain, gap, unreadable, truncated)."""
    chain = []
    current = head
    for _ in range(max_hops):
        status, body = retrieve(current, key)
        if status == 404:
            return (chain, current, "", False)
        if status in (401, 403):
            return (chain, "", "HTTP %d reading %s" % (status, current), False)
        if status != 200:
            return (chain, "", "HTTP %s reading %s" % (status, current), False)
        row = link_row(body)
        chain.append(row)
        if not row["previous_response_id"]:
            return (chain, "", "", False)
        current = row["previous_response_id"]
    return (chain, "", "", True)


def main():
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("--ids", required=True,
                    help="file of response ids, newest per thread, one per line")
    ap.add_argument("--max-hops", type=int, default=20,
                    help="how far up one chain to walk")
    ap.add_argument("--warn-days", type=float, default=5.0,
                    help="days of remaining retention that count as a finding")
    args = ap.parse_args()

    key = os.environ.get("OPENAI_API_KEY")
    if not key:
        log.error("set OPENAI_API_KEY to a project key set to Read Only. Every "
                  "call is a GET of /v1/responses/{response_id}")
        return 2
    try:
        with open(args.ids, "r", encoding="utf-8") as fh:
            heads = parse_ids(fh.read())
    except OSError as exc:
        log.error("could not read %s: %s", args.ids, exc)
        return 2
    if not heads:
        log.error("no response ids in %s. /v1/responses cannot be listed, so "
                  "the chains have to start from ids you recorded", args.ids)
        return 2

    now = int(time.time())
    findings = 0
    for head in heads:
        chain, gap, unreadable, truncated = walk(head, key, args.max_hops)
        row = oldest_link(chain)
        if row:
            age = age_days(row["created_at"], now)
            log.info("%-10s chain of %d, oldest %s, %.1f days old",
                     head, len(chain), row["id"], age or 0.0)
        else:
            log.info("%-10s chain of %d", head, len(chain))

        state, detail = classify_chain(head, chain, gap, unreadable, truncated,
                                       now, args.warn_days)
        emit = log.warning if state in FINDINGS else log.info
        emit("%-20s %s", state, detail)
        for line in repair_lines(state):
            emit("  repair: %s", line)
        if state in FINDINGS:
            findings += 1

    log.info("%d chain(s) walked, %d finding(s)", len(heads), findings)
    return 1 if findings else 0


if __name__ == "__main__":
    sys.exit(main())
openai-response-chain-probe.mjs
/**
 * Walk recorded previous_response_id chains and find the links already gone.
 *
 * Read only. One GET of /v1/responses/{response_id} per link. No completion is
 * created, nothing is stored and nothing is deleted.
 *
 * Response objects are saved for 30 days by default, so a chain is only as
 * durable as its oldest surviving link. A response attached to a conversation
 * has its items persisted with no 30 day TTL, which is the repair.
 *
 * /v1/responses has no list endpoint, so the ids come from your own records.
 */
import { readFile } from 'node:fs/promises';

const BASE_URL = 'https://api.openai.com/v1/responses';

export const RETENTION_DAYS = 30;

const FINDINGS = new Set(['chain-broken', 'chain-expiring', 'chain-unreadable']);

/** Response ids from a file. Pure. Blanks, comments and repeats dropped. */
export function parseIds(text) {
  const seen = [];
  for (const line of String(text ?? '').split('\n')) {
    const item = line.split('#')[0].trim();
    if (item && !seen.includes(item)) seen.push(item);
  }
  return seen;
}

/** One retrieved response, reduced. Pure. Four fields and no invention. */
export function linkRow(body) {
  const row = (body && typeof body === 'object') ? body : {};
  const conversation = (row.conversation && typeof row.conversation === 'object')
    ? row.conversation.id : row.conversation;
  const created = Number(row.created_at ?? 0);
  return {
    id: String(row.id ?? ''),
    created_at: Number.isFinite(created) ? Math.trunc(created) : 0,
    previous_response_id: String(row.previous_response_id ?? ''),
    conversation: String(conversation ?? ''),
    status: String(row.status ?? ''),
  };
}

/** Age of one link in days. Pure. The clock is an argument. */
export function ageDays(createdAt, now) {
  const created = Number(createdAt);
  const at = Number(now);
  if (!Number.isFinite(created) || !Number.isFinite(at) || created <= 0) return null;
  return (at - created) / 86400;
}

/** The link that decides the chain. Pure. Null for an empty chain. */
export function oldestLink(chain) {
  const usable = (chain ?? []).filter((row) => Number(row?.created_at ?? 0) > 0);
  if (!usable.length) return null;
  return usable.reduce((a, b) => (Number(a.created_at) <= Number(b.created_at) ? a : b));
}

/** Days left on the oldest link. Pure. Null when nothing is datable. */
export function runwayDays(chain, now, retention = RETENTION_DAYS) {
  const row = oldestLink(chain);
  if (!row) return null;
  const age = ageDays(row.created_at, now);
  return age === null ? null : retention - age;
}

/** Grade one walked chain. Pure. Returns [state, detail]. */
export function classifyChain(head, chain, gap, unreadable, truncated, now, warnDays) {
  if (unreadable) {
    return ['chain-unreadable',
      `${head}: ${unreadable}, so nothing about this chain was established`];
  }
  if (gap) {
    if ((chain ?? []).length) {
      return ['chain-broken',
        `${head}: the parent ${gap} no longer resolves, so the next turn on this `
        + 'thread will 404'];
    }
    return ['chain-broken',
      `${head}: this id itself does not resolve. It has either aged out of the `
      + '30 day retention or was never stored'];
  }
  if (!(chain ?? []).length) return ['nothing-walked', `${head}: no links were read`];

  const conversations = new Set(chain.map((row) => row.conversation ?? ''));
  if (!conversations.has('')) {
    return ['conversation-backed',
      `${head}: items attached to a conversation are persisted with no 30 day TTL`];
  }

  const left = runwayDays(chain, now);
  if (left === null) {
    return ['undatable',
      `${head}: no link carried a usable created_at, so the runway cannot be computed`];
  }
  if (left <= 0) {
    return ['chain-broken',
      `${head}: the oldest link is past the documented ${RETENTION_DAYS} day `
      + 'retention and is only resolving on borrowed time'];
  }
  if (left <= warnDays) {
    const row = oldestLink(chain);
    return ['chain-expiring',
      `${head}: the oldest link is ${ageDays(row.created_at, now).toFixed(1)} days `
      + `old, so this chain has about ${left.toFixed(1)} days of the documented `
      + `${RETENTION_DAYS} day retention left`];
  }
  if (truncated) {
    return ['chain-unfinished',
      `${head}: stopped at the hop limit before reaching a root, so the oldest `
      + 'link was never seen'];
  }
  return ['chain-intact',
    `${head}: walked to a root with ${left.toFixed(1)} days left on the oldest link`];
}

/** The repair for one verdict. Pure. Printed, never performed. */
export function repairLines(state) {
  const move = 'move this thread onto a conversation object, whose items are '
    + 'persisted with no 30 day TTL, or keep the full message history in your '
    + 'own store and replay it.';
  if (state === 'chain-broken') {
    return ['fall back to replaying local history for this thread, and stop '
      + 'chaining from an id you did not verify.', move];
  }
  if (state === 'chain-expiring') {
    return [move, 'until then, verify the parent resolves before continuing an '
      + 'old thread rather than discovering it inside a user request.'];
  }
  if (state === 'chain-unreadable') {
    return ['the key could not read this response. Check that it belongs to the '
      + 'project that created it before concluding anything about retention.'];
  }
  if (state === 'chain-unfinished') {
    return ['raise --max-hops for this thread. A chain graded without reaching '
      + 'its oldest link has not been graded.'];
  }
  if (state === 'undatable') {
    return ['the links resolved but carried no created_at, which is odd enough '
      + 'to read one of them by hand before trusting the rest.'];
  }
  return [];
}

async function retrieve(responseId, key) {
  try {
    const res = await fetch(`${BASE_URL}/${responseId}`,
      { headers: { Authorization: `Bearer ${key}` } });
    let body = null;
    try { body = await res.json(); } catch { body = null; }
    return [res.status, body];
  } catch {
    return [null, null];
  }
}

async function walk(head, key, maxHops) {
  const chain = [];
  let current = head;
  for (let hop = 0; hop < maxHops; hop += 1) {
    const [status, body] = await retrieve(current, key);
    if (status === 404) return [chain, current, '', false];
    if (status !== 200) return [chain, '', `HTTP ${status} reading ${current}`, false];
    const row = linkRow(body);
    chain.push(row);
    if (!row.previous_response_id) return [chain, '', '', false];
    current = row.previous_response_id;
  }
  return [chain, '', '', true];
}

function args(argv) {
  const out = { maxHops: 20, warnDays: 5 };
  for (let i = 0; i < argv.length; i += 1) {
    if (argv[i] === '--ids') out.ids = argv[i += 1];
    else if (argv[i] === '--max-hops') out.maxHops = Number.parseInt(argv[i += 1], 10);
    else if (argv[i] === '--warn-days') out.warnDays = Number(argv[i += 1]);
  }
  return out;
}

async function main() {
  const opts = args(process.argv.slice(2));
  const key = process.env.OPENAI_API_KEY;
  if (!key) {
    console.error('set OPENAI_API_KEY to a project key set to Read Only. Every '
      + 'call is a GET of /v1/responses/{response_id}');
    process.exitCode = 2;
    return;
  }
  if (!opts.ids) {
    console.error('usage: --ids <file> [--max-hops 20] [--warn-days 5]');
    process.exitCode = 2;
    return;
  }
  let heads;
  try {
    heads = parseIds(await readFile(opts.ids, 'utf8'));
  } catch (err) {
    console.error(`could not read ${opts.ids}: ${err.message}`);
    process.exitCode = 2;
    return;
  }
  if (!heads.length) {
    console.error(`no response ids in ${opts.ids}. /v1/responses cannot be `
      + 'listed, so the chains have to start from ids you recorded');
    process.exitCode = 2;
    return;
  }

  const now = Math.trunc(Date.now() / 1000);
  let findings = 0;
  for (const head of heads) {
    const [chain, gap, unreadable, truncated] = await walk(head, key, opts.maxHops);
    const row = oldestLink(chain);
    if (row) {
      console.log(`${head.padEnd(10)} chain of ${chain.length}, oldest ${row.id}, `
        + `${(ageDays(row.created_at, now) ?? 0).toFixed(1)} days old`);
    } else {
      console.log(`${head.padEnd(10)} chain of ${chain.length}`);
    }
    const [state, detail] = classifyChain(head, chain, gap, unreadable, truncated,
                                          now, opts.warnDays);
    console.log(`${state.padEnd(20)} ${detail}`);
    for (const line of repairLines(state)) console.log(`  repair: ${line}`);
    if (FINDINGS.has(state)) findings += 1;
  }
  console.log(`${heads.length} chain(s) walked, ${findings} finding(s)`);
  process.exitCode = findings ? 1 : 0;
}

if (import.meta.url === `file://${process.argv[1]}`) await main();

Add a test

The first test is the failure the note is named after: a chain whose parent 404s comes back as chain-broken, names the missing parent, and says that the next turn will fail rather than that a request failed. The second is the one that decides whether the script is useful at all — the runway has to be computed from the oldest link, so a chain with a fresh leaf and a month-old root is a finding, and a test asserts that reading the newest link instead would have called it healthy. The third is the documented exception: every link carrying a conversation is not this note's problem, and mixed chains do not qualify for it. Then the hop limit, which must produce chain-unfinished rather than a clean bill on a chain whose oldest link was never seen. Then link_row, asserted to invent no store field, because the object does not have one. And last the id file, read the way a file exported in a hurry actually looks.

test_openai_response_chain_probe.py
from openai_response_chain_probe import (RETENTION_DAYS, age_days,
                                        classify_chain, link_row, oldest_link,
                                        parse_ids, repair_lines, runway_days)

NOW = 1_800_000_000
DAY = 86400


def link(rid, days_old, parent="", conversation=""):
    return link_row({"id": rid, "created_at": NOW - int(days_old * DAY),
                     "previous_response_id": parent,
                     "conversation": ({"id": conversation} if conversation
                                      else None),
                     "status": "completed"})


def test_a_missing_parent_is_the_finding_and_names_the_next_turn():
    chain = [link("resp_c9", 1.0, parent="resp_a1")]
    state, detail = classify_chain("resp_c9", chain, "resp_a1", "", False,
                                   NOW, 5.0)
    assert state == "chain-broken"
    assert "resp_a1 no longer resolves" in detail
    assert "next turn on this thread will 404" in detail
    lines = repair_lines(state)
    assert any("replaying local history" in line for line in lines)
    assert any("no 30 day TTL" in line for line in lines)

    # The head itself missing is the same verdict with a different sentence,
    # because a 404 there has two causes and the script names both.
    state, detail = classify_chain("resp_c9", [], "resp_c9", "", False, NOW, 5.0)
    assert state == "chain-broken"
    assert "aged out" in detail and "never stored" in detail


def test_the_runway_comes_from_the_oldest_link_and_not_the_newest():
    chain = [link("resp_f2", 0.5, parent="resp_e1"),
             link("resp_e1", 12.0, parent="resp_d7"),
             link("resp_d7", 26.4)]
    assert oldest_link(chain)["id"] == "resp_d7"
    assert abs(runway_days(chain, NOW) - (RETENTION_DAYS - 26.4)) < 0.01
    state, detail = classify_chain("resp_f2", chain, "", "", False, NOW, 5.0)
    assert state == "chain-expiring"
    assert "26.4 days old" in detail and "3.6 days" in detail
    # Read from the newest link this chain looks like it has 29.5 days left.
    assert age_days(chain[0]["created_at"], NOW) < 1.0


def test_a_conversation_backed_chain_is_not_this_note():
    chain = [link("resp_k4", 1.0, parent="resp_k3", conversation="conv_x1"),
             link("resp_k3", 44.0, conversation="conv_x1")]
    state, detail = classify_chain("resp_k4", chain, "", "", False, NOW, 5.0)
    assert state == "conversation-backed"
    assert "no 30 day TTL" in detail
    assert repair_lines(state) == []
    # One link without a conversation is not a conversation backed chain.
    mixed = [chain[0], link("resp_k3", 44.0)]
    assert classify_chain("resp_k4", mixed, "", "", False, NOW, 5.0)[0] \
        == "chain-broken"


def test_a_chain_cut_short_by_the_hop_limit_is_not_graded_healthy():
    chain = [link("resp_z9", 1.0, parent="resp_z8"),
             link("resp_z8", 2.0, parent="resp_z7")]
    state, detail = classify_chain("resp_z9", chain, "", "", True, NOW, 5.0)
    assert state == "chain-unfinished"
    assert "oldest link was never seen" in detail
    assert any("--max-hops" in line for line in repair_lines(state))
    # The same chain walked to a root is intact.
    rooted = [chain[0], link("resp_z8", 2.0)]
    assert classify_chain("resp_z9", rooted, "", "", False, NOW, 5.0)[0] \
        == "chain-intact"


def test_link_row_reads_four_fields_and_invents_no_store_flag():
    row = link_row({"id": "resp_a1", "created_at": 1700000000,
                    "previous_response_id": None,
                    "conversation": {"id": "conv_x1"}, "status": "completed"})
    assert row == {"id": "resp_a1", "created_at": 1700000000,
                   "previous_response_id": "", "conversation": "conv_x1",
                   "status": "completed"}
    assert "store" not in row and "stored" not in row
    assert link_row(None)["id"] == ""
    assert link_row({"created_at": "nonsense"})["created_at"] == 0
    assert age_days(0, NOW) is None and runway_days([], NOW) is None


def test_the_id_file_is_read_the_way_it_is_actually_exported():
    ids = parse_ids("resp_a1\n\n# heads exported 2026-08-30\nresp_b2  # oldest\n"
                    "resp_a1\n   \nresp_c3\n")
    assert ids == ["resp_a1", "resp_b2", "resp_c3"]
    assert parse_ids("") == [] and parse_ids(None) == []
    state, detail = classify_chain("resp_a1", [], "", "HTTP 403 reading resp_a1",
                                   False, NOW, 5.0)
    assert state == "chain-unreadable"
    assert "nothing about this chain was established" in detail
openai-response-chain-probe.test.mjs
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { RETENTION_DAYS, ageDays, classifyChain, linkRow, oldestLink, parseIds,
         repairLines, runwayDays } from './openai-response-chain-probe.mjs';

const NOW = 1_800_000_000;
const DAY = 86400;

const link = (id, daysOld, parent = '', conversation = '') => linkRow({
  id,
  created_at: NOW - Math.trunc(daysOld * DAY),
  previous_response_id: parent,
  conversation: conversation ? { id: conversation } : null,
  status: 'completed',
});

test('a missing parent is the finding and names the next turn', () => {
  const chain = [link('resp_c9', 1, 'resp_a1')];
  const [state, detail] = classifyChain('resp_c9', chain, 'resp_a1', '', false, NOW, 5);
  assert.equal(state, 'chain-broken');
  assert.ok(detail.includes('resp_a1 no longer resolves'));
  assert.ok(detail.includes('next turn on this thread will 404'));
  const lines = repairLines(state);
  assert.ok(lines.some((l) => l.includes('replaying local history')));
  assert.ok(lines.some((l) => l.includes('no 30 day TTL')));

  const [headState, headDetail] = classifyChain('resp_c9', [], 'resp_c9', '', false, NOW, 5);
  assert.equal(headState, 'chain-broken');
  assert.ok(headDetail.includes('aged out') && headDetail.includes('never stored'));
});

test('the runway comes from the oldest link and not the newest', () => {
  const chain = [link('resp_f2', 0.5, 'resp_e1'),
                 link('resp_e1', 12, 'resp_d7'),
                 link('resp_d7', 26.4)];
  assert.equal(oldestLink(chain).id, 'resp_d7');
  assert.ok(Math.abs(runwayDays(chain, NOW) - (RETENTION_DAYS - 26.4)) < 0.01);
  const [state, detail] = classifyChain('resp_f2', chain, '', '', false, NOW, 5);
  assert.equal(state, 'chain-expiring');
  assert.ok(detail.includes('26.4 days old') && detail.includes('3.6 days'));
  assert.ok(ageDays(chain[0].created_at, NOW) < 1);
});

test('a conversation backed chain is not this note', () => {
  const chain = [link('resp_k4', 1, 'resp_k3', 'conv_x1'),
                 link('resp_k3', 44, '', 'conv_x1')];
  const [state, detail] = classifyChain('resp_k4', chain, '', '', false, NOW, 5);
  assert.equal(state, 'conversation-backed');
  assert.ok(detail.includes('no 30 day TTL'));
  assert.deepEqual(repairLines(state), []);
  const mixed = [chain[0], link('resp_k3', 44)];
  assert.equal(classifyChain('resp_k4', mixed, '', '', false, NOW, 5)[0], 'chain-broken');
});

test('a chain cut short by the hop limit is not graded healthy', () => {
  const chain = [link('resp_z9', 1, 'resp_z8'), link('resp_z8', 2, 'resp_z7')];
  const [state, detail] = classifyChain('resp_z9', chain, '', '', true, NOW, 5);
  assert.equal(state, 'chain-unfinished');
  assert.ok(detail.includes('oldest link was never seen'));
  assert.ok(repairLines(state).some((l) => l.includes('--max-hops')));
  const rooted = [chain[0], link('resp_z8', 2)];
  assert.equal(classifyChain('resp_z9', rooted, '', '', false, NOW, 5)[0], 'chain-intact');
});

test('linkRow reads four fields and invents no store flag', () => {
  const row = linkRow({ id: 'resp_a1', created_at: 1700000000,
                        previous_response_id: null,
                        conversation: { id: 'conv_x1' }, status: 'completed' });
  assert.deepEqual(row, { id: 'resp_a1', created_at: 1700000000,
                          previous_response_id: '', conversation: 'conv_x1',
                          status: 'completed' });
  assert.ok(!('store' in row) && !('stored' in row));
  assert.equal(linkRow(null).id, '');
  assert.equal(linkRow({ created_at: 'nonsense' }).created_at, 0);
  assert.equal(ageDays(0, NOW), null);
  assert.equal(runwayDays([], NOW), null);
});

test('the id file is read the way it is actually exported', () => {
  const ids = parseIds('resp_a1\n\n# heads exported 2026-08-30\nresp_b2  # oldest\n'
    + 'resp_a1\n   \nresp_c3\n');
  assert.deepEqual(ids, ['resp_a1', 'resp_b2', 'resp_c3']);
  assert.deepEqual(parseIds(''), []);
  assert.deepEqual(parseIds(null), []);
  const [state, detail] = classifyChain('resp_a1', [], '', 'HTTP 403 reading resp_a1',
                                        false, NOW, 5);
  assert.equal(state, 'chain-unreadable');
  assert.ok(detail.includes('nothing about this chain was established'));
});

FAQ

How long do stored responses actually last?

Response objects are saved for 30 days by default. That is the number the script computes against, and it is the reason a thread that a customer returns to after a month is the one that breaks. The exception is documented and specific: any response attached to a conversation has its items persisted with no 30 day TTL. So the durability of a thread depends on which object you built it out of, not on how you feel about it.

Why walk the whole chain instead of checking the id we are about to use?

Because the id you are about to use is the newest one, and it always passes. The request that fails is the one that resolves your parent, and its parent, back to the beginning of a conversation that started weeks ago. A chain is exactly as durable as its oldest surviving link, so a script that checked the leaf would report perfect health on a thread that is four days from breaking. That is the specific mistake this note exists to prevent, and there is a test that fails if the calculation ever moves to the newest link.

Can we just list our stored responses and audit all of them?

No, and this is a real gap rather than an omission here. /v1/responses has no list endpoint: stored responses are reachable only by an id you already hold. So the chains have to come from your own records, and every verdict is bounded by the ids you supplied, which is why the script prints how many chains it walked next to how many findings it produced. If you are not recording response ids, that is the first repair and it costs a column.

Is a 404 always retention? It could be a delete, or a call that was never stored.

It could be any of the three and the script says so instead of guessing. A 404 on the head id gets a sentence naming both possibilities, aged out or never stored. A 404 on a parent inside a chain whose other links resolve is stronger evidence of the clock, because storage was clearly working for that thread. Deletion looks identical from outside; if you run a cleanup sweep over responses, it is doing the same damage as the retention window, only faster.

Should we move to conversations, or keep our own history?

Both are correct and they solve different problems. A conversation is an explicit, deletable object whose items are persisted with no 30 day TTL, which fixes the expiry without you building anything. Keeping the full message history in your own store fixes it as well and additionally makes your product portable, testable and replayable, at the cost of assembling every turn yourself. What is not an option is the third thing, which is chaining from an id nobody verified and finding out inside a user request.

Related field notes

Sources

Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.

Stuck on a tricky one?

If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.