Diagnostic LLM APIs
Claude Code edits rejected more often than they are kept
The diff comes up, it is forty lines, it is confidently wrong about where the validation lives, and it gets rejected in about two seconds. That happens eleven times before lunch, and none of those eleven rejections is recorded anywhere a person would look, because rejecting a proposal is not an error and not a failure and not, from the tool's point of view, an event worth complaining about. Every one of those forty-line diffs was generated at frontier output rates and paid for in full before anybody read it.
The same report as the cache note, a different block of it. GET /v1/organizations/usage_report/claude_code?starting_at=YYYY-MM-DD&limit=1000 with an Admin API key returns tool_actions per record, with accepted and rejected counts for edit_tool, multi_edit_tool, write_tool and notebook_edit_tool.
The rate is directly computable: accepted / (accepted + rejected). Below one half means a majority of the diffs an actor was shown got thrown away. Pair it with model_breakdown[].estimated_cost.amount and with core_metrics.commits_by_claude_code, pull_requests_by_claude_code and lines_of_code, because a low rate next to real commits is a different story from a low rate next to nothing.
This is the only measurement in this section whose subject is billed output that a human read and discarded. Every other note here is about tokens spent on machinery, on retries, on prefixes or on capacity. This one is about generation that completed successfully, cost full output rates, and was then declined by the person it was produced for.
The cause is almost never the tool. A sustained low acceptance rate points at missing project context, task framing that is too broad, or a model and effort level mismatched to the work — and the API cannot tell you which, so the script names the three and stops.
The problem in plain words
Rejection is a healthy operation. Somebody read a proposed change, decided it was wrong, and did not apply it, which is the review step working exactly as intended. Nothing about a single rejection is a problem, and that is precisely why a sustained one is invisible: there is no threshold at which the tool starts objecting, no error, and no aggregate anyone watches.
The money is already spent by then. A rejected edit was fully generated before it was displayed, so the output tokens are billed identically whether the diff is applied or dismissed with a keystroke. At frontier output rates, on multi-file edits, an actor running below fifty per cent acceptance is spending the majority of their output budget on work that reaches nobody.
And the loss compounds past the invoice. A rejected proposal costs the developer the time to read it and the context switch to recover from it, and it usually costs another turn to try again, which is another full generation. The cost per accepted change is the number that matters and it is not on any dashboard, because no dashboard joins these two fields.
Why it happens
This is the sibling of the cache note, not a variation of it. They read the same endpoint and they do not merge, because they read different blocks and answer different questions. The cache note reads model_breakdown[].tokens and asks whether a prefix is being repurchased; it is about money spent twice on the same input. This reads tool_actions and asks whether generated output survived contact with a person; it is about money spent once, correctly, on something nobody wanted. One is a caching repair and the other is a project-setup repair, and neither of them is a substitute for reading the other.
A low rate on a handful of proposals is noise, and the floor is the difference between an audit and an accusation. Three rejections out of four is a bad afternoon. Three hundred out of five hundred is a pattern. The script sets a proposal floor before it grades anything, and an actor under it comes back as insufficient evidence rather than as a finding, because this measurement is attached to a named person and getting it wrong is not a neutral event.
The cost and the rate go side by side and are deliberately never multiplied. It is tempting to report "sixty per cent of this developer's spend was wasted", and it would be false: the report gives no per-proposal token counts, so there is no way to know whether the rejected diffs were the large ones or the small ones. The output prints the acceptance rate and the estimated cost as two separate readings and states that the product of them is not a number this API can support.
Per-tool rates say more than a single average. edit_tool, multi_edit_tool, write_tool and notebook_edit_tool are counted separately, and they fail differently: a bad multi_edit_tool rate usually means a task that was scoped too wide, while a bad write_tool rate often means whole files being generated where an edit was wanted. Averaging them hides which of those is happening.
The API will never tell you why, and pretending otherwise would be the failure mode of this note. There is no reason code on a rejection, no diff content, no prompt. What the report supports is naming the actors and the tools, printing the rate beside the volume and the commits, and handing that to the team that owns the repository. The repair — a project context file, a narrower task, a different model or effort level — is a judgement made by people who can see the code.
The fix, as a flow
The sibling of the note above, on the same endpoint and a different block of it, and the only measurement in this section whose subject is billed output that a person deliberately threw away. Rejecting a bad diff is the review step working. A sustained majority of them is an output budget going to work that reaches nobody, and there is no threshold at which anything starts objecting.
How to fix it
Read the same report, one UTC day at a time
GET /v1/organizations/usage_report/claude_code?starting_at=YYYY-MM-DD&limit=1000, paged on next_page, once per day. Records arrive with up to an hour's delay, so the window ends yesterday.
Sum tool_actions per actor and per tool
Keep the four edit-producing tools apart: edit_tool, multi_edit_tool, write_tool, notebook_edit_tool. Sum accepted and rejected across days, per tool, per actor.
Apply the proposal floor before grading anyone
An actor with fewer proposals than the floor is reported as insufficient evidence. This number sits next to a person's name, and a bad afternoon must not read as a pattern.
Compute the rate, and put the productivity metrics beside it
accepted / (accepted + rejected), per tool and overall. Then commits_by_claude_code, pull_requests_by_claude_code and lines_of_code, because a low rate that still lands commits is a different conversation from a low rate that lands nothing.
Print the actors, the worst tool, and the cost as a separate reading
The rate and the estimated cost are printed side by side and never multiplied, because the report carries no per-proposal token counts. The suggestion is to review project setup for those repositories, and it goes to the team, not into a change.
How to check it worked
Re-run a fortnight after a project context file lands or a task pattern changes. The acceptance rate should move on the same volume of proposals. If the proposal count collapsed instead, the change was people using the tool less, which is not the improvement it looks like on this chart.
python3 claude_code_edit_acceptance.py --days 14
# rejected-more-than-kept d***@example.com 38% accepted over 412 proposal(s); worst tool multi_edit_tool at 21%, $310.40
# repair: a majority of proposals are being discarded. Review project setup: a CLAUDE.md context file, and narrower task scoping.
# low-acceptance e***@example.com 61% accepted over 208 proposal(s); worst tool write_tool at 44%, $96.10
# too-few-proposals f***@example.com 9 proposal(s), under the floor of 20
# healthy g***@example.com 88% accepted over 340 proposal(s), 26 commit(s)
# 4 actor(s) over 14 day(s), 2 finding(s)
# the rate and the cost are separate readings: no per-proposal token counts exist to join themThe full code
One GET per day and eight pure functions, none of which is shared with the cache note even though both read this endpoint. The actor reader and the mask; the tool_actions reader, which keeps the four tools apart; the fold, which also carries the commits, pull requests and lines so a rate is never read alone; the acceptance calculation, which returns None rather than zero when nobody proposed anything; the worst-tool picker with its own volume floor; the verdict with the proposal floor; and the repair lines, which suggest a conversation rather than a change. There is no function that multiplies a rate by a cost, and the absence is the point.
"""Find Claude Code actors whose edit proposals are mostly rejected.
Read only. One paged GET per UTC day against the Claude Code usage report with
an Admin API key. No message is sent and nothing is written.
Every rejected proposal was fully generated and fully billed before it was
displayed, so this is the one audit in the set whose subject is output that
succeeded, cost full rates, and was then thrown away by the person it was
produced for.
The acceptance rate and the estimated cost are printed as two separate
readings and are never multiplied. The report carries no per-proposal token
counts, so there is no way to know whether the rejected diffs were the large
ones or the small ones, and "60% of the spend was wasted" would be a sentence
this API cannot support.
The API never says why a proposal was rejected. The repair is a conversation
with the team that owns the repository, so it is printed rather than performed.
"""
import argparse
import datetime as dt
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("claude_code_edit_acceptance")
API = "https://api.anthropic.com/v1"
VERSION = "2023-06-01"
# The tools that propose a change a person then keeps or discards. Counted
# apart rather than averaged: a bad multi_edit rate usually means a task scoped
# too wide, and a bad write rate usually means whole files where an edit was
# wanted, and one number cannot say either.
EDIT_TOOLS = ("edit_tool", "multi_edit_tool", "write_tool", "notebook_edit_tool")
FINDINGS = ("rejected-more-than-kept", "low-acceptance")
def actor_name(record):
"""Who the record belongs to. Pure. Both actor shapes, plus neither."""
actor = (record or {}).get("actor")
if not isinstance(actor, dict):
return "unattributed"
for field in ("email_address", "api_key_name"):
value = str(actor.get(field) or "").strip()
if value:
return value
return "unattributed"
def mask(name):
"""Hide the local part of an email address. Pure. Non-emails pass through.
This output attaches a quality number to a named person. Masked by default
is the only sensible default for that.
"""
text = str(name or "").strip()
if "@" not in text:
return text or "unattributed"
local, _, domain = text.partition("@")
if not local:
return text
return local[0] + "***@" + domain
def actions_of(record):
"""Accepted and rejected counts per edit tool on one record. Pure.
Tools absent from the record are omitted rather than zeroed, because a tool
nobody used and a tool used with nothing accepted must not look alike.
"""
actions = (record or {}).get("tool_actions")
actions = actions if isinstance(actions, dict) else {}
out = {}
for tool in EDIT_TOOLS:
row = actions.get(tool)
if not isinstance(row, dict):
continue
counts = {}
for field in ("accepted", "rejected"):
try:
counts[field] = max(0, int(row.get(field) or 0))
except (TypeError, ValueError):
counts[field] = 0
if counts["accepted"] or counts["rejected"]:
out[tool] = counts
return out
def fold(pages):
"""Fold every record into one row per actor. Pure.
The productivity fields travel with the rate on purpose. A low acceptance
rate beside twenty-six commits is a different conversation from a low rate
beside none, and separating them invites the wrong one.
"""
rows = {}
for page in pages or []:
for record in (page or {}).get("data") or []:
if not isinstance(record, dict):
continue
who = actor_name(record)
row = rows.setdefault(who, {
"tools": {}, "days": 0, "sessions": 0, "commits": 0, "prs": 0,
"added": 0, "removed": 0, "cents": 0.0})
row["days"] += 1
for tool, counts in actions_of(record).items():
into = row["tools"].setdefault(tool, {"accepted": 0, "rejected": 0})
into["accepted"] += counts["accepted"]
into["rejected"] += counts["rejected"]
core = record.get("core_metrics")
core = core if isinstance(core, dict) else {}
for field, key in (("num_sessions", "sessions"),
("commits_by_claude_code", "commits"),
("pull_requests_by_claude_code", "prs")):
try:
row[key] += max(0, int(core.get(field) or 0))
except (TypeError, ValueError):
pass
lines = core.get("lines_of_code")
lines = lines if isinstance(lines, dict) else {}
for field, key in (("added", "added"), ("removed", "removed")):
try:
row[key] += max(0, int(lines.get(field) or 0))
except (TypeError, ValueError):
pass
for entry in record.get("model_breakdown") or []:
cost = (entry or {}).get("estimated_cost")
cost = cost if isinstance(cost, dict) else {}
try:
row["cents"] += float(cost.get("amount") or 0)
except (TypeError, ValueError):
pass
return rows
def totals(row):
"""Accepted and rejected across every edit tool for one actor. Pure."""
accepted = 0
rejected = 0
for counts in ((row or {}).get("tools") or {}).values():
accepted += max(0, int((counts or {}).get("accepted") or 0))
rejected += max(0, int((counts or {}).get("rejected") or 0))
return accepted, rejected
def acceptance(counts):
"""accepted / (accepted + rejected). Pure. None when nothing was proposed.
None rather than 0.0. An actor who proposed nothing has no acceptance rate,
and reporting one as zero would put them at the top of a list of the worst.
"""
data = counts or {}
accepted = max(0, int(data.get("accepted") or 0))
rejected = max(0, int(data.get("rejected") or 0))
total = accepted + rejected
if total <= 0:
return None
return accepted / float(total)
def worst_tool(row, min_proposals=10):
"""The lowest-scoring tool with enough volume to mean it. Pure.
Returns (tool, rate) or None. The per-tool floor is separate from and lower
than the actor floor, because one tool is a slice of an actor's traffic.
"""
worst = None
for tool, counts in sorted(((row or {}).get("tools") or {}).items()):
total = (max(0, int((counts or {}).get("accepted") or 0))
+ max(0, int((counts or {}).get("rejected") or 0)))
if total < min_proposals:
continue
rate = acceptance(counts)
if rate is None:
continue
if worst is None or rate < worst[1]:
worst = (tool, rate)
return worst
def verdict(row, min_proposals=20, keep_floor=0.50, thin=0.70):
"""Classify one actor's acceptance. Pure. Returns (state, detail)."""
accepted, rejected = totals(row)
total = accepted + rejected
if total < min_proposals:
return ("too-few-proposals",
"%d proposal(s), under the floor of %d: a bad afternoon is not "
"a pattern" % (total, min_proposals))
rate = acceptance({"accepted": accepted, "rejected": rejected})
worst = worst_tool(row)
tail = ""
if worst is not None:
tail = "; worst tool %s at %.0f%%" % (worst[0], worst[1] * 100)
if rate < keep_floor:
return ("rejected-more-than-kept",
"%.0f%% accepted over %d proposal(s)%s: a majority of the "
"diffs shown were discarded after being generated and billed"
% (rate * 100, total, tail))
if rate < thin:
return ("low-acceptance",
"%.0f%% accepted over %d proposal(s)%s"
% (rate * 100, total, tail))
return ("healthy",
"%.0f%% accepted over %d proposal(s)%s" % (rate * 100, total, tail))
def repair_lines(state, row):
"""The repair for one classified actor. Pure. A conversation, not a change."""
if state not in FINDINGS:
return []
commits = max(0, int((row or {}).get("commits") or 0))
lines = [
"review project setup for these repositories: a CLAUDE.md context "
"file so the model knows where things live, and narrower task "
"scoping so a proposal is small enough to be judged.",
"check the model and effort level against the work. A frontier model "
"on a mechanical edit produces confident, wide diffs that get "
"rejected on scope rather than on correctness.",
]
if commits > 0:
lines.append("this actor still landed %d commit(s) in the window, so "
"the tool is producing accepted work as well. Read the "
"rate as a cost per accepted change, not as a failure."
% commits)
else:
lines.append("no commits landed through Claude Code in the window, so "
"there is no accepted work to weigh the rejections "
"against.")
return lines
def get(session, path, params):
r = session.get(API + path, params=params, timeout=60)
if r.status_code in (401, 403):
raise SystemExit("%d from Anthropic: /v1/organizations/* needs an Admin "
"API key (sk-ant-admin...), not a workspace key"
% r.status_code)
r.raise_for_status()
return r.json()
def pages(session, path, params):
"""Walk one day of the paginated Claude Code usage report."""
params = dict(params)
while True:
page = get(session, path, params)
yield page
if not page.get("has_more") or not page.get("next_page"):
return
params["page"] = page["next_page"]
def day_strings(days, today=None):
"""The UTC dates to request, newest first. Pure. Today is excluded."""
end = today or dt.datetime.now(dt.timezone.utc).date()
return [(end - dt.timedelta(days=n)).isoformat()
for n in range(1, max(1, int(days)) + 1)]
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--days", type=int, default=14,
help="UTC days to read, ending yesterday (default 14)")
ap.add_argument("--min-proposals", type=int, default=20,
help="proposals below which no claim is made (default 20)")
ap.add_argument("--floor", type=float, default=0.70,
help="acceptance below which a rate is called low")
ap.add_argument("--show-actors", action="store_true",
help="print email addresses in full instead of masked")
args = ap.parse_args()
admin = os.environ.get("ANTHROPIC_ADMIN_KEY")
if not admin:
log.error("set ANTHROPIC_ADMIN_KEY to an Admin API key (sk-ant-admin...); "
"a workspace key cannot read /v1/organizations/*")
return 2
s = requests.Session()
s.headers.update({"x-api-key": admin, "anthropic-version": VERSION})
collected = []
days = day_strings(args.days)
for day in days:
collected.extend(pages(s, "/organizations/usage_report/claude_code",
{"starting_at": day, "limit": 1000}))
rows = fold(collected)
if not rows:
log.info("no Claude Code records over %d day(s). This report covers "
"Claude Code on the Claude API only.", len(days))
return 0
bad = 0
for who in sorted(rows, key=lambda a: -rows[a]["cents"]):
row = rows[who]
state, detail = verdict(row, args.min_proposals, thin=args.floor)
label = who if args.show_actors else mask(who)
line = "%-24s %-20s %s, $%.2f" % (state, label, detail,
row["cents"] / 100.0)
if state in FINDINGS:
bad += 1
log.warning(line)
for repair in repair_lines(state, row):
log.warning(" repair: %s", repair)
else:
log.info(line)
log.info("%d actor(s) over %d day(s), %d finding(s)",
len(rows), len(days), bad)
log.info("the rate and the cost are separate readings: no per-proposal "
"token counts exist to join them, so the share of spend that was "
"discarded is not a number this API can support")
return 1 if bad else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Find Claude Code actors whose edit proposals are mostly rejected.
*
* Read only. One paged GET per UTC day against the Claude Code usage report
* with an Admin API key.
*
* Every rejected proposal was fully generated and fully billed before it was
* displayed. The acceptance rate and the estimated cost are printed as two
* separate readings and are never multiplied: the report carries no
* per-proposal token counts, so the share of spend that was discarded is not a
* number this API can support.
*/
const API = 'https://api.anthropic.com/v1';
const VERSION = '2023-06-01';
// The tools that propose a change a person then keeps or discards. Counted
// apart rather than averaged, because they fail for different reasons.
const EDIT_TOOLS = ['edit_tool', 'multi_edit_tool', 'write_tool',
'notebook_edit_tool'];
const FINDINGS = new Set(['rejected-more-than-kept', 'low-acceptance']);
/** Who the record belongs to. Pure. Both actor shapes, plus neither. */
export function actorName(record) {
const actor = record?.actor;
if (!actor || typeof actor !== 'object') return 'unattributed';
for (const field of ['email_address', 'api_key_name']) {
const value = String(actor[field] ?? '').trim();
if (value) return value;
}
return 'unattributed';
}
/** Hide the local part of an email address. Pure. Non-emails pass through. */
export function mask(name) {
const text = String(name ?? '').trim();
if (!text.includes('@')) return text || 'unattributed';
const at = text.indexOf('@');
const local = text.slice(0, at);
if (!local) return text;
return `${local[0]}***${text.slice(at)}`;
}
/**
* Accepted and rejected counts per edit tool on one record. Pure.
* A tool nobody used is omitted rather than zeroed.
*/
export function actionsOf(record) {
const actions = record?.tool_actions && typeof record.tool_actions === 'object'
? record.tool_actions : {};
const out = {};
for (const tool of EDIT_TOOLS) {
const row = actions[tool];
if (!row || typeof row !== 'object') continue;
const counts = {};
for (const field of ['accepted', 'rejected']) {
const n = Number(row[field] ?? 0);
counts[field] = Number.isFinite(n) ? Math.max(0, Math.trunc(n)) : 0;
}
if (counts.accepted || counts.rejected) out[tool] = counts;
}
return out;
}
/** Fold every record into one row per actor. Pure. Productivity travels with the rate. */
export function fold(pages) {
const rows = {};
for (const page of pages ?? []) {
for (const record of page?.data ?? []) {
if (!record || typeof record !== 'object') continue;
const who = actorName(record);
const row = rows[who] ?? { tools: {}, days: 0, sessions: 0, commits: 0,
prs: 0, added: 0, removed: 0, cents: 0 };
rows[who] = row;
row.days += 1;
for (const [tool, counts] of Object.entries(actionsOf(record))) {
const into = row.tools[tool] ?? { accepted: 0, rejected: 0 };
into.accepted += counts.accepted;
into.rejected += counts.rejected;
row.tools[tool] = into;
}
const core = record.core_metrics && typeof record.core_metrics === 'object'
? record.core_metrics : {};
for (const [field, key] of [['num_sessions', 'sessions'],
['commits_by_claude_code', 'commits'],
['pull_requests_by_claude_code', 'prs']]) {
const n = Number(core[field] ?? 0);
if (Number.isFinite(n)) row[key] += Math.max(0, Math.trunc(n));
}
const lines = core.lines_of_code && typeof core.lines_of_code === 'object'
? core.lines_of_code : {};
for (const key of ['added', 'removed']) {
const n = Number(lines[key] ?? 0);
if (Number.isFinite(n)) row[key] += Math.max(0, Math.trunc(n));
}
for (const entry of record.model_breakdown ?? []) {
const cost = entry?.estimated_cost && typeof entry.estimated_cost === 'object'
? entry.estimated_cost : {};
const n = Number(cost.amount ?? 0);
if (Number.isFinite(n)) row.cents += n;
}
}
}
return rows;
}
/** Accepted and rejected across every edit tool for one actor. Pure. */
export function totals(row) {
let accepted = 0;
let rejected = 0;
for (const counts of Object.values(row?.tools ?? {})) {
accepted += Math.max(0, Number(counts?.accepted ?? 0));
rejected += Math.max(0, Number(counts?.rejected ?? 0));
}
return [accepted, rejected];
}
/**
* accepted / (accepted + rejected). Pure. Null when nothing was proposed.
* Null rather than 0: an actor who proposed nothing has no rate, and zero
* would put them at the top of a list of the worst.
*/
export function acceptance(counts) {
const accepted = Math.max(0, Number(counts?.accepted ?? 0));
const rejected = Math.max(0, Number(counts?.rejected ?? 0));
const total = accepted + rejected;
if (total <= 0) return null;
return accepted / total;
}
/** The lowest-scoring tool with enough volume to mean it. Pure. Null when none. */
export function worstTool(row, minProposals = 10) {
let worst = null;
for (const [tool, counts] of Object.entries(row?.tools ?? {}).sort()) {
const total = Math.max(0, Number(counts?.accepted ?? 0))
+ Math.max(0, Number(counts?.rejected ?? 0));
if (total < minProposals) continue;
const rate = acceptance(counts);
if (rate === null) continue;
if (worst === null || rate < worst[1]) worst = [tool, rate];
}
return worst;
}
/** Classify one actor's acceptance. Pure. Returns [state, detail]. */
export function verdict(row, minProposals = 20, keepFloor = 0.5, thin = 0.7) {
const [accepted, rejected] = totals(row);
const total = accepted + rejected;
if (total < minProposals) {
return ['too-few-proposals',
`${total} proposal(s), under the floor of ${minProposals}: a bad ` +
'afternoon is not a pattern'];
}
const rate = acceptance({ accepted, rejected });
const worst = worstTool(row);
const tail = worst === null ? ''
: `; worst tool ${worst[0]} at ${(worst[1] * 100).toFixed(0)}%`;
if (rate < keepFloor) {
return ['rejected-more-than-kept',
`${(rate * 100).toFixed(0)}% accepted over ${total} proposal(s)${tail}: ` +
'a majority of the diffs shown were discarded after being generated and billed'];
}
if (rate < thin) {
return ['low-acceptance',
`${(rate * 100).toFixed(0)}% accepted over ${total} proposal(s)${tail}`];
}
return ['healthy',
`${(rate * 100).toFixed(0)}% accepted over ${total} proposal(s)${tail}`];
}
/** The repair for one classified actor. Pure. A conversation, not a change. */
export function repairLines(state, row) {
if (!FINDINGS.has(state)) return [];
const commits = Math.max(0, Number(row?.commits ?? 0));
const lines = [
'review project setup for these repositories: a CLAUDE.md context file so ' +
'the model knows where things live, and narrower task scoping so a ' +
'proposal is small enough to be judged.',
'check the model and effort level against the work. A frontier model on a ' +
'mechanical edit produces confident, wide diffs that get rejected on ' +
'scope rather than on correctness.',
];
if (commits > 0) {
lines.push(`this actor still landed ${commits} commit(s) in the window, so ` +
'the tool is producing accepted work as well. Read the rate as ' +
'a cost per accepted change, not as a failure.');
} else {
lines.push('no commits landed through Claude Code in the window, so there ' +
'is no accepted work to weigh the rejections against.');
}
return lines;
}
/** The UTC dates to request, newest first. Pure. Today is excluded. */
export function dayStrings(days, today = new Date()) {
const out = [];
const count = Math.max(1, Math.trunc(Number(days) || 1));
for (let n = 1; n <= count; n += 1) {
const d = new Date(Date.UTC(today.getUTCFullYear(), today.getUTCMonth(),
today.getUTCDate() - n));
out.push(d.toISOString().slice(0, 10));
}
return out;
}
async function get(key, path, params) {
const url = new URL(API + path);
for (const [k, v] of params) url.searchParams.append(k, v);
const res = await fetch(url, {
headers: { 'x-api-key': key, 'anthropic-version': VERSION },
});
if (res.status === 401 || res.status === 403) {
throw new Error(`${res.status} from Anthropic: /v1/organizations/* needs ` +
'an Admin API key (sk-ant-admin...), not a workspace key');
}
if (!res.ok) throw new Error(`${res.status} from ${path}`);
return res.json();
}
async function main() {
const admin = process.env.ANTHROPIC_ADMIN_KEY;
if (!admin) {
console.error('set ANTHROPIC_ADMIN_KEY to an Admin API key (sk-ant-admin...); ' +
'a workspace key cannot read /v1/organizations/*');
process.exitCode = 2;
return;
}
const days = Number(process.env.DAYS ?? 14);
const minProposals = Number(process.env.MIN_PROPOSALS ?? 20);
const showActors = process.env.SHOW_ACTORS === '1';
const dates = dayStrings(days);
const collected = [];
for (const day of dates) {
const base = [['starting_at', day], ['limit', '1000']];
let params = base;
for (;;) {
const page = await get(admin, '/organizations/usage_report/claude_code', params);
collected.push(page);
if (!page?.has_more || !page?.next_page) break;
params = [...base, ['page', page.next_page]];
}
}
const rows = fold(collected);
const actors = Object.keys(rows);
if (actors.length === 0) {
console.log(`no Claude Code records over ${dates.length} day(s). This ` +
'report covers Claude Code on the Claude API only.');
return;
}
let bad = 0;
for (const who of actors.sort((a, b) => rows[b].cents - rows[a].cents)) {
const row = rows[who];
const [state, detail] = verdict(row, minProposals);
const label = showActors ? who : mask(who);
const line = `${state.padEnd(24)} ${label.padEnd(20)} ${detail}, ` +
`$${(row.cents / 100).toFixed(2)}`;
if (FINDINGS.has(state)) {
bad += 1;
console.warn(line);
for (const repair of repairLines(state, row)) console.warn(` repair: ${repair}`);
} else {
console.log(line);
}
}
console.log(`${actors.length} actor(s) over ${dates.length} day(s), ${bad} finding(s)`);
console.log('the rate and the cost are separate readings: no per-proposal ' +
'token counts exist to join them, so the share of spend that was ' +
'discarded is not a number this API can support');
process.exitCode = bad ? 1 : 0;
}
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The headline test is four hundred and twelve proposals with a thirty-eight per cent acceptance rate, and the assertion that goes with it names the worst tool rather than the average, because multi_edit_tool at twenty-one per cent and edit_tool at sixty are two different repairs hiding in one number. The test beside it is the floor: nine proposals is a bad afternoon and must come back as insufficient evidence, because this figure sits next to a person's name. The rest pin the null rate for an actor who proposed nothing, the commits that travel with the verdict so a low rate is never read alone, tools that are absent rather than zeroed, and the accumulation across days.
import datetime as dt
from claude_code_edit_acceptance import (acceptance, actions_of, actor_name,
day_strings, fold, mask, repair_lines,
totals, verdict, worst_tool)
def record(email, tools, commits=0, cents="0", prs=0):
return {"date": "2026-08-30",
"actor": {"type": "user_actor", "email_address": email},
"core_metrics": {"num_sessions": 6,
"commits_by_claude_code": commits,
"pull_requests_by_claude_code": prs,
"lines_of_code": {"added": 400, "removed": 90}},
"tool_actions": tools,
"model_breakdown": [{"model": "claude-opus-5",
"tokens": {"input": 1, "output": 1},
"estimated_cost": {"currency": "USD",
"amount": cents}}]}
def page(records):
return {"data": records, "has_more": False}
def test_a_majority_of_generated_diffs_being_thrown_away_is_the_finding():
# The note in one assertion. Every one of these 256 rejections was fully
# generated and fully billed before anybody read it.
rows = fold([page([record("busy@example.com", {
"edit_tool": {"accepted": 120, "rejected": 80},
"multi_edit_tool": {"accepted": 36, "rejected": 136},
"write_tool": {"accepted": 0, "rejected": 40},
}, commits=4, cents="31040")])])
row = rows["busy@example.com"]
assert totals(row) == (156, 256)
state, detail = verdict(row)
assert state == "rejected-more-than-kept"
assert "38% accepted over 412 proposal(s)" in detail
# The average hides which tool is the problem, so the worst is named.
assert "worst tool write_tool at 0%" in detail
assert worst_tool(row)[0] == "write_tool"
assert any("CLAUDE.md" in line for line in repair_lines(state, row))
def test_a_bad_afternoon_is_not_a_pattern():
rows = fold([page([record("quiet@example.com", {
"edit_tool": {"accepted": 2, "rejected": 7}})])])
state, detail = verdict(rows["quiet@example.com"])
assert state == "too-few-proposals"
assert "under the floor of 20" in detail
assert repair_lines(state, rows["quiet@example.com"]) == []
def test_the_commits_travel_with_the_rate_so_it_is_never_read_alone():
landing = fold([page([record("lands@example.com", {
"edit_tool": {"accepted": 90, "rejected": 160}}, commits=26)])])
row = landing["lands@example.com"]
state, _ = verdict(row)
assert state == "rejected-more-than-kept"
assert any("26 commit(s)" in line for line in repair_lines(state, row))
empty = fold([page([record("none@example.com", {
"edit_tool": {"accepted": 90, "rejected": 160}}, commits=0)])])
lines = repair_lines("rejected-more-than-kept", empty["none@example.com"])
assert any("no commits landed" in line for line in lines)
def test_an_actor_who_proposed_nothing_has_no_rate_rather_than_zero():
assert acceptance({"accepted": 0, "rejected": 0}) is None
assert acceptance({}) is None
assert acceptance({"accepted": 3, "rejected": 1}) == 0.75
assert worst_tool({"tools": {}}) is None
# Under the per-tool floor there is no worst tool to name.
assert worst_tool({"tools": {"edit_tool": {"accepted": 1, "rejected": 2}}}) is None
def test_a_tool_nobody_used_is_absent_and_not_a_zero():
actions = actions_of({"tool_actions": {
"edit_tool": {"accepted": 4, "rejected": 1},
"write_tool": {"accepted": 0, "rejected": 0},
"bash_tool": {"accepted": 99, "rejected": 99}}})
assert list(actions) == ["edit_tool"]
assert actions_of({}) == {}
assert actions_of(None) == {}
assert actions_of({"tool_actions": {"edit_tool": {"accepted": "x",
"rejected": 3}}}) == \
{"edit_tool": {"accepted": 0, "rejected": 3}}
def test_counts_accumulate_across_days_and_across_actor_shapes():
day = [record("a@example.com", {"edit_tool": {"accepted": 10, "rejected": 5}},
commits=1, cents="500"),
{"actor": {"type": "api_actor", "api_key_name": "ci-runner"},
"core_metrics": {"num_sessions": 1},
"tool_actions": {"edit_tool": {"accepted": 30, "rejected": 2}},
"model_breakdown": []}]
rows = fold([page(day), page(day)])
assert totals(rows["a@example.com"]) == (20, 10)
assert rows["a@example.com"]["commits"] == 2
assert rows["a@example.com"]["cents"] == 1000
assert rows["a@example.com"]["added"] == 800
assert totals(rows["ci-runner"]) == (60, 4)
assert verdict(rows["ci-runner"])[0] == "healthy"
def test_actors_are_resolved_and_masked_before_being_printed():
assert actor_name({"actor": {"email_address": "a@example.com"}}) == "a@example.com"
assert actor_name({"actor": {"api_key_name": "ci"}}) == "ci"
assert actor_name({}) == "unattributed"
assert mask("someone@example.com") == "s***@example.com"
assert mask("ci-runner") == "ci-runner"
assert mask(None) == "unattributed"
def test_the_thin_band_sits_between_kept_and_healthy():
row = {"tools": {"edit_tool": {"accepted": 61, "rejected": 39}}, "commits": 0}
assert verdict(row)[0] == "low-acceptance"
row = {"tools": {"edit_tool": {"accepted": 88, "rejected": 12}}, "commits": 0}
assert verdict(row)[0] == "healthy"
assert day_strings(2, dt.date(2026, 3, 1)) == ["2026-02-28", "2026-02-27"]
assert fold([]) == {} and fold(None) == {}
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { acceptance, actionsOf, actorName, dayStrings, fold, mask, repairLines,
totals, verdict, worstTool }
from './claude-code-edit-acceptance.mjs';
const record = (email, tools, commits = 0, cents = 0, prs = 0) => ({
date: '2026-08-30',
actor: { type: 'user_actor', email_address: email },
core_metrics: { num_sessions: 6, commits_by_claude_code: commits,
pull_requests_by_claude_code: prs,
lines_of_code: { added: 400, removed: 90 } },
tool_actions: tools,
model_breakdown: [{ model: 'claude-opus-5', tokens: { input: 1, output: 1 },
estimated_cost: { currency: 'USD', amount: cents } }],
});
const page = (records) => ({ data: records, has_more: false });
test('a majority of generated diffs being thrown away is the finding', () => {
const rows = fold([page([record('busy@example.com', {
edit_tool: { accepted: 120, rejected: 80 },
multi_edit_tool: { accepted: 36, rejected: 136 },
write_tool: { accepted: 0, rejected: 40 },
}, 4, 31040)])]);
const row = rows['busy@example.com'];
assert.deepEqual(totals(row), [156, 256]);
const [state, detail] = verdict(row);
assert.equal(state, 'rejected-more-than-kept');
assert.match(detail, /38% accepted over 412 proposal/);
assert.match(detail, /worst tool write_tool at 0%/);
assert.equal(worstTool(row)[0], 'write_tool');
assert.ok(repairLines(state, row).some((l) => l.includes('CLAUDE.md')));
});
test('a bad afternoon is not a pattern', () => {
const rows = fold([page([record('quiet@example.com', {
edit_tool: { accepted: 2, rejected: 7 } })])]);
const [state, detail] = verdict(rows['quiet@example.com']);
assert.equal(state, 'too-few-proposals');
assert.match(detail, /under the floor of 20/);
assert.deepEqual(repairLines(state, rows['quiet@example.com']), []);
});
test('the commits travel with the rate so it is never read alone', () => {
const landing = fold([page([record('lands@example.com', {
edit_tool: { accepted: 90, rejected: 160 } }, 26)])]);
const row = landing['lands@example.com'];
const [state] = verdict(row);
assert.equal(state, 'rejected-more-than-kept');
assert.ok(repairLines(state, row).some((l) => l.includes('26 commit(s)')));
const empty = fold([page([record('none@example.com', {
edit_tool: { accepted: 90, rejected: 160 } }, 0)])]);
assert.ok(repairLines('rejected-more-than-kept', empty['none@example.com'])
.some((l) => l.includes('no commits landed')));
});
test('an actor who proposed nothing has no rate rather than zero', () => {
assert.equal(acceptance({ accepted: 0, rejected: 0 }), null);
assert.equal(acceptance({}), null);
assert.equal(acceptance({ accepted: 3, rejected: 1 }), 0.75);
assert.equal(worstTool({ tools: {} }), null);
assert.equal(worstTool({ tools: { edit_tool: { accepted: 1, rejected: 2 } } }), null);
});
test('a tool nobody used is absent and not a zero', () => {
const actions = actionsOf({ tool_actions: {
edit_tool: { accepted: 4, rejected: 1 },
write_tool: { accepted: 0, rejected: 0 },
bash_tool: { accepted: 99, rejected: 99 } } });
assert.deepEqual(Object.keys(actions), ['edit_tool']);
assert.deepEqual(actionsOf({}), {});
assert.deepEqual(actionsOf(null), {});
assert.deepEqual(actionsOf({ tool_actions: { edit_tool: { accepted: 'x', rejected: 3 } } }),
{ edit_tool: { accepted: 0, rejected: 3 } });
});
test('counts accumulate across days and across actor shapes', () => {
const day = [
record('a@example.com', { edit_tool: { accepted: 10, rejected: 5 } }, 1, 500),
{ actor: { type: 'api_actor', api_key_name: 'ci-runner' },
core_metrics: { num_sessions: 1 },
tool_actions: { edit_tool: { accepted: 30, rejected: 2 } },
model_breakdown: [] },
];
const rows = fold([page(day), page(day)]);
assert.deepEqual(totals(rows['a@example.com']), [20, 10]);
assert.equal(rows['a@example.com'].commits, 2);
assert.equal(rows['a@example.com'].cents, 1000);
assert.equal(rows['a@example.com'].added, 800);
assert.deepEqual(totals(rows['ci-runner']), [60, 4]);
assert.equal(verdict(rows['ci-runner'])[0], 'healthy');
});
test('actors are resolved and masked before being printed', () => {
assert.equal(actorName({ actor: { email_address: 'a@example.com' } }),
'a@example.com');
assert.equal(actorName({ actor: { api_key_name: 'ci' } }), 'ci');
assert.equal(actorName({}), 'unattributed');
assert.equal(mask('someone@example.com'), 's***@example.com');
assert.equal(mask('ci-runner'), 'ci-runner');
assert.equal(mask(null), 'unattributed');
});
test('the thin band sits between kept and healthy', () => {
assert.equal(verdict({ tools: { edit_tool: { accepted: 61, rejected: 39 } },
commits: 0 })[0], 'low-acceptance');
assert.equal(verdict({ tools: { edit_tool: { accepted: 88, rejected: 12 } },
commits: 0 })[0], 'healthy');
assert.deepEqual(dayStrings(2, new Date('2026-03-01T06:00:00Z')),
['2026-02-28', '2026-02-27']);
assert.deepEqual(fold([]), {});
assert.deepEqual(fold(null), {});
});
FAQ
Is a rejected edit wasted money?
The tokens are spent either way, so in the narrow sense yes: the diff was generated at full output rates before it was displayed, and dismissing it does not refund anything. But rejection is also the review step working, and an occasional no is the system behaving correctly. What the note is about is a sustained majority, where the bulk of an actor's output budget goes to work nobody keeps, and that is a pattern rather than an event.
Why does the script not report the share of my spend that was discarded?
Because the report carries no per-proposal token counts. To turn a rejection rate into a share of cost you would have to know whether the rejected diffs were the large ones or the small ones, and nothing on this endpoint says. The acceptance rate and the estimated cost are printed side by side and never multiplied, because a plausible-looking product of the two would be indistinguishable from a measurement.
How is this different from the other Claude Code note?
Same endpoint, different block, opposite kind of conclusion. The cache note reads model_breakdown[].tokens and finds input being paid for twice because a prefix was not reused; that is a caching repair. This reads tool_actions and finds output that was generated once, correctly, and then discarded by the person it was for; that is a project-setup repair. Neither number tells you anything about the other, which is why they are two notes.
Which tool usually drags the rate down?
In practice multi_edit_tool and write_tool, and for different reasons. A poor multi-edit rate usually means the task was scoped too wide, so one proposal touches more than a reviewer is willing to accept in one go. A poor write rate often means whole files being generated where a targeted edit was wanted. That is why the script keeps the four tools apart and names the worst one, instead of reporting a single average that hides both.
Can the API tell me why a proposal was rejected?
No. There is no reason code, no diff content and no prompt on this endpoint; there is a count of accepted and a count of rejected, and that is the whole surface. The script names the actors, the tools and the rates, puts the commits and lines of code beside them so a low rate is not read in isolation, and hands the question to people who can see the repository. Anything more would be the script inventing an explanation.
Related field notes
- The same report read for cache reads instead of acceptance
- Why output tokens are where a generation bill actually sits
- A model chosen larger than the work it is being given
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Claude Code Analytics API — Claude Docs
- Analytics API — Claude Docs
- Pricing — Claude Docs
- Usage and Cost API — Claude Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.