Diagnostic GitHub API
a hot endpoint burns 900 points a minute and gets throttled
One endpoint in the job keeps failing and the rest are fine. It fails for about a minute, recovers, and fails again twenty minutes later. The hourly quota is barely touched and the concurrency is one, because someone already serialised it after the last incident. What is left is the cap nobody budgets for: not how many requests you make, but how much work you asked one path to do inside sixty seconds.
There are two per-minute ceilings on a single endpoint and either can bind first. The point cap allows 900 points a minute, where a read costs 1 point and a write costs 5. The CPU cap allows 90 seconds of CPU time per 60 seconds of real time, and GitHub documents total response time as a rough estimate of it.
That second cap is why a modest request rate gets throttled. At 60 ms a request the point cap binds and you can run 900 a minute. At 500 ms a request the CPU cap binds at 180 a minute, and the point counter never comes close. So the number to compute is not "how many requests" but "how many of this endpoint", and it falls straight out of the response time you can measure in a few seconds.
The problem in plain words
It is selective, which sends the investigation the wrong way. Every other call in the same process, on the same token, in the same second, keeps working. That looks like the endpoint is broken or the resource is special, so people go looking at permissions on that one path, or at the size of the repository behind it, and the answer is neither.
It also recovers on its own, quickly, which makes it hard to catch and easy to dismiss. A minute of 403s inside a fifteen-minute job shows up as a handful of retried items and a run that took slightly longer. Nobody opens an incident for that. It becomes a background failure rate that everyone has stopped seeing, until the day the job grows and the minute becomes ten.
And every number people habitually check looks healthy. The hourly bucket is fine. The concurrency is one. There is no header for the limit that fired, because secondary limits do not publish one. The only artefacts are a 403 or 429 whose body says "secondary rate limit", a retry-after, and the fact that the failures cluster on a single path.
Why it happens
Points are charged per request, by method. A GET, HEAD or OPTIONS costs one point. Anything that changes something costs five. The REST allowance is 900 points a minute, so 900 reads or 180 writes, and the two are drawn from the same number.
CPU time is charged per request, by how hard it was. No more than 90 seconds of CPU per 60 seconds of wall clock. Endpoints are not equal here: a search, a large diff, a commit comparison across a long range, or a repository listing for an org with thousands of repositories costs the server far more than reading one issue. GitHub's own guidance is to estimate this from total response time, which is generous to you as a measurement because response time also includes the network.
Whichever cap is lower is the one that decides. Divide 900 by the points per request and you get the point ceiling. Divide 90 by the mean response time in seconds and you get the CPU ceiling. The smaller of the two is your real rate for that path. Below about a hundred milliseconds a request the points bind; above it, CPU does, and from there on the ceiling drops as the endpoint gets slower.
The cap is per endpoint, which cuts both ways. It means one expensive path can be throttled while everything else runs, and it means moving work off that path fixes it. It also means x-ratelimit-resource on the failing response is worth reading: it names the bucket the request was billed to, which identifies the endpoint family rather than leaving you to guess from a URL.
Retrying immediately makes it last longer. The retries are themselves requests to the same expensive path, so they keep the minute full. This is the mechanism by which a sixty-second throttle becomes a ten-minute one, and it is the reason the repair is to spread the calls rather than to catch the error.
The fix, as a flow
There is no bucket to read for a secondary limit, so the script builds the ceiling itself: time a few calls to one path, then divide 900 by the points per request and 90 by the mean seconds. The smaller answer is the rate that path will actually take.
How to fix it
Find which path the failures cluster on
Secondary limits are per endpoint, so the distribution of failures is the diagnosis. If the 403s are spread evenly across every call you make, this is not your problem. If nine in ten of them are on one path, it is. Read x-ratelimit-resource on a failing response to see which bucket GitHub billed it to.
Measure what one call to that path actually costs
A handful of sequential requests, spaced out, is enough. Take the mean response time. It is not the server's CPU time — it includes network and queueing — but GitHub's own documentation offers total response time as the estimate, and over-estimating cost here means under-estimating your safe rate, which is the direction you want to be wrong in.
Compute both ceilings and take the smaller
900 divided by the points per request, and 90 divided by the mean seconds. A 40 ms read gives 900 and 2,250, so points bind at 900 a minute. A 600 ms read gives 900 and 150, so CPU binds at 150. That second number is the one that surprises people, because 150 requests a minute is a rate a plain loop reaches without trying.
Compare it against the rate you are actually configured for
A job with 4,000 items to fetch and no pacing will run at whatever the endpoint allows, which is exactly the rate that trips this. Divide the work by the safe rate and you get the honest duration: 4,000 items at 150 a minute is 27 minutes, and the alternative to 27 minutes is not 9 minutes, it is 9 minutes of failures followed by a retry storm.
Make the work cheaper before you make it slower
Pacing is the fallback, not the fix. One GraphQL query that returns fields for fifty repositories replaces fifty expensive REST calls and is billed to a different allowance. A list endpoint with per_page=100 replaces a hundred item reads. A conditional request that returns 304 costs the server almost nothing. Each of those lowers the numerator instead of raising the clock.
How to check it worked
Re-measure the path and check the configured rate sits under the computed ceiling.
python3 github_endpoint_cost_audit.py --path /repos/octocat/hello-world/commits --rate 90
# clear: 0.21 s a call, CPU binds at 428/min, configured 90/minThe full code
Four pure functions and one small sampler. points_for encodes the documented method costs, cost_profile collapses samples into a mean per path, safe_rate computes both ceilings and reports which one binds, and verdict compares that against the rate you say you run at. The sampler defaults to GET /rate_limit, which is free, and warns before it measures anything that is not — the irony of a script that trips the limit it is measuring is available to anyone who forgets to pace the sampling.
"""Compute the request rate one endpoint can sustain before it is throttled.
Read only. Every request is a GET, and the default sampled path is
/rate_limit, which does not count against the primary rate limit.
Two ceilings apply per minute to a single endpoint: 900 points, where a read
costs one point and a write costs five, and 90 seconds of CPU time per 60
seconds of real time. GitHub documents total response time as a rough estimate
of the second one, so a few timed GETs are enough to compute both and see which
binds first. That is the number a caller needs, and it is usually far lower
than 900.
"""
import argparse
import json
import logging
import os
import sys
import time
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("github_endpoint_cost_audit")
API = "https://api.github.com"
UA = "github-endpoint-cost-audit/1.0"
# Documented secondary limits for a single REST endpoint.
POINT_CAP = 900 # points per minute
CPU_CAP = 90.0 # seconds of CPU per 60 seconds of real time
# Reads cost one point; everything that changes state costs five.
CHEAP_METHODS = ("GET", "HEAD", "OPTIONS")
def points_for(method):
"""Documented point cost of one request. Pure.
Anything unrecognised is charged the expensive rate. Guessing low here
would produce a safe-looking ceiling for a request that is not safe, and
the whole output of this script is a number people will pace against.
"""
try:
name = str(method).strip().upper()
except (TypeError, ValueError):
return 5
return 1 if name in CHEAP_METHODS else 5
def cost_profile(samples):
"""Collapse timed samples into one entry per path. Pure.
samples: [{"path", "method", "seconds"}, ...]
Keeps the max as well as the mean because a path whose mean is comfortable
and whose worst case is four times that will be throttled during the worst
case and nowhere else, which is exactly the intermittent shape people
struggle to reproduce.
"""
grouped = {}
for s in samples or []:
try:
path = str(s["path"])
seconds = float(s["seconds"])
except (KeyError, TypeError, ValueError):
continue
if seconds < 0:
continue
entry = grouped.setdefault(path, {"path": path, "calls": 0, "total": 0.0,
"max_seconds": 0.0,
"points": points_for(s.get("method", "GET"))})
entry["calls"] += 1
entry["total"] += seconds
entry["max_seconds"] = max(entry["max_seconds"], seconds)
out = {}
for path, entry in grouped.items():
entry["mean_seconds"] = round(entry["total"] / entry["calls"], 4)
entry["max_seconds"] = round(entry["max_seconds"], 4)
del entry["total"]
out[path] = entry
return out
def safe_rate(mean_seconds, points=1, point_cap=POINT_CAP, cpu_cap=CPU_CAP):
"""Requests per minute this endpoint sustains, and which cap binds. Pure.
The point ceiling is a constant per method. The CPU ceiling falls as the
endpoint gets slower, and it crosses under the point ceiling at around a
tenth of a second a call, which is why an endpoint that feels fast can
still be throttled at a rate nowhere near 900.
"""
try:
seconds = float(mean_seconds)
except (TypeError, ValueError):
seconds = 0.0
points = max(1, int(points))
by_points = point_cap / points
by_cpu = (cpu_cap / seconds) if seconds > 0 else float("inf")
if by_cpu < by_points:
binding, per_minute = "cpu", by_cpu
else:
binding, per_minute = "points", by_points
return {"by_points": round(by_points, 1),
"by_cpu": None if by_cpu == float("inf") else round(by_cpu, 1),
"binding": binding, "per_minute": round(per_minute, 1),
"mean_seconds": round(seconds, 4), "points": points}
def verdict(path, entry, safe, configured=None):
"""Compare the computed ceiling against the rate you run at. Pure."""
mean = safe["mean_seconds"]
ceiling = safe["per_minute"]
cap_name = ("the 90s-of-CPU-per-60s cap" if safe["binding"] == "cpu"
else "the 900-points-a-minute cap")
if configured is None:
return ("ceiling",
"%s costs %.3f s a call, so %s allows about %d request(s) a "
"minute on this path." % (path, mean, cap_name, ceiling))
try:
configured = float(configured)
except (TypeError, ValueError):
return ("ceiling", "%s allows about %d a minute; no configured rate "
"was given to compare it against." % (path, ceiling))
if configured > ceiling:
return ("over-budget",
"%s is configured for %d a minute against a ceiling of %d. %s "
"binds first at %.3f s a call, so the surplus is refused, "
"retried, and refused again."
% (path, configured, ceiling, cap_name, mean))
if configured >= ceiling * 0.8:
return ("near-budget",
"%s runs at %d a minute against a ceiling of %d. One slower "
"response, or one worst case of %.3f s, closes that gap."
% (path, configured, ceiling, entry.get("max_seconds", mean)))
if mean >= 1.0:
return ("expensive",
"%s costs %.3f s a call, which caps it at %d a minute however "
"little you are asking for today. Treat it as a path to move "
"work off rather than a path to pace." % (path, mean, ceiling))
return ("clear",
"%s runs at %d a minute against a ceiling of %d, %s binding."
% (path, configured, ceiling, cap_name))
def sample_path(session, path, count, pause):
"""Time a few sequential GETs. Sequential and paced on purpose: a sampler
that fans out would measure the limit it is trying to describe."""
url = API + path if path.startswith("/") else path
samples, resource, throttled = [], None, False
for i in range(count):
if i:
time.sleep(pause)
start = time.monotonic()
try:
r = session.get(url, timeout=60)
except requests.RequestException as exc:
log.warning("%s sample %d failed: %s", path, i, exc)
continue
elapsed = time.monotonic() - start
headers = {k.lower(): v for k, v in r.headers.items()}
resource = resource or headers.get("x-ratelimit-resource")
if r.status_code in (403, 429) and "secondary rate limit" in r.text.lower():
throttled = True
log.warning("%s was throttled while being measured; retry-after %s",
path, headers.get("retry-after", "absent"))
continue
if r.status_code >= 400:
log.warning("%s sample %d returned %d", path, i, r.status_code)
continue
samples.append({"path": path, "method": "GET", "seconds": elapsed})
return samples, resource, throttled
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--path", action="append", default=None,
help="path to measure; repeatable (default /rate_limit)")
ap.add_argument("--samples", type=int, default=4,
help="timed requests per path")
ap.add_argument("--pause", type=float, default=1.0,
help="seconds between samples")
ap.add_argument("--rate", type=float, default=None,
help="requests per minute your job runs at on these paths")
args = ap.parse_args()
token = os.environ.get("GITHUB_TOKEN")
if not token:
log.error("set GITHUB_TOKEN (a read-only token is enough)")
return 2
paths = args.path or ["/rate_limit"]
billed = [p for p in paths if p.rstrip("/") != "/rate_limit"]
if billed:
log.warning("measuring %d path(s) that do cost quota: %d sample(s) "
"each, one point per sample", len(billed),
args.samples)
session = requests.Session()
session.headers.update({
"Authorization": "Bearer " + token,
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": UA,
})
all_samples, resources, worst = [], {}, "clear"
findings = []
for path in paths:
samples, resource, throttled = sample_path(session, path, max(1, args.samples),
max(0.0, args.pause))
all_samples.extend(samples)
if resource:
resources[path] = resource
if throttled:
log.warning("%s tripped a secondary limit during measurement, which "
"is itself the finding: the endpoint is already over "
"budget at the rate this sampler used", path)
profile = cost_profile(all_samples)
if not profile:
log.error("no successful samples, so there is nothing to cost")
return 2
ranked = sorted(profile.values(), key=lambda e: e["mean_seconds"], reverse=True)
for entry in ranked:
safe = safe_rate(entry["mean_seconds"], entry["points"])
state, detail = verdict(entry["path"], entry, safe, args.rate)
findings.append({"path": entry["path"], "state": state,
"mean_seconds": entry["mean_seconds"],
"max_seconds": entry["max_seconds"],
"billed_to": resources.get(entry["path"]), **safe})
log.info("%-14s %s", state, detail)
if resources.get(entry["path"]):
log.info(" billed to the %s bucket",
resources[entry["path"]])
if state in ("over-budget", "near-budget", "expensive"):
worst = state if worst == "clear" else worst
if worst != "clear":
log.info("repair: replace per-item calls on the expensive path with one "
"GraphQL query returning the same fields, which is billed to a "
"different allowance entirely.")
log.info("repair: raise per_page to 100 on list endpoints so the same "
"data arrives in a third of the calls.")
log.info("repair: send If-None-Match with the stored etag. A 304 costs "
"the server almost nothing and costs you nothing at all.")
log.info("repair: where the calls are unavoidable, spread them across "
"the minute rather than bursting, and on a throttled response "
"sleep the whole retry-after before resuming that path.")
print(json.dumps({"findings": findings, "configured_per_minute": args.rate},
indent=2))
return 1 if worst == "over-budget" else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Compute the request rate one endpoint can sustain before it is throttled.
*
* Read only. Every request is a GET, and the default sampled path is
* /rate_limit, which does not count against the primary rate limit.
*
* Two per-minute ceilings apply to a single endpoint: 900 points, and 90
* seconds of CPU per 60 seconds of real time. Response time is the documented
* rough estimate of the second, so timed GETs are enough to see which binds.
*/
const API = 'https://api.github.com';
const UA = 'github-endpoint-cost-audit/1.0';
// Documented secondary limits for a single REST endpoint.
export const POINT_CAP = 900;
export const CPU_CAP = 90;
// Reads cost one point; everything that changes state costs five.
const CHEAP_METHODS = ['GET', 'HEAD', 'OPTIONS'];
/**
* Documented point cost of one request. Pure.
* Unrecognised methods are charged the expensive rate, because guessing low
* produces a safe-looking ceiling for a request that is not safe.
*/
export function pointsFor(method) {
const name = String(method ?? '').trim().toUpperCase();
return CHEAP_METHODS.includes(name) ? 1 : 5;
}
/**
* Collapse timed samples into one entry per path. Pure.
* Keeps the max as well as the mean: a path with a comfortable mean and a bad
* worst case is throttled during the worst case and nowhere else.
*/
export function costProfile(samples) {
const grouped = new Map();
for (const s of samples ?? []) {
const path = s?.path === undefined ? null : String(s.path);
const seconds = Number(s?.seconds);
if (path === null || !Number.isFinite(seconds) || seconds < 0) continue;
if (!grouped.has(path)) {
grouped.set(path, {
path, calls: 0, total: 0, max_seconds: 0, points: pointsFor(s.method ?? 'GET'),
});
}
const entry = grouped.get(path);
entry.calls += 1;
entry.total += seconds;
entry.max_seconds = Math.max(entry.max_seconds, seconds);
}
const out = {};
for (const [path, entry] of grouped) {
entry.mean_seconds = Math.round((entry.total / entry.calls) * 10000) / 10000;
entry.max_seconds = Math.round(entry.max_seconds * 10000) / 10000;
delete entry.total;
out[path] = entry;
}
return out;
}
/**
* Requests per minute this endpoint sustains, and which cap binds. Pure.
* The CPU ceiling falls as the endpoint gets slower and crosses under the
* point ceiling at around a tenth of a second a call.
*/
export function safeRate(meanSeconds, points = 1, pointCap = POINT_CAP, cpuCap = CPU_CAP) {
const secs = Number.isFinite(Number(meanSeconds)) ? Number(meanSeconds) : 0;
const p = Math.max(1, Number.parseInt(points, 10) || 1);
const byPoints = pointCap / p;
const byCpu = secs > 0 ? cpuCap / secs : Infinity;
const binding = byCpu < byPoints ? 'cpu' : 'points';
const perMinute = Math.min(byCpu, byPoints);
return {
by_points: Math.round(byPoints * 10) / 10,
by_cpu: Number.isFinite(byCpu) ? Math.round(byCpu * 10) / 10 : null,
binding,
per_minute: Math.round(perMinute * 10) / 10,
mean_seconds: Math.round(secs * 10000) / 10000,
points: p,
};
}
/** Compare the computed ceiling against the rate you run at. Pure. */
export function verdict(path, entry, safe, configured = null) {
const mean = safe.mean_seconds;
const ceiling = safe.per_minute;
const capName = safe.binding === 'cpu'
? 'the 90s-of-CPU-per-60s cap'
: 'the 900-points-a-minute cap';
if (configured === null || configured === undefined) {
return ['ceiling',
`${path} costs ${mean.toFixed(3)} s a call, so ${capName} allows about ` +
`${Math.trunc(ceiling)} request(s) a minute on this path.`];
}
const rate = Number(configured);
if (!Number.isFinite(rate)) {
return ['ceiling',
`${path} allows about ${Math.trunc(ceiling)} a minute; no configured ` +
'rate was given to compare it against.'];
}
if (rate > ceiling) {
return ['over-budget',
`${path} is configured for ${Math.trunc(rate)} a minute against a ` +
`ceiling of ${Math.trunc(ceiling)}. ${capName} binds first at ` +
`${mean.toFixed(3)} s a call, so the surplus is refused, retried, and ` +
'refused again.'];
}
if (rate >= ceiling * 0.8) {
return ['near-budget',
`${path} runs at ${Math.trunc(rate)} a minute against a ceiling of ` +
`${Math.trunc(ceiling)}. One slower response, or one worst case of ` +
`${(entry?.max_seconds ?? mean).toFixed(3)} s, closes that gap.`];
}
if (mean >= 1) {
return ['expensive',
`${path} costs ${mean.toFixed(3)} s a call, which caps it at ` +
`${Math.trunc(ceiling)} a minute however little you are asking for ` +
'today. Treat it as a path to move work off rather than a path to pace.'];
}
return ['clear',
`${path} runs at ${Math.trunc(rate)} a minute against a ceiling of ` +
`${Math.trunc(ceiling)}, ${capName} binding.`];
}
const sleep = (ms) => new Promise((r) => { setTimeout(r, ms); });
/** Time a few sequential GETs. Sequential on purpose: a sampler that fanned
* out would measure the limit it is trying to describe. */
async function samplePath(token, path, count, pause) {
const url = path.startsWith('/') ? API + path : path;
const samples = [];
let resource = null;
let throttled = false;
for (let i = 0; i < count; i += 1) {
if (i) await sleep(pause * 1000);
const start = performance.now();
let res;
try {
res = await fetch(url, {
headers: {
Authorization: `Bearer ${token}`,
Accept: 'application/vnd.github+json',
'X-GitHub-Api-Version': '2022-11-28',
'User-Agent': UA,
},
});
} catch (err) {
console.warn(`${path} sample ${i} failed: ${err.message}`);
continue;
}
const text = await res.text();
const elapsed = (performance.now() - start) / 1000;
const lowered = {};
for (const [k, v] of res.headers.entries()) lowered[k.toLowerCase()] = v;
resource = resource ?? lowered['x-ratelimit-resource'] ?? null;
if ((res.status === 403 || res.status === 429)
&& text.toLowerCase().includes('secondary rate limit')) {
throttled = true;
console.warn(`${path} was throttled while being measured; retry-after ` +
`${lowered['retry-after'] ?? 'absent'}`);
continue;
}
if (res.status >= 400) {
console.warn(`${path} sample ${i} returned ${res.status}`);
continue;
}
samples.push({ path, method: 'GET', seconds: elapsed });
}
return { samples, resource, throttled };
}
async function main() {
const token = process.env.GITHUB_TOKEN;
if (!token) {
console.error('set GITHUB_TOKEN (a read-only token is enough)');
process.exitCode = 2;
return;
}
const path = process.argv[2] ?? '/rate_limit';
const count = Math.max(1, Number.parseInt(process.argv[3] ?? '4', 10) || 4);
const rate = process.argv[4] === undefined ? null : Number(process.argv[4]);
if (path.replace(/\/$/, '') !== '/rate_limit') {
console.warn(`measuring a path that does cost quota: ${count} sample(s), ` +
'one point each');
}
const { samples, resource, throttled } = await samplePath(token, path, count, 1);
if (throttled) {
console.warn(`${path} tripped a secondary limit during measurement, which ` +
'is itself the finding: the endpoint is already over budget at the rate ' +
'this sampler used');
}
const profile = costProfile(samples);
const entries = Object.values(profile).sort((a, b) => b.mean_seconds - a.mean_seconds);
if (!entries.length) {
console.error('no successful samples, so there is nothing to cost');
process.exitCode = 2;
return;
}
const findings = [];
let worst = 'clear';
for (const entry of entries) {
const safe = safeRate(entry.mean_seconds, entry.points);
const [state, detail] = verdict(entry.path, entry, safe, rate);
findings.push({ path: entry.path, state, max_seconds: entry.max_seconds,
billed_to: resource, ...safe });
console.log(`${state.padEnd(14)} ${detail}`);
if (resource) console.log(` billed to the ${resource} bucket`);
if (['over-budget', 'near-budget', 'expensive'].includes(state) && worst === 'clear') {
worst = state;
}
}
if (worst !== 'clear') {
console.log('repair: replace per-item calls on the expensive path with one ' +
'GraphQL query, which is billed to a different allowance entirely.');
console.log('repair: raise per_page to 100 on list endpoints so the same ' +
'data arrives in a third of the calls.');
console.log('repair: send If-None-Match with the stored etag. A 304 costs ' +
'the server almost nothing and costs you nothing at all.');
console.log('repair: where the calls are unavoidable, spread them across ' +
'the minute rather than bursting, and sleep the whole retry-after before ' +
'resuming that path.');
}
console.log(JSON.stringify({ findings, configured_per_minute: rate }, null, 2));
process.exitCode = worst === 'over-budget' ? 1 : 0;
}
// Only run when invoked directly, so importing this from the test file does not
// start main() and set an exit code the tests never asked for.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The crossover is the whole point of this note, so it is the case the tests spend most of their time on: the response time at which the CPU ceiling drops below the point ceiling, and what the answer looks like either side of it. Around a tenth of a second the two swap, and a script that only ever knew about 900 points a minute reports a rate three times too high for a slow endpoint. The rest pins the defensive choices — an unknown method is charged as a write, a zero response time does not divide by zero, and a sample with a missing field is dropped rather than counted as instant.
from github_endpoint_cost_audit import points_for, cost_profile, safe_rate, verdict
def test_reads_cost_one_point_and_writes_cost_five():
assert points_for("GET") == 1
assert points_for("head") == 1
assert points_for("OPTIONS") == 1
assert points_for("patch") == 5
assert points_for("delete") == 5
def test_an_unknown_method_is_charged_the_expensive_rate():
# Guessing low would produce a safe-looking ceiling for a request that is
# not safe, and the ceiling is the number people pace against.
assert points_for("QUERY") == 5
assert points_for(None) == 5
assert points_for("") == 5
def test_samples_are_grouped_by_path_and_averaged():
profile = cost_profile([
{"path": "/a", "method": "GET", "seconds": 0.1},
{"path": "/a", "method": "GET", "seconds": 0.3},
{"path": "/b", "method": "GET", "seconds": 1.0},
])
assert profile["/a"]["calls"] == 2
assert profile["/a"]["mean_seconds"] == 0.2
assert profile["/a"]["max_seconds"] == 0.3
assert profile["/b"]["mean_seconds"] == 1.0
def test_a_malformed_sample_is_dropped_rather_than_counted_as_instant():
profile = cost_profile([
{"path": "/a", "seconds": 0.5},
{"path": "/a", "seconds": "slow"},
{"seconds": 0.5},
{"path": "/a", "seconds": -1},
])
assert profile["/a"]["calls"] == 1
assert profile["/a"]["mean_seconds"] == 0.5
def test_no_samples_profile_nothing():
assert cost_profile([]) == {}
assert cost_profile(None) == {}
def test_a_fast_endpoint_is_bound_by_points():
safe = safe_rate(0.04)
assert safe["binding"] == "points"
assert safe["per_minute"] == 900.0
assert safe["by_cpu"] == 2250.0
def test_a_slow_endpoint_is_bound_by_cpu_time_instead():
safe = safe_rate(0.6)
assert safe["binding"] == "cpu"
assert safe["per_minute"] == 150.0
def test_the_two_ceilings_cross_at_a_tenth_of_a_second():
assert safe_rate(0.09)["binding"] == "points"
assert safe_rate(0.11)["binding"] == "cpu"
def test_a_very_expensive_endpoint_collapses_to_a_handful_a_minute():
assert safe_rate(3.0)["per_minute"] == 30.0
def test_a_write_costs_five_points_so_its_ceiling_is_a_fifth():
assert safe_rate(0.01, points=5)["per_minute"] == 180.0
def test_a_zero_response_time_does_not_divide_by_zero():
safe = safe_rate(0.0)
assert safe["by_cpu"] is None
assert safe["binding"] == "points"
assert safe_rate("unmeasured")["per_minute"] == 900.0
def test_with_no_configured_rate_the_ceiling_is_simply_reported():
safe = safe_rate(0.5)
state, detail = verdict("/x", {}, safe)
assert state == "ceiling"
assert "180" in detail
def test_a_rate_above_the_ceiling_names_the_cap_that_binds():
safe = safe_rate(0.6)
state, detail = verdict("/x", {"max_seconds": 0.9}, safe, configured=400)
assert state == "over-budget"
assert "CPU" in detail
def test_a_rate_just_under_the_ceiling_is_not_reported_as_fine():
safe = safe_rate(0.6) # 150 a minute
state, detail = verdict("/x", {"max_seconds": 0.9}, safe, configured=130)
assert state == "near-budget"
assert "0.900 s" in detail
def test_an_expensive_path_is_flagged_even_at_a_low_rate():
safe = safe_rate(2.0) # 45 a minute
state, detail = verdict("/x", {"max_seconds": 2.4}, safe, configured=5)
assert state == "expensive"
assert "move work off" in detail
def test_a_cheap_path_at_a_modest_rate_is_clear():
state, detail = verdict("/x", {"max_seconds": 0.05}, safe_rate(0.04), configured=60)
assert state == "clear"
assert "900-points" in detail
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
pointsFor, costProfile, safeRate, verdict,
} from './github-endpoint-cost-audit.mjs';
test('reads cost one point and writes cost five', () => {
assert.equal(pointsFor('GET'), 1);
assert.equal(pointsFor('head'), 1);
assert.equal(pointsFor('OPTIONS'), 1);
assert.equal(pointsFor('patch'), 5);
assert.equal(pointsFor('delete'), 5);
});
test('an unknown method is charged the expensive rate', () => {
assert.equal(pointsFor('QUERY'), 5);
assert.equal(pointsFor(null), 5);
assert.equal(pointsFor(''), 5);
});
test('samples are grouped by path and averaged', () => {
const profile = costProfile([
{ path: '/a', method: 'GET', seconds: 0.1 },
{ path: '/a', method: 'GET', seconds: 0.3 },
{ path: '/b', method: 'GET', seconds: 1.0 },
]);
assert.equal(profile['/a'].calls, 2);
assert.equal(profile['/a'].mean_seconds, 0.2);
assert.equal(profile['/a'].max_seconds, 0.3);
assert.equal(profile['/b'].mean_seconds, 1);
});
test('a malformed sample is dropped rather than counted as instant', () => {
const profile = costProfile([
{ path: '/a', seconds: 0.5 },
{ path: '/a', seconds: 'slow' },
{ seconds: 0.5 },
{ path: '/a', seconds: -1 },
]);
assert.equal(profile['/a'].calls, 1);
assert.equal(profile['/a'].mean_seconds, 0.5);
});
test('no samples profile nothing', () => {
assert.deepEqual(costProfile([]), {});
assert.deepEqual(costProfile(null), {});
});
test('a fast endpoint is bound by points', () => {
const safe = safeRate(0.04);
assert.equal(safe.binding, 'points');
assert.equal(safe.per_minute, 900);
assert.equal(safe.by_cpu, 2250);
});
test('a slow endpoint is bound by CPU time instead', () => {
const safe = safeRate(0.6);
assert.equal(safe.binding, 'cpu');
assert.equal(safe.per_minute, 150);
});
test('the two ceilings cross at a tenth of a second', () => {
assert.equal(safeRate(0.09).binding, 'points');
assert.equal(safeRate(0.11).binding, 'cpu');
});
test('a very expensive endpoint collapses to a handful a minute', () => {
assert.equal(safeRate(3).per_minute, 30);
});
test('a write costs five points so its ceiling is a fifth', () => {
assert.equal(safeRate(0.01, 5).per_minute, 180);
});
test('a zero response time does not divide by zero', () => {
const safe = safeRate(0);
assert.equal(safe.by_cpu, null);
assert.equal(safe.binding, 'points');
assert.equal(safeRate('unmeasured').per_minute, 900);
});
test('with no configured rate the ceiling is simply reported', () => {
const [state, detail] = verdict('/x', {}, safeRate(0.5));
assert.equal(state, 'ceiling');
assert.match(detail, /180/);
});
test('a rate above the ceiling names the cap that binds', () => {
const [state, detail] = verdict('/x', { max_seconds: 0.9 }, safeRate(0.6), 400);
assert.equal(state, 'over-budget');
assert.match(detail, /CPU/);
});
test('a rate just under the ceiling is not reported as fine', () => {
const [state, detail] = verdict('/x', { max_seconds: 0.9 }, safeRate(0.6), 130);
assert.equal(state, 'near-budget');
assert.match(detail, /0\.900 s/);
});
test('an expensive path is flagged even at a low rate', () => {
const [state, detail] = verdict('/x', { max_seconds: 2.4 }, safeRate(2), 5);
assert.equal(state, 'expensive');
assert.match(detail, /move work off/);
});
test('a cheap path at a modest rate is clear', () => {
const [state, detail] = verdict('/x', { max_seconds: 0.05 }, safeRate(0.04), 60);
assert.equal(state, 'clear');
assert.match(detail, /900-points/);
});
FAQ
How do I know it is the points cap and not the CPU cap?
Compute both and take the smaller. 900 divided by the points per request gives the point ceiling, which is 900 for reads and 180 for writes. 90 divided by the mean response time in seconds gives the CPU ceiling. They cross at about a tenth of a second: faster than that and points bind, slower and CPU does. GitHub does not tell you which one fired, so knowing which is lower for your path is the closest you get to an answer.
Is response time really a fair proxy for CPU time?
It is an over-estimate, and GitHub's own documentation offers total response time as the way to approximate the CPU cap. Your measurement includes network latency and queueing that the server never spent, so the ceiling it produces is lower than the real one. That is the useful direction to be wrong in: you end up pacing slightly more conservatively than you strictly need to.
Why is only one endpoint failing when the token is shared?
Because this cap is per endpoint, unlike the hourly quota, which is per token. One expensive path can be throttled continuously while every other call on the same credentials succeeds. It is also the reason the repair is local: moving that one path to GraphQL, or to a list endpoint, or behind a cache, fixes it without touching anything else.
Does the retry itself count?
Yes, and this is how a one-minute problem becomes a ten-minute one. A retry is another request to the same expensive path inside the same minute, so it keeps the budget full and the window keeps re-arming. Sleep the full retry-after and pause that path rather than that item, because every other in-flight request to the same endpoint is about to be refused too.
What if I cannot make the endpoint any cheaper?
Then the ceiling is the schedule. If a path allows 150 requests a minute and you have 4,000 items, the job takes 27 minutes and no amount of parallelism changes that; parallelism only converts the wait into failures. Accepting the duration up front is usually cheaper than a job that appears to finish in nine minutes and quietly drops whatever was refused.
Related field notes
- Over 100 concurrent requests trips a limit
- Ignoring retry-after extends the throttle
- per_page is unset so every list costs more
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Rate limits for the REST API — GitHub Docs
- Best practices for using the REST API — GitHub Docs
- Rate limits and node limits for the GraphQL API — GitHub Docs
- Troubleshooting the REST API — GitHub Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.