Diagnostic GitHub API
incomplete_results is true and the search answer is partial
The same search runs every morning and the number it reports moves: 412 on Monday, 380 on Tuesday, 412 again on Wednesday. Nothing failed. Every response was a 200 with well-formed JSON and a sensible-looking list of items. The field that says the answer was cut short is sitting at the top of every one of those payloads, and no code in the pipeline has ever read it.
Search queries run against a server-side timeout. When a query outruns it, GitHub does not fail the request — it returns whatever it managed to find, with the top-level incomplete_results boolean set to true. The status is 200, the JSON is valid, and the only difference between a partial answer and a complete one is that flag.
So read it, on every /search/* response, and treat true as a retryable failure rather than as a result: do not count it, do not diff against it, and never cache it. If the flag keeps coming back on the same query, retrying is not the repair — narrow the query with repo:, org: or a created: range until it completes inside the timeout. This is not the thousand-result ceiling, which is a different, deterministic limit with its own note; a query well under a hundred matches can be flagged, and one with fifty thousand may not be.
The problem in plain words
A flaky number is the hardest kind of wrong to act on, because the first response to it is disbelief and the second is a workaround. The count moved, so the data moved, so somebody explains it: issues were closed overnight, the index is catching up, search is eventually consistent, it settles down. Every one of those explanations is plausible, none of them is checkable, and all of them let the pipeline carry on treating a partial answer as an answer.
It is worse when the consumer is a comparison rather than a report. A job that diffs today's search results against yesterday's, and acts on the difference, will open tickets, page people or revoke access based on rows that did not disappear — they were simply not found this morning. The flag was true and the code read items.
And a cache turns an intermittent problem into a permanent one. A partial answer stored under a key that does not encode its own partiality gets served for the rest of its lifetime as though it were the whole set. The one run in ten that timed out becomes the answer everybody sees, and by the time somebody notices, the request that produced it is long gone and unreproducible.
Why it happens
The timeout is served rather than raised. Search has a bounded amount of time to answer, and the design decision when it runs out is to return partial results with a flag rather than to return an error. That is defensible — a partial answer is usually more useful than a 504 — and it puts the entire burden on the client to read one boolean. There is no status code, no header and no error object to trip over.
It is not the thousand-result cap. The Search API also refuses to page past 1,000 results per query, which is a hard, deterministic ceiling that announces itself with a 422 and has its own note. This flag is a different animal: it is non-deterministic, it fires on the first page, and it fires on queries nowhere near the cap. The script reads total_count and says explicitly whether the ceiling could be an explanation, because being able to rule it out by name is what stops the two problems being conflated.
It is a property of the query, not of the API's health. Broad queries time out: no repo: or org:, no date range, a full-text term that matches across everything, a regular expression or a long list of ORed qualifiers. A narrow query with the same result count usually completes. That is why the durable repair is to make the question smaller rather than to ask it again, and why a query that is flagged on every attempt will not be fixed by any retry policy.
One run cannot tell you what you need to know. A single flagged response says this attempt was partial. It does not say whether the query is occasionally unlucky, in which case retrying is a genuine repair, or reliably too broad, in which case retrying is a way of spending your search bucket on the same partial answer. The distinction is only visible across repeated runs of the same query, which is why the check makes a few and reasons about the sequence.
An unflagged response can still be partial. If the item counts move between identical consecutive runs and no round was flagged, something is truncating or reordering underneath you regardless. The honest verdict there is the same as for a flagged round: treat it as a retry, do not cache it, and do not diff against it.
Search has its own rate bucket, so the check has to be small. Search does not draw on the core quota; it has a separate and much tighter allowance, and exhausting it is its own failure. The check therefore defaults to one query and three rounds, three requests in total, prints that number before it spends anything, and refuses to run a plan that would not fit in the bucket.
The fix, as a flow
The same query sent a small number of times with a pause between, because one run cannot separate a partial answer from a small one. Three fields are kept from each response and the rest is discarded: the flag, the reported total and the number of items actually delivered. The verdict is built from the sequence rather than from any one response, which is what lets it say whether retrying is a repair or a waste, and the total is checked against the retrievable ceiling purely so the script can rule that explanation out by name.
How to fix it
Read the flag on every search response
incomplete_results is a top-level boolean next to total_count and items. One branch. If it is true, the list you are holding is a subset of the matches and you do not know which subset. This is the entire fix for most integrations and it takes a line.
Run the same query a few times before concluding anything
One flagged response tells you this attempt was cut short. Three tell you whether the query is occasionally unlucky or reliably too broad, and those two states have opposite repairs. The check defaults to three rounds with a short pause between them, which is three of your search requests.
Rule the thousand-result ceiling out by name
Read total_count and say plainly whether it is above 1,000. If it is not, the ceiling cannot be the explanation and nobody needs to spend an afternoon on it. If it is, you have two separate problems that happen to look alike from the outside, and they need repairing separately.
Narrow the query rather than retrying it
Where every round is flagged, no retry policy will help. Add a scope (repo: or org:), add a date range (created: or updated:), or split one broad query into several narrow ones and union the results yourself. The script reports which of those qualifiers your query does not already have.
Make a flagged response uncacheable and undiffable
Anything with the flag set must not be written to a cache, compared against a previous run, or used to decide that something has disappeared. A partial answer stored without a note of its partiality becomes the permanent truth, and a diff against a partial answer manufactures deletions that never happened.
How to check it worked
After the query is scoped to one organization and a date range, the same probe reports three clean rounds with a stable count and nothing to retry.
GITHUB_TOKEN=$READ_ONLY_TOKEN python3 github_search_incomplete.py \
--query 'is:issue is:open label:security'
# read cost: 3 search request(s) of the 30 per minute search bucket
# round 1: 412 item(s), total_count 412, incomplete_results=true
# round 2: 380 item(s), total_count 412, incomplete_results=true
# round 3: 412 item(s), total_count 412, incomplete_results=false
# timed-out-intermittent: 2 of 3 rounds came back partial. total_count is 412,
# well inside the 1000-result ceiling, so the ceiling is not the explanation
# missing from the query: repo: or org:, created: or updated: date range
# repair: treat a flagged response as a retry, never as a result, and never cache it
# after the query is scoped
GITHUB_TOKEN=$READ_ONLY_TOKEN python3 github_search_incomplete.py \
--query 'org:acme is:issue is:open label:security created:>2026-01-01'
# complete: 3 of 3 rounds were unflagged and the item count did not moveThe full code
The same query sent a small number of times with a pause between, because a single response cannot separate a partial answer from a small one. Three fields are kept from each round and the rest of the payload is discarded: the flag, the reported total and the number of items actually delivered. Everything after that is a pure function over the sequence, which is what lets the script say whether retrying is a repair or a waste. The reported total is checked against the retrievable ceiling for one reason only, which is so the script can rule that explanation out by name rather than leaving two similar-looking problems tangled together.
"""Say whether a GitHub search is being answered in part and nobody noticed.
Read only. One GET per round against /search/*, three rounds by default, with a
pause between them. Nothing is written.
Search runs against a server-side timeout. When a query outruns it GitHub does
not fail: it returns what it found with the top-level incomplete_results boolean
set to true. Status 200, valid JSON, fewer items, no error anywhere. A client
that reads items and ignores the flag treats a partial answer as a complete one,
and the count quietly moves between runs.
This is not the 1,000-result ceiling. That limit is deterministic, announces
itself with a 422 when you page past it, and applies to queries with more than a
thousand matches. This flag is non-deterministic, arrives on the first page, and
fires on queries nowhere near the cap. total_count is read here purely so the
ceiling can be ruled out by name.
Search has its own rate bucket, separate from and much tighter than core, so
this check is deliberately tiny and refuses a plan that would not fit in it.
Environment:
GITHUB_TOKEN any token with read access; search needs authentication to
get the larger of the two search buckets
"""
import argparse
import json
import logging
import os
import re
import sys
import time
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("github_search_incomplete")
API = "https://api.github.com"
UA = "github-search-incomplete/1.0"
# The other Search limit, read only so it can be excluded as an explanation.
RESULT_CAP = 1000
# Authenticated search requests per minute. The check is sized against this.
SEARCH_BUCKET = 30
QUALIFIER = re.compile(r"(?:^|\s)-?([A-Za-z_]+):\S")
SCOPES = ("repo", "org", "user")
RANGES = ("created", "updated", "merged", "closed")
def flagged(body):
"""Whether this response says it is partial. Pure."""
return bool(isinstance(body, dict) and body.get("incomplete_results") is True)
def total_of(body):
"""The reported match count, or None. Pure."""
if not isinstance(body, dict):
return None
try:
return int(body.get("total_count"))
except (TypeError, ValueError):
return None
def item_count(body):
"""How many items were actually delivered. Pure."""
if not isinstance(body, dict):
return 0
items = body.get("items")
return len(items) if isinstance(items, list) else 0
def cacheable(body):
"""Whether this response may be stored. Pure.
The rule the whole note comes down to: a partial answer written to a cache
without a note of its partiality becomes the permanent truth.
"""
return isinstance(body, dict) and not flagged(body)
def above_result_cap(total):
"""Whether the ceiling could also be in play here. Pure."""
try:
return int(total) > RESULT_CAP
except (TypeError, ValueError):
return False
def qualifiers(query):
"""The qualifier names used in a search query. Pure."""
return set(QUALIFIER.findall(" " + str(query or "")))
def narrowing(query):
"""Which narrowing devices the query is not already using. Pure."""
have = qualifiers(query)
out = []
if not have & set(SCOPES):
out.append("repo: or org:")
if not have & set(RANGES):
out.append("created: or updated: date range")
if "language" not in have:
out.append("language:")
return out
def observe(body):
"""The three fields worth keeping from one response. Pure."""
return {"incomplete": flagged(body),
"total": total_of(body),
"items": item_count(body)}
def summarise(observations):
"""Counts over the sequence of rounds. Pure."""
obs = list(observations or [])
return {"rounds": len(obs),
"flagged": sum(1 for o in obs if o.get("incomplete")),
"item_counts": [o.get("items") for o in obs],
"totals": [o.get("total") for o in obs]}
def counts_stable(observations):
"""Whether identical queries returned identical item counts. Pure."""
counts = [o.get("items") for o in (observations or [])]
return len(set(counts)) <= 1
def max_total(observations):
"""The largest reported match count across the rounds, or None. Pure."""
totals = [o.get("total") for o in (observations or [])
if o.get("total") is not None]
return max(totals) if totals else None
def verdict(observations):
"""Classify the sequence. Pure. Returns (state, detail).
Built from the sequence rather than from any one response, because a single
flagged round cannot distinguish an unlucky query from a hopeless one, and
those two have opposite repairs.
"""
s = summarise(observations)
if not s["rounds"]:
return ("no-observations", "no round completed, so there is nothing to judge.")
top = max_total(observations)
ceiling = (" total_count is %s, well inside the %d-result ceiling, so the "
"ceiling is not the explanation." % (top, RESULT_CAP)
if top is not None and not above_result_cap(top) else "")
if s["flagged"] and top is not None and above_result_cap(top):
return ("timed-out-and-capped",
"%d of %d round(s) came back partial and total_count is %s, "
"which is also above the %d-result ceiling. These are two "
"separate problems that look alike from outside and need "
"repairing separately."
% (s["flagged"], s["rounds"], top, RESULT_CAP))
if s["flagged"] == s["rounds"]:
return ("timed-out-always",
"every one of %d round(s) came back partial, so this query does "
"not finish inside the search timeout. No retry policy will fix "
"that.%s" % (s["rounds"], ceiling))
if s["flagged"]:
return ("timed-out-intermittent",
"%d of %d round(s) came back partial, so the query sometimes "
"finishes and sometimes does not. A flagged response is a retry, "
"not a result.%s" % (s["flagged"], s["rounds"], ceiling))
if not counts_stable(observations):
return ("unstable-counts",
"no round was flagged, but identical queries returned %s "
"item(s) across the rounds. Something is truncating or "
"reordering underneath you, and the answer should be treated "
"the same way as a flagged one."
% " and ".join(str(c) for c in sorted(set(s["item_counts"]))))
return ("complete",
"%d of %d round(s) were unflagged and the item count did not move."
% (s["rounds"], s["rounds"]))
def retry_or_narrow(observations):
"""What actually helps: retry, narrow, or nothing. Pure."""
state = verdict(observations)[0]
if state in ("timed-out-always", "timed-out-and-capped"):
return "narrow"
if state in ("timed-out-intermittent", "unstable-counts"):
return "retry"
return "nothing"
def repair(state, query=""):
"""The sentence a reader has to act on. Pure."""
missing = ", ".join(narrowing(query)) or "nothing obvious"
if state == "timed-out-always":
return ("narrow the query until it finishes: add %s. Retrying will "
"spend your search bucket on the same partial answer." % missing)
if state == "timed-out-and-capped":
return ("narrow the query until it finishes and until each slice reports "
"under %d matches: add %s, then union the slices yourself."
% (RESULT_CAP, missing))
if state == "timed-out-intermittent":
return ("treat a flagged response as a retry, never as a result, and "
"never cache it. If the flag keeps coming back, add %s." % missing)
if state == "unstable-counts":
return ("treat this the same as a flagged response: do not cache it and "
"do not diff against it. A moving count with no flag is still a "
"moving count.")
return "nothing."
def read_cost(queries, rounds):
"""Search requests this run will spend. Pure."""
try:
return max(0, len(queries or [])) * max(0, int(rounds))
except (TypeError, ValueError):
return 0
def within_search_bucket(cost):
"""Whether a plan of this size fits the per-minute search allowance. Pure."""
try:
return 0 < int(cost) <= SEARCH_BUCKET
except (TypeError, ValueError):
return False
def search(session, kind, query, per_page):
"""One search GET. Returns the decoded body or None."""
r = session.get("%s/search/%s" % (API, kind),
params={"q": query, "per_page": per_page}, timeout=30)
if r.status_code == 401:
raise SystemExit("401 from GitHub: GITHUB_TOKEN is missing, malformed "
"or revoked")
if r.status_code == 403:
raise SystemExit("403 from search. The search bucket is separate from "
"core and much tighter; wait for the window to reset")
if r.status_code == 422:
raise SystemExit("422 from search: the query is invalid, or you paged "
"past the %d-result ceiling" % RESULT_CAP)
if r.status_code != 200:
log.info("search returned %d", r.status_code)
return None
try:
return r.json()
except ValueError:
return None
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--query", required=True, help="the search query, as sent")
ap.add_argument("--kind", default="issues",
help="issues, repositories, code, users, commits")
ap.add_argument("--rounds", type=int, default=3,
help="how many times to run the same query")
ap.add_argument("--pause", type=float, default=2.0,
help="seconds between rounds")
ap.add_argument("--per-page", type=int, default=100,
help="page size for each round")
args = ap.parse_args()
token = os.environ.get("GITHUB_TOKEN")
if not token:
log.error("set GITHUB_TOKEN. Unauthenticated search gets a much smaller "
"bucket and this check would exhaust it")
return 2
cost = read_cost([args.query], args.rounds)
if not within_search_bucket(cost):
log.error("%d request(s) does not fit the %d per minute search bucket; "
"lower --rounds", cost, SEARCH_BUCKET)
return 2
log.info("read cost: %d search request(s) of the %d per minute search bucket",
cost, SEARCH_BUCKET)
session = requests.Session()
session.headers.update({
"Authorization": "Bearer " + token,
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
# GitHub rejects requests with no User-Agent outright.
"User-Agent": UA,
})
observations = []
for n in range(args.rounds):
if n:
time.sleep(args.pause)
body = search(session, args.kind, args.query, args.per_page)
if body is None:
continue
o = observe(body)
observations.append(o)
log.info("round %d: %d item(s), total_count %s, incomplete_results=%s",
n + 1, o["items"], o["total"], str(o["incomplete"]).lower())
if not cacheable(body):
log.info("round %d must not be cached or diffed against", n + 1)
state, detail = verdict(observations)
log.info("%s: %s", state, detail)
missing = narrowing(args.query)
if missing:
log.info("missing from the query: %s", ", ".join(missing))
log.info("what helps: %s", retry_or_narrow(observations))
log.info("repair: %s", repair(state, args.query))
print(json.dumps({"query": args.query,
"requests_spent": len(observations),
"summary": summarise(observations),
"counts_stable": counts_stable(observations),
"total_above_cap": above_result_cap(max_total(observations)),
"qualifiers_used": sorted(qualifiers(args.query)),
"narrowing_available": missing,
"action": retry_or_narrow(observations),
"state": state,
"detail": detail}, indent=2, default=str))
return 1 if state not in ("complete", "no-observations") else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Say whether a GitHub search is being answered in part and nobody noticed.
*
* Read only. One GET per round against /search/*, three rounds by default,
* with a pause between them. Nothing is written.
*
* Search runs against a server-side timeout. When a query outruns it, GitHub
* returns what it found with incomplete_results set to true rather than
* failing. This is not the 1,000-result ceiling: total_count is read here
* purely so that ceiling can be ruled out by name.
*
* Environment:
* GITHUB_TOKEN any token with read access
* GITHUB_QUERY the search query, as sent
* GITHUB_KIND issues, repositories, code, users, commits. Default issues
* GITHUB_ROUNDS how many times to run the same query. Default 3
*/
const API = 'https://api.github.com';
const UA = 'github-search-incomplete/1.0';
/** The other Search limit, read only so it can be excluded as an explanation. */
export const RESULT_CAP = 1000;
/** Authenticated search requests per minute. The check is sized against this. */
export const SEARCH_BUCKET = 30;
const QUALIFIER = /(?:^|\s)-?([A-Za-z_]+):\S/g;
export const SCOPES = ['repo', 'org', 'user'];
export const RANGES = ['created', 'updated', 'merged', 'closed'];
/** Whether this response says it is partial. Pure. */
export function flagged(body) {
return Boolean(body && typeof body === 'object' && body.incomplete_results === true);
}
/** The reported match count, or null. Pure. */
export function totalOf(body) {
if (!body || typeof body !== 'object') return null;
const raw = body.total_count;
if (raw === null || raw === undefined || raw === '') return null;
const n = Number(raw);
return Number.isFinite(n) ? Math.trunc(n) : null;
}
/** How many items were actually delivered. Pure. */
export function itemCount(body) {
if (!body || typeof body !== 'object') return 0;
return Array.isArray(body.items) ? body.items.length : 0;
}
/** Whether this response may be stored. Pure. */
export function cacheable(body) {
return Boolean(body && typeof body === 'object') && !flagged(body);
}
/** Whether the ceiling could also be in play here. Pure. */
export function aboveResultCap(total) {
const n = Number(total);
return Number.isFinite(n) && n > RESULT_CAP;
}
/** The qualifier names used in a search query. Pure. */
export function qualifiers(query) {
const out = new Set();
for (const m of ` ${String(query ?? '')}`.matchAll(QUALIFIER)) out.add(m[1]);
return out;
}
/** Which narrowing devices the query is not already using. Pure. */
export function narrowing(query) {
const have = qualifiers(query);
const out = [];
if (!SCOPES.some((s) => have.has(s))) out.push('repo: or org:');
if (!RANGES.some((s) => have.has(s))) out.push('created: or updated: date range');
if (!have.has('language')) out.push('language:');
return out;
}
/** The three fields worth keeping from one response. Pure. */
export function observe(body) {
return { incomplete: flagged(body), total: totalOf(body), items: itemCount(body) };
}
/** Counts over the sequence of rounds. Pure. */
export function summarise(observations) {
const obs = Array.isArray(observations) ? observations : [];
return {
rounds: obs.length,
flagged: obs.filter((o) => o && o.incomplete).length,
item_counts: obs.map((o) => (o || {}).items),
totals: obs.map((o) => (o || {}).total),
};
}
/** Whether identical queries returned identical item counts. Pure. */
export function countsStable(observations) {
const counts = (Array.isArray(observations) ? observations : []).map((o) => (o || {}).items);
return new Set(counts).size <= 1;
}
/** The largest reported match count across the rounds, or null. Pure. */
export function maxTotal(observations) {
const totals = (Array.isArray(observations) ? observations : [])
.map((o) => (o || {}).total).filter((t) => t !== null && t !== undefined);
return totals.length ? Math.max(...totals) : null;
}
/** Classify the sequence. Pure. Returns [state, detail]. */
export function verdict(observations) {
const s = summarise(observations);
if (!s.rounds) return ['no-observations', 'no round completed, so there is nothing to judge.'];
const top = maxTotal(observations);
const ceiling = top !== null && !aboveResultCap(top)
? ` total_count is ${top}, well inside the ${RESULT_CAP}-result ceiling, so the ceiling is not the explanation.`
: '';
if (s.flagged && top !== null && aboveResultCap(top)) {
return ['timed-out-and-capped',
`${s.flagged} of ${s.rounds} round(s) came back partial and total_count is `
+ `${top}, which is also above the ${RESULT_CAP}-result ceiling. These are `
+ 'two separate problems that look alike from outside and need repairing separately.'];
}
if (s.flagged === s.rounds) {
return ['timed-out-always',
`every one of ${s.rounds} round(s) came back partial, so this query does not `
+ `finish inside the search timeout. No retry policy will fix that.${ceiling}`];
}
if (s.flagged) {
return ['timed-out-intermittent',
`${s.flagged} of ${s.rounds} round(s) came back partial, so the query `
+ 'sometimes finishes and sometimes does not. A flagged response is a '
+ `retry, not a result.${ceiling}`];
}
if (!countsStable(observations)) {
const seen = [...new Set(s.item_counts)].sort((a, b) => a - b).join(' and ');
return ['unstable-counts',
`no round was flagged, but identical queries returned ${seen} item(s) across `
+ 'the rounds. Something is truncating or reordering underneath you, and the '
+ 'answer should be treated the same way as a flagged one.'];
}
return ['complete',
`${s.rounds} of ${s.rounds} round(s) were unflagged and the item count did not move.`];
}
/** What actually helps: retry, narrow, or nothing. Pure. */
export function retryOrNarrow(observations) {
const state = verdict(observations)[0];
if (['timed-out-always', 'timed-out-and-capped'].includes(state)) return 'narrow';
if (['timed-out-intermittent', 'unstable-counts'].includes(state)) return 'retry';
return 'nothing';
}
/** The sentence a reader has to act on. Pure. */
export function repair(state, query = '') {
const missing = narrowing(query).join(', ') || 'nothing obvious';
if (state === 'timed-out-always') {
return `narrow the query until it finishes: add ${missing}. Retrying will `
+ 'spend your search bucket on the same partial answer.';
}
if (state === 'timed-out-and-capped') {
return 'narrow the query until it finishes and until each slice reports under '
+ `${RESULT_CAP} matches: add ${missing}, then union the slices yourself.`;
}
if (state === 'timed-out-intermittent') {
return 'treat a flagged response as a retry, never as a result, and never '
+ `cache it. If the flag keeps coming back, add ${missing}.`;
}
if (state === 'unstable-counts') {
return 'treat this the same as a flagged response: do not cache it and do not '
+ 'diff against it. A moving count with no flag is still a moving count.';
}
return 'nothing.';
}
/** Search requests this run will spend. Pure. */
export function readCost(queries, rounds) {
const q = Array.isArray(queries) ? queries.length : 0;
const r = Number(rounds);
return q * (Number.isFinite(r) && r > 0 ? Math.trunc(r) : 0);
}
/** Whether a plan of this size fits the per-minute search allowance. Pure. */
export function withinSearchBucket(cost) {
const n = Number(cost);
return Number.isFinite(n) && n > 0 && n <= SEARCH_BUCKET;
}
function headers(token) {
return {
Authorization: `Bearer ${token}`,
Accept: 'application/vnd.github+json',
'X-GitHub-Api-Version': '2022-11-28',
'User-Agent': UA,
};
}
const wait = (ms) => new Promise((resolve) => { setTimeout(resolve, ms); });
async function main() {
const token = process.env.GITHUB_TOKEN;
const query = process.env.GITHUB_QUERY;
if (!token || !query) {
console.error('set GITHUB_TOKEN (read-only is enough) and GITHUB_QUERY');
process.exitCode = 2;
return;
}
const kind = process.env.GITHUB_KIND || 'issues';
const rounds = Number(process.env.GITHUB_ROUNDS || 3);
const cost = readCost([query], rounds);
if (!withinSearchBucket(cost)) {
console.error(`${cost} request(s) does not fit the ${SEARCH_BUCKET} per minute search bucket`);
process.exitCode = 2;
return;
}
console.log(`read cost: ${cost} search request(s) of the ${SEARCH_BUCKET} per minute search bucket`);
const observations = [];
for (let n = 0; n < rounds; n += 1) {
if (n) await wait(2000);
const url = new URL(`${API}/search/${kind}`);
url.searchParams.set('q', query);
url.searchParams.set('per_page', '100');
const res = await fetch(url, { headers: headers(token) });
if (res.status !== 200) {
console.error(`search returned ${res.status}`);
continue;
}
let body = null;
try { body = await res.json(); } catch { body = null; }
if (body === null) continue;
const o = observe(body);
observations.push(o);
console.log(`round ${n + 1}: ${o.items} item(s), total_count ${o.total}, `
+ `incomplete_results=${o.incomplete}`);
if (!cacheable(body)) console.log(`round ${n + 1} must not be cached or diffed against`);
}
const [state, detail] = verdict(observations);
console.log(`${state}: ${detail}`);
const missing = narrowing(query);
if (missing.length) console.log(`missing from the query: ${missing.join(', ')}`);
console.log(`what helps: ${retryOrNarrow(observations)}`);
console.log(`repair: ${repair(state, query)}`);
console.log(JSON.stringify({
query,
requests_spent: observations.length,
summary: summarise(observations),
counts_stable: countsStable(observations),
total_above_cap: aboveResultCap(maxTotal(observations)),
qualifiers_used: [...qualifiers(query)].sort(),
narrowing_available: missing,
action: retryOrNarrow(observations),
state,
detail,
}, null, 2));
process.exitCode = ['complete', 'no-observations'].includes(state) ? 0 : 1;
}
// Guarded so importing this file from the test runner does not start a run and
// fail the suite even as every test passes.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The sequence logic carries the note, so it is what gets tested hardest: a query flagged on every round and one flagged on some of them have to reach different verdicts, because the first is narrowed and the second is retried and telling a reader to do the wrong one wastes a week. Beside that, the ceiling is asserted to be excluded by name on queries under a thousand matches and reported as a second, separate problem above it; a moving count with no flag has to be caught at all; the flag has to be strictly true rather than merely truthy; and the caching rule is asserted directly, since it is the sentence most likely to be skipped on the way to the code.
from github_search_incomplete import (
RESULT_CAP, SEARCH_BUCKET, above_result_cap, cacheable, counts_stable,
flagged, item_count, max_total, narrowing, observe, qualifiers, read_cost,
repair, retry_or_narrow, summarise, total_of, verdict, within_search_bucket,
)
PARTIAL = {"total_count": 412, "incomplete_results": True,
"items": [{"id": 1}, {"id": 2}]}
WHOLE = {"total_count": 412, "incomplete_results": False,
"items": [{"id": 1}, {"id": 2}, {"id": 3}]}
def test_the_flag_is_read_strictly_rather_than_truthily():
assert flagged(PARTIAL)
assert not flagged(WHOLE)
assert not flagged({"incomplete_results": "true"})
assert not flagged({"incomplete_results": 1})
assert not flagged({})
assert not flagged(None)
def test_the_three_kept_fields_survive_a_malformed_payload():
assert observe(PARTIAL) == {"incomplete": True, "total": 412, "items": 2}
assert total_of({"total_count": "412"}) == 412
assert total_of({"total_count": None}) is None
assert total_of([]) is None
assert item_count({"items": None}) == 0
assert item_count(None) == 0
def test_a_flagged_response_may_never_be_cached():
assert not cacheable(PARTIAL)
assert cacheable(WHOLE)
assert not cacheable(None)
def test_every_round_partial_is_narrowed_not_retried():
obs = [observe(PARTIAL)] * 3
state, detail = verdict(obs)
assert state == "timed-out-always"
assert "No retry policy will fix that" in detail
assert retry_or_narrow(obs) == "narrow"
def test_some_rounds_partial_is_retried_not_narrowed():
obs = [observe(PARTIAL), observe(WHOLE), observe(WHOLE)]
state, detail = verdict(obs)
assert state == "timed-out-intermittent"
assert "1 of 3" in detail
assert retry_or_narrow(obs) == "retry"
def test_the_thousand_result_ceiling_is_ruled_out_by_name():
detail = verdict([observe(PARTIAL)] * 2)[1]
assert "1000-result ceiling" in detail
assert "not the explanation" in detail
def test_a_query_over_the_ceiling_is_reported_as_two_problems():
big = dict(PARTIAL, total_count=24831)
state, detail = verdict([observe(big)] * 2)
assert state == "timed-out-and-capped"
assert "two separate problems" in detail
assert retry_or_narrow([observe(big)] * 2) == "narrow"
def test_a_moving_count_with_no_flag_is_still_caught():
obs = [observe(WHOLE), observe(dict(WHOLE, items=[{"id": 1}]))]
state, detail = verdict(obs)
assert state == "unstable-counts"
assert "no round was flagged" in detail
assert retry_or_narrow(obs) == "retry"
def test_three_clean_stable_rounds_are_not_a_finding():
obs = [observe(WHOLE)] * 3
assert verdict(obs)[0] == "complete"
assert retry_or_narrow(obs) == "nothing"
assert counts_stable(obs)
def test_no_rounds_is_not_reported_as_a_clean_result():
assert verdict([])[0] == "no-observations"
assert summarise([]) == {"rounds": 0, "flagged": 0, "item_counts": [], "totals": []}
assert max_total([]) is None
def test_the_ceiling_predicate_is_strictly_above_the_cap():
assert above_result_cap(RESULT_CAP + 1)
assert not above_result_cap(RESULT_CAP)
assert not above_result_cap(None)
def test_the_query_is_read_for_the_qualifiers_it_already_has():
assert qualifiers("is:issue repo:acme/api label:bug") == {"is", "repo", "label"}
assert qualifiers("-org:acme is:open") == {"org", "is"}
assert qualifiers("") == set()
assert qualifiers(None) == set()
def test_narrowing_suggests_only_what_is_missing():
assert narrowing("is:issue state:open") == [
"repo: or org:", "created: or updated: date range", "language:"]
assert narrowing("org:acme created:>2026-01-01 language:go") == []
assert narrowing("repo:acme/api updated:>2026-01-01") == ["language:"]
def test_the_repair_tells_a_hopeless_query_not_to_retry():
fix = repair("timed-out-always", "is:issue state:open")
assert "narrow the query" in fix
assert "repo: or org:" in fix
assert "Retrying will" in fix
assert "never cache it" in repair("timed-out-intermittent", "is:issue")
def test_the_check_refuses_a_plan_that_would_not_fit_the_search_bucket():
assert read_cost(["q"], 3) == 3
assert read_cost(["a", "b"], 4) == 8
assert read_cost([], 3) == 0
assert within_search_bucket(3)
assert within_search_bucket(SEARCH_BUCKET)
assert not within_search_bucket(SEARCH_BUCKET + 1)
assert not within_search_bucket(0)
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
RESULT_CAP, SEARCH_BUCKET, aboveResultCap, cacheable, countsStable, flagged,
itemCount, maxTotal, narrowing, observe, qualifiers, readCost, repair,
retryOrNarrow, summarise, totalOf, verdict, withinSearchBucket,
} from './github-search-incomplete.mjs';
const PARTIAL = { total_count: 412, incomplete_results: true, items: [{ id: 1 }, { id: 2 }] };
const WHOLE = {
total_count: 412, incomplete_results: false, items: [{ id: 1 }, { id: 2 }, { id: 3 }],
};
test('the flag is read strictly rather than truthily', () => {
assert.ok(flagged(PARTIAL));
assert.ok(!flagged(WHOLE));
assert.ok(!flagged({ incomplete_results: 'true' }));
assert.ok(!flagged({ incomplete_results: 1 }));
assert.ok(!flagged({}));
assert.ok(!flagged(null));
});
test('the three kept fields survive a malformed payload', () => {
assert.deepEqual(observe(PARTIAL), { incomplete: true, total: 412, items: 2 });
assert.equal(totalOf({ total_count: '412' }), 412);
assert.equal(totalOf({ total_count: null }), null);
assert.equal(itemCount({ items: null }), 0);
assert.equal(itemCount(null), 0);
});
test('a flagged response may never be cached', () => {
assert.ok(!cacheable(PARTIAL));
assert.ok(cacheable(WHOLE));
assert.ok(!cacheable(null));
});
test('every round partial is narrowed, not retried', () => {
const obs = [observe(PARTIAL), observe(PARTIAL), observe(PARTIAL)];
const [state, detail] = verdict(obs);
assert.equal(state, 'timed-out-always');
assert.match(detail, /No retry policy will fix that/);
assert.equal(retryOrNarrow(obs), 'narrow');
});
test('some rounds partial is retried, not narrowed', () => {
const obs = [observe(PARTIAL), observe(WHOLE), observe(WHOLE)];
const [state, detail] = verdict(obs);
assert.equal(state, 'timed-out-intermittent');
assert.match(detail, /1 of 3/);
assert.equal(retryOrNarrow(obs), 'retry');
});
test('the thousand-result ceiling is ruled out by name', () => {
const detail = verdict([observe(PARTIAL), observe(PARTIAL)])[1];
assert.match(detail, /1000-result ceiling/);
assert.match(detail, /not the explanation/);
});
test('a query over the ceiling is reported as two problems', () => {
const big = { ...PARTIAL, total_count: 24831 };
const obs = [observe(big), observe(big)];
const [state, detail] = verdict(obs);
assert.equal(state, 'timed-out-and-capped');
assert.match(detail, /two separate problems/);
assert.equal(retryOrNarrow(obs), 'narrow');
});
test('a moving count with no flag is still caught', () => {
const obs = [observe(WHOLE), observe({ ...WHOLE, items: [{ id: 1 }] })];
const [state, detail] = verdict(obs);
assert.equal(state, 'unstable-counts');
assert.match(detail, /no round was flagged/);
assert.equal(retryOrNarrow(obs), 'retry');
});
test('three clean stable rounds are not a finding', () => {
const obs = [observe(WHOLE), observe(WHOLE), observe(WHOLE)];
assert.equal(verdict(obs)[0], 'complete');
assert.equal(retryOrNarrow(obs), 'nothing');
assert.ok(countsStable(obs));
});
test('no rounds is not reported as a clean result', () => {
assert.equal(verdict([])[0], 'no-observations');
assert.deepEqual(summarise([]), {
rounds: 0, flagged: 0, item_counts: [], totals: [],
});
assert.equal(maxTotal([]), null);
});
test('the ceiling predicate is strictly above the cap', () => {
assert.ok(aboveResultCap(RESULT_CAP + 1));
assert.ok(!aboveResultCap(RESULT_CAP));
assert.ok(!aboveResultCap(null));
});
test('the query is read for the qualifiers it already has', () => {
assert.deepEqual([...qualifiers('is:issue repo:acme/api label:bug')].sort(),
['is', 'label', 'repo']);
assert.deepEqual([...qualifiers('-org:acme is:open')].sort(), ['is', 'org']);
assert.equal(qualifiers('').size, 0);
assert.equal(qualifiers(null).size, 0);
});
test('narrowing suggests only what is missing', () => {
assert.deepEqual(narrowing('is:issue state:open'),
['repo: or org:', 'created: or updated: date range', 'language:']);
assert.deepEqual(narrowing('org:acme created:>2026-01-01 language:go'), []);
assert.deepEqual(narrowing('repo:acme/api updated:>2026-01-01'), ['language:']);
});
test('the repair tells a hopeless query not to retry', () => {
const fix = repair('timed-out-always', 'is:issue state:open');
assert.match(fix, /narrow the query/);
assert.match(fix, /repo: or org:/);
assert.match(fix, /Retrying will/);
assert.match(repair('timed-out-intermittent', 'is:issue'), /never cache it/);
});
test('the check refuses a plan that would not fit the search bucket', () => {
assert.equal(readCost(['q'], 3), 3);
assert.equal(readCost(['a', 'b'], 4), 8);
assert.equal(readCost([], 3), 0);
assert.ok(withinSearchBucket(3));
assert.ok(withinSearchBucket(SEARCH_BUCKET));
assert.ok(!withinSearchBucket(SEARCH_BUCKET + 1));
assert.ok(!withinSearchBucket(0));
});
FAQ
Is this the same as the 1,000-result cap?
No, and keeping them apart is most of the value of the note. The cap is deterministic: it applies to queries with more than a thousand matches, it announces itself with a 422 when you page past it, and the fix is to partition the query into slices that each report under a thousand. This flag is non-deterministic, arrives on the very first page, and fires on queries with a few hundred matches. The script reads total_count and says in the output whether the ceiling could be an explanation, so nobody spends an afternoon on the wrong limit.
Should I just retry until the flag is false?
Only if it is sometimes false. A query that comes back flagged on every attempt is not unlucky, it is too broad to finish inside the timeout, and a retry loop against it will spend your search bucket producing the same partial answer more times. That is why the check runs the query more than once: the difference between sometimes and always is the difference between a retry policy and a rewritten query, and one run cannot tell you which you have.
Can I cache a response that had incomplete_results set?
No. It is the single rule most worth taking from this note. A partial answer written to a cache under a key that does not encode its own partiality becomes the permanent answer for the lifetime of that entry, and the run that produced it is long gone by the time anyone questions it. The same applies to diffing: comparing today's partial results against yesterday's complete ones manufactures deletions that never happened, and if something acts on those deletions the damage is real.
The count moves but the flag is always false. What then?
Treat it exactly like a flagged response. If two identical queries a few seconds apart return different item counts and nothing was flagged, something is truncating or reordering underneath you regardless of what the payload claims, and the answer is not stable enough to cache or to diff against. The check gives that its own state rather than reporting it as clean, because a clean verdict on a moving number is how a diagnostic loses somebody's trust.
How much of the search bucket does the check use?
One request per round, three by default. Search does not draw on the core hourly quota; it has a separate and much tighter allowance, roughly thirty requests a minute authenticated, and exhausting it is a failure with its own note. The script computes the plan up front, prints it, and refuses to run at all if the number of rounds you asked for would not fit inside the bucket, because a diagnostic that causes the failure next door is not worth having.
Related field notes
- Search returns at most 1,000 results
- The search rate bucket is exhausted
- Code search has its own tighter bucket
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Search — GitHub REST API
- Understanding the search syntax — GitHub Docs
- Rate limits for the REST API — GitHub Docs
- Best practices for using the REST API — GitHub Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.