Diagnostic GitHub API
GraphQL data is present but individual fields are null
The query asked for fifty repositories and fifty came back. Nothing threw, nothing logged, the job took the usual eleven seconds. Eight of the fifty have null where diskUsage should be, because the token cannot read that field on a private repository, and the total that gets written to the dashboard is the sum of forty-two numbers presented as the sum of fifty. The response told you which eight, in an errors array that arrived alongside perfectly good data.
GraphQL resolves every field independently. A field your token cannot see resolves to null and adds an entry to the top-level errors array with a type and, crucially, a path pointing at the exact field that was nulled. Everything else in the query succeeds. The response is genuinely partial by design, and it is a third outcome rather than a softer kind of failure.
The repair is not a retry, because the same token will produce the same nulls forever. Walk errors[].path, mark those fields unknown rather than zero, and refuse to publish an aggregate computed across them without saying how many were withheld. Then decide whether to widen the token or to accept the gap. The distinction that matters is between a null that has a matching entry in errors, which was withheld from you, and a null that has none, which is genuinely empty.
The problem in plain words
The client that hits this has usually already been fixed once. Somebody found out the hard way that a 200 can carry errors, added a check, and made it throw. That check is right for a response where data is null and wrong for this one, because throwing here discards forty-two repositories that resolved perfectly in order to react to eight that did not. So the check gets softened to a warning, the warning goes to a log nobody reads, and the aggregation runs over the whole array as though nothing happened.
The reason it stays hidden is that a null is a perfectly ordinary value in this API. A repository with no license has a null licenseInfo. An unassigned issue has a null assignee. Code that walks a response is full of null handling already, and one more null looks like all the others. There is no visual difference between a field that is empty and a field you were not allowed to read; the difference is recorded somewhere else entirely, in an array most code never looks at.
What it produces is the worst kind of wrong number: a plausible one. Not zero, not an exception, just a total that is quietly sixteen per cent low and moves whenever somebody makes a repository private. The finance-adjacent version of this is a storage report that under-bills. The security-adjacent version is a vulnerability count that misses the repositories the token cannot see, which are disproportionately the ones you would most want counted.
Why it happens
Field-level resolution is the whole design. A GraphQL server resolves each field on its own, so a permission failure on one field does not fail the document. The failed field takes the value null in data and contributes an entry to errors. That is the specification working as intended, and it is exactly why a client cannot decide anything from data alone.
path is the field that makes this actionable. Every execution error that belongs to a field carries a path array naming the route to it, list indices included, so ["repository", "pullRequests", "nodes", 3, "author"] tells you which element of which connection was withheld. Without reading path you know only that something went wrong somewhere; with it you know precisely which cells of your result set are unknown.
A null with an errors entry and a null without one mean opposite things. The first was withheld and its true value is unknown. The second is a real answer: the repository genuinely has no license, the issue genuinely has no milestone. Treating the first as the second is what produces the under-count, and it is the single distinction this script exists to draw. Every null it reports is labelled as one or the other, never as "a null".
This is not the note about a call that failed. When data is null and errors is non-empty, the query failed outright, the status was still 200, and that has its own note and its own repair. This note starts where that one ends: the call worked, most of the data is correct and usable, and throwing it away would be a bug of its own. Same envelope, opposite instruction.
Widening the token is a decision, not automatically the fix. Some of these fields need read:org, some need admin rights on the repository, and some are only visible to an installation with an explicit permission granted. Handing a reporting job the permissions it would need to see everything is sometimes right and sometimes a much larger problem than an under-count. The script tells you which permission each nulled field would want and leaves the choice with you.
The API cannot see your aggregation. Nothing GitHub returns says whether you summed across the nulls. What a script can do is run your own query shape against a repository you name, count how many of the requested fields resolved, and say which of the nulls were explained by an errors entry. That is a measurement of the response, not an audit of your code, and it is enough to tell you whether the trap is live for this token.
The fix, as a flow
One query, sent in the shape your integration sends it, because the nulls follow the fields and a simplified probe comes back clean. What follows is a set difference done in both directions: every path in the data tree that resolved to null, every path named in errors, and the two compared so a field that was withheld can never be mistaken for a field that is empty. The path resolver walks list indices as well as object keys, since real GraphQL error paths contain them. The last step asks the only question that matters downstream, which is whether a total computed over this response is still a total.
How to fix it
Send the query your integration actually sends
Not a simplified version. The nulls follow the fields, so a probe that asks for name and id will come back clean while the real query loses diskUsage and collaborators. One query, one point. Pass your own document with --query if the default does not resemble yours.
Walk errors[].path rather than counting errors
The number of errors is not the finding; the paths are. Each one names a field that resolved to null because the server refused it, list indices and all. Print them, because a list of eight dotted paths is a bug report somebody can act on and "8 errors" is not.
Separate the withheld nulls from the real ones
Walk the data tree, collect every path that is null, and subtract the paths that appear in errors. What is left is genuinely empty: no license, no milestone, no assignee. What matched is unknown. This is the step that turns an ambiguous response into two unambiguous sets, and every later decision depends on it.
Ask whether the aggregate is still safe to publish
Give the script the root you sum over and it will tell you how many withheld fields lie underneath it. If the answer is more than zero, the total is a lower bound and has to be labelled as one. Publishing it as a total is the actual harm here; the nulls themselves are only the mechanism.
Decide on the permission, then keep the run cheap
For each nulled field the script names the permission that would let the token read it, so widening the token is a considered choice rather than a reflex. The audit costs one point per query, one by default, out of 5,000 an hour, and it prints that before it spends it. Retrying is never the repair: the same token produces the same nulls every time.
How to check it worked
Once the nulls are separated and the aggregate is labelled, the same run reports the gap instead of hiding it.
GITHUB_TOKEN=$READ_ONLY_TOKEN python3 github_graphql_partial.py --repo acme/monorepo --root repository
# point cost: 1 point(s) against the 5,000/hour GraphQL budget
# HTTP 200, errors=2, data usable=yes
# partial-withheld: 2 field(s) resolved to null and errors[].path explains both
# repository.diskUsage withheld FORBIDDEN wants: metadata + admin on the repository
# repository.collaborators withheld FORBIDDEN wants: read access to members
# repository.licenseInfo absent - genuinely empty, safe to read as none
# aggregation over 'repository' is NOT safe: 2 withheld field(s) underneath it
# repair: record the withheld paths as unknown, not zero, and label the total a
# lower bound. Do not retry: this token returns the same nulls every time.The full code
One query, sent as the integration sends it, and refused outright if the document turns out to contain a mutation or a subscription. What follows is a set-difference done properly: every path in the data tree that is null, every path named in errors, and the two sets compared so a withheld field can never be confused with an empty one. The path resolver handles list indices because real GraphQL error paths contain them. Nothing about the rule needs a network, so the entire classification is tested against recorded envelopes, including the ones that are inconvenient to produce on purpose.
"""Separate the fields a GraphQL response withheld from the ones that are empty.
Read only, and queries only. GitHub's GraphQL endpoint takes a document in the
request body, so a read is carried by POST there just as a write would be; that
is transport, not intent. This script sends queries and refuses any document
containing a mutation or a subscription before it opens a socket. Nothing is
written and the repair is printed rather than performed.
GraphQL resolves each field independently. A field the token cannot see becomes
null in data and adds an entry to errors carrying a path that names it exactly,
while the rest of the response succeeds. The response is genuinely partial. The
danger is that a withheld null and a real null look identical in the data and
mean opposite things: unknown versus none.
What this can and cannot see: the API has no idea whether your code aggregates
across the nulls. It can measure this response, name which nulls were explained
by an errors entry and say whether a sum over a given root is still honest.
Environment:
GITHUB_TOKEN a token with read access to the GraphQL API
"""
import argparse
import json
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("github_graphql_partial")
API = "https://api.github.com"
UA = "github-graphql-partial/1.0"
POINTS_PER_QUERY = 1
# Deliberately a mixture. Two of these fields commonly resolve to null because a
# read-only token is not allowed to see them, and one commonly resolves to null
# because the repository really has nothing there. Telling those apart is the
# whole job.
DEFAULT_QUERY = (
"query($owner: String!, $name: String!) {"
" repository(owner: $owner, name: $name) {"
" name isPrivate diskUsage"
" licenseInfo { key }"
" collaborators(first: 1) { totalCount }"
" } }"
)
# What a withheld field would need to be readable. Not a promise that granting
# it is the right move: some of these are a much bigger decision than an
# under-count, which is exactly why the script names them instead of advising.
PERMISSION_HINT = {
"diskUsage": "metadata read plus admin on the repository",
"collaborators": "read access to repository members",
"vulnerabilityAlerts": "Dependabot alerts read",
"projectsV2": "organization projects read",
"members": "read:org on the organization",
"email": "user email read, and the user must have a public email",
}
MISSING = object()
def strip_noise(document):
"""Remove GraphQL comments and string literals from a document. Pure."""
src = str(document or "")
out = []
i, n = 0, len(src)
while i < n:
ch = src[i]
if ch == "#":
while i < n and src[i] != "\n":
i += 1
continue
if src.startswith('"""', i):
j = src.find('"""', i + 3)
i = n if j < 0 else j + 3
out.append(" ")
continue
if ch == '"':
i += 1
while i < n and src[i] != '"':
i += 2 if src[i] == "\\" else 1
i += 1
out.append(" ")
continue
out.append(ch)
i += 1
return "".join(out)
def operations(document):
"""The top-level operations in a document, in order. Pure."""
src = strip_noise(document)
ops, depth, word, declared = [], 0, "", None
for ch in src + " ":
if ch.isalnum() or ch == "_":
word += ch
continue
if word:
if depth == 0 and word in ("query", "mutation", "subscription", "fragment"):
declared = word
word = ""
if ch == "{":
if depth == 0:
ops.append(declared or "query")
declared = None
depth += 1
elif ch == "}":
depth = max(0, depth - 1)
return ops
def refusal(document):
"""Why this document will not be sent, or None if it is a read. Pure."""
ops = operations(document)
if not ops:
return "the document contains no operation to send."
for kind in ("mutation", "subscription"):
if kind in ops:
return ("the document contains a %s. This script sends queries only: "
"a query is a read, and the section it belongs to promises "
"its scripts never write." % kind)
return None
def path_key(path):
"""A GraphQL error path rendered as a dotted string. Pure.
List indices are kept as segments, because an error on element 3 of a
connection is a different fact from an error on the connection.
"""
if isinstance(path, str):
return path
return ".".join(str(p) for p in (path or []))
def value_at(data, dotted):
"""Resolve a dotted path in a data tree. Pure. MISSING if there is no such path."""
cur = data
if not dotted:
return cur
for seg in str(dotted).split("."):
if isinstance(cur, dict):
if seg not in cur:
return MISSING
cur = cur[seg]
elif isinstance(cur, list):
try:
cur = cur[int(seg)]
except (ValueError, IndexError):
return MISSING
else:
return MISSING
return cur
def null_paths(data, prefix=""):
"""Every path in a data tree whose value is null. Pure."""
out = []
if isinstance(data, dict):
items = data.items()
elif isinstance(data, list):
items = ((str(i), v) for i, v in enumerate(data))
else:
return out
for key, value in items:
here = key if not prefix else prefix + "." + str(key)
if value is None:
out.append(here)
else:
out.extend(null_paths(value, here))
return sorted(out)
def error_paths(body):
"""Dotted paths named by the errors array, mapped to their type. Pure."""
out = {}
if not isinstance(body, dict):
return out
for err in body.get("errors") or []:
if not isinstance(err, dict) or not err.get("path"):
continue
out[path_key(err["path"])] = err.get("type") or "UNTYPED"
return out
def unpathed_errors(body):
"""Errors that name no field, so nothing can be attributed to them. Pure."""
if not isinstance(body, dict):
return 0
return sum(1 for e in (body.get("errors") or [])
if not isinstance(e, dict) or not e.get("path"))
def has_usable_data(body):
"""Whether any top-level field resolved to something other than null. Pure."""
if not isinstance(body, dict):
return False
data = body.get("data")
if not isinstance(data, dict):
return False
return any(v is not None for v in data.values())
def withheld(body):
"""Paths that are null in data and explained by an errors entry. Pure."""
if not isinstance(body, dict):
return []
named = error_paths(body)
nulls = set(null_paths(body.get("data")))
return sorted(p for p in named if p in nulls)
def absent(body):
"""Paths that are null with no errors entry: genuinely empty, not hidden. Pure."""
if not isinstance(body, dict):
return []
named = set(error_paths(body))
return sorted(p for p in null_paths(body.get("data")) if p not in named)
def orphan_error_paths(body):
"""Error paths that do not resolve to a null in data. Pure.
Rare, and reported rather than swallowed: it usually means the path points
into a list element that was dropped entirely, and a script that silently
ignored it would be under-reporting the very thing it exists to count.
"""
if not isinstance(body, dict):
return []
nulls = set(null_paths(body.get("data")))
return sorted(p for p in error_paths(body) if p not in nulls)
def permission_hint(dotted):
"""The permission a withheld field would want. Pure."""
leaf = str(dotted).split(".")[-1]
return PERMISSION_HINT.get(leaf, "the permission that covers this field")
def tally(body):
"""Counts for one response. Pure."""
return {
"withheld": len(withheld(body)),
"absent": len(absent(body)),
"orphaned": len(orphan_error_paths(body)),
"unpathed_errors": unpathed_errors(body),
}
def is_partial_success(body):
"""Data survived and errors arrived beside it. Pure."""
if not isinstance(body, dict):
return False
return bool(body.get("errors")) and has_usable_data(body)
def safe_to_aggregate(body, root):
"""Whether a sum under this root is honest. Pure. Returns (bool, sentence)."""
under = [p for p in withheld(body)
if not root or p == root or p.startswith(str(root) + ".")]
if not under:
return True, ("no withheld fields under %r, so a total over it is a "
"total." % root)
return False, ("%d withheld field(s) under %r, so a total over it is a lower "
"bound and has to be labelled as one." % (len(under), root))
def classify(body):
"""Classify one response. Pure. Returns (state, detail).
Total failure is named and handed on rather than absorbed, because the
repair for a query that failed outright is not the repair for one that
mostly worked.
"""
if not isinstance(body, dict):
return ("unreadable",
"the response was not a JSON object, so nothing can be counted "
"in it.")
errs = body.get("errors") or []
hidden = withheld(body)
empty = absent(body)
if errs and not has_usable_data(body):
return ("total-failure",
"%d error(s) arrived and no field resolved, so this is a failed "
"query wearing a 200 rather than a partial one." % len(errs))
if errs and not hidden and unpathed_errors(body):
return ("errors-without-path",
"%d error(s) arrived beside usable data but none of them names a "
"field, so nothing can be attributed to a column."
% unpathed_errors(body))
if hidden:
return ("partial-withheld",
"%d field(s) resolved to null and errors[].path explains %s."
% (len(hidden), "both" if len(hidden) == 2 else "each of them"))
if empty:
return ("nulls-unexplained",
"%d null(s) in the data and no errors entry for any of them, so "
"they are genuinely empty rather than withheld." % len(empty))
return ("complete",
"every requested field resolved and the errors array is empty.")
def repair(state):
"""The sentence a reader has to act on. Pure."""
if state == "partial-withheld":
return ("record the withheld paths as unknown, not zero, and label the "
"total a lower bound. Do not retry: this token returns the same "
"nulls every time.")
if state == "nulls-unexplained":
return ("nothing on the nulls: with no errors entry beside them they are "
"real answers. Keep reading errors[].path anyway, because that "
"is what will tell you when one of them stops being real.")
if state == "total-failure":
return ("see /github/graphql-200-with-errors/ -- nothing resolved here, "
"so this is the total-failure case and partial-response handling "
"does not apply.")
if state == "errors-without-path":
return ("log these errors verbatim and treat the whole response as "
"suspect. An error with no path cannot be attributed to a "
"column, so no per-field repair is available.")
if state == "complete":
return "nothing."
return "point the check at a document this endpoint can answer."
def point_cost(queries):
"""Points this run will spend against the GraphQL budget. Pure."""
try:
return max(0, int(queries)) * POINTS_PER_QUERY
except (TypeError, ValueError):
return 0
def run_query(session, document, variables):
"""Send one query. Returns (status, body-or-None).
A GraphQL query is a read; POST is only how the document reaches the
endpoint, which is why the verb is written here beside the URL rather than
hidden in a constant where it could be mistaken for a write path.
"""
r = session.post(API + "/graphql",
json={"query": document, "variables": variables or {}},
timeout=30)
if r.status_code == 401:
raise SystemExit("401 from GitHub: GITHUB_TOKEN is missing, malformed "
"or revoked")
try:
return r.status_code, r.json()
except ValueError:
return r.status_code, None
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--repo", required=True, help="owner/name")
ap.add_argument("--query",
help="send your own query document instead of the default. "
"Use the one your integration actually sends: the nulls "
"follow the fields. Mutations are refused.")
ap.add_argument("--root", default="repository",
help="the path you aggregate over, checked for withheld "
"fields underneath it")
args = ap.parse_args()
token = os.environ.get("GITHUB_TOKEN")
if not token:
log.error("set GITHUB_TOKEN (a read-only token is enough)")
return 2
try:
owner, name = args.repo.split("/", 1)
except ValueError:
log.error("--repo takes owner/name")
return 2
document = args.query or DEFAULT_QUERY
why_not = refusal(document)
if why_not:
log.error("refusing to send: %s", why_not)
return 2
log.info("point cost: %d point(s) against the 5,000/hour GraphQL budget",
point_cost(1))
session = requests.Session()
session.headers.update({
"Authorization": "Bearer " + token,
"Accept": "application/vnd.github+json",
"User-Agent": UA,
})
status, body = run_query(session, document, {"owner": owner, "name": name})
state, detail = classify(body)
counts = tally(body)
named = error_paths(body)
log.info("HTTP %s, errors=%d, data usable=%s", status,
len(((body or {}).get("errors")) or []),
"yes" if has_usable_data(body) else "no")
log.info("%s: %s", state, detail)
for p in withheld(body):
log.info(" %-34s withheld %-11s wants: %s",
p, named.get(p, "UNTYPED"), permission_hint(p))
for p in absent(body):
log.info(" %-34s absent %-11s genuinely empty, safe to read as none",
p, "-")
for p in orphan_error_paths(body):
log.info(" %-34s orphaned %-11s named by errors but not null in data",
p, named.get(p, "UNTYPED"))
ok, sentence = safe_to_aggregate(body, args.root)
log.info("aggregation over %r is %s: %s", args.root,
"safe" if ok else "NOT safe", sentence)
log.info("repair: %s", repair(state))
print(json.dumps({
"points_spent": point_cost(1),
"status": status,
"state": state,
"detail": detail,
"partial_success": is_partial_success(body),
"withheld": withheld(body),
"absent": absent(body),
"orphan_error_paths": orphan_error_paths(body),
"tally": counts,
"aggregation_root": args.root,
"aggregation_safe": ok,
}, indent=2, default=str))
return 1 if state in ("partial-withheld", "errors-without-path") else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Separate the fields a GraphQL response withheld from the ones that are empty.
*
* Read only, and queries only. GitHub's GraphQL endpoint takes a document in
* the request body, so a read is carried by POST there just as a write would
* be; that is transport, not intent. Any document containing a mutation or a
* subscription is refused before a socket opens.
*
* Environment:
* GITHUB_TOKEN a token with read access to the GraphQL API
* GITHUB_REPO owner/name
* GITHUB_QUERY send your own query document instead of the default
* GITHUB_ROOT the path you aggregate over, default 'repository'
*/
const API = 'https://api.github.com';
const UA = 'github-graphql-partial/1.0';
export const POINTS_PER_QUERY = 1;
const DEFAULT_QUERY = 'query($owner: String!, $name: String!) {'
+ ' repository(owner: $owner, name: $name) {'
+ ' name isPrivate diskUsage'
+ ' licenseInfo { key }'
+ ' collaborators(first: 1) { totalCount }'
+ ' } }';
/** What a withheld field would need to be readable. */
export const PERMISSION_HINT = {
diskUsage: 'metadata read plus admin on the repository',
collaborators: 'read access to repository members',
vulnerabilityAlerts: 'Dependabot alerts read',
projectsV2: 'organization projects read',
members: 'read:org on the organization',
email: 'user email read, and the user must have a public email',
};
const MISSING = Symbol('missing');
/** Remove GraphQL comments and string literals from a document. Pure. */
export function stripNoise(document) {
const src = String(document ?? '');
const out = [];
let i = 0;
while (i < src.length) {
const ch = src[i];
if (ch === '#') {
while (i < src.length && src[i] !== '\n') i += 1;
continue;
}
if (src.startsWith('"""', i)) {
const j = src.indexOf('"""', i + 3);
i = j < 0 ? src.length : j + 3;
out.push(' ');
continue;
}
if (ch === '"') {
i += 1;
while (i < src.length && src[i] !== '"') i += src[i] === '\\' ? 2 : 1;
i += 1;
out.push(' ');
continue;
}
out.push(ch);
i += 1;
}
return out.join('');
}
/** The top-level operations in a document, in order. Pure. */
export function operations(document) {
const src = `${stripNoise(document)} `;
const ops = [];
let depth = 0;
let word = '';
let declared = null;
for (const ch of src) {
if (/[A-Za-z0-9_]/.test(ch)) { word += ch; continue; }
if (word) {
if (depth === 0 && ['query', 'mutation', 'subscription', 'fragment'].includes(word)) {
declared = word;
}
word = '';
}
if (ch === '{') {
if (depth === 0) { ops.push(declared || 'query'); declared = null; }
depth += 1;
} else if (ch === '}') {
depth = Math.max(0, depth - 1);
}
}
return ops;
}
/** Why this document will not be sent, or null if it is a read. Pure. */
export function refusal(document) {
const ops = operations(document);
if (ops.length === 0) return 'the document contains no operation to send.';
for (const kind of ['mutation', 'subscription']) {
if (ops.includes(kind)) {
return `the document contains a ${kind}. This script sends queries only: `
+ 'a query is a read, and the section it belongs to promises its '
+ 'scripts never write.';
}
}
return null;
}
/** A GraphQL error path rendered as a dotted string. Pure. */
export function pathKey(path) {
if (typeof path === 'string') return path;
return (path || []).map(String).join('.');
}
/** Resolve a dotted path in a data tree. Pure. MISSING if there is no such path. */
export function valueAt(data, dotted) {
let cur = data;
if (!dotted) return cur;
for (const seg of String(dotted).split('.')) {
if (Array.isArray(cur)) {
const i = Number(seg);
if (!Number.isInteger(i) || i < 0 || i >= cur.length) return MISSING;
cur = cur[i];
} else if (cur && typeof cur === 'object') {
if (!Object.prototype.hasOwnProperty.call(cur, seg)) return MISSING;
cur = cur[seg];
} else {
return MISSING;
}
}
return cur;
}
/** Every path in a data tree whose value is null. Pure. */
export function nullPaths(data, prefix = '') {
const out = [];
let entries;
if (Array.isArray(data)) entries = data.map((v, i) => [String(i), v]);
else if (data && typeof data === 'object') entries = Object.entries(data);
else return out;
for (const [key, value] of entries) {
const here = prefix ? `${prefix}.${key}` : key;
if (value === null || value === undefined) out.push(here);
else out.push(...nullPaths(value, here));
}
return out.sort();
}
/** Dotted paths named by the errors array, mapped to their type. Pure. */
export function errorPaths(body) {
const out = {};
if (!body || typeof body !== 'object' || !Array.isArray(body.errors)) return out;
for (const err of body.errors) {
if (!err || typeof err !== 'object' || !err.path || err.path.length === 0) continue;
out[pathKey(err.path)] = err.type || 'UNTYPED';
}
return out;
}
/** Errors that name no field, so nothing can be attributed to them. Pure. */
export function unpathedErrors(body) {
if (!body || typeof body !== 'object' || !Array.isArray(body.errors)) return 0;
return body.errors.filter(
(e) => !e || typeof e !== 'object' || !e.path || e.path.length === 0,
).length;
}
/** Whether any top-level field resolved to something other than null. Pure. */
export function hasUsableData(body) {
if (!body || typeof body !== 'object') return false;
const data = body.data;
if (!data || typeof data !== 'object' || Array.isArray(data)) return false;
return Object.values(data).some((v) => v !== null && v !== undefined);
}
/** Paths that are null in data and explained by an errors entry. Pure. */
export function withheld(body) {
if (!body || typeof body !== 'object') return [];
const named = errorPaths(body);
const nulls = new Set(nullPaths(body.data));
return Object.keys(named).filter((p) => nulls.has(p)).sort();
}
/** Paths that are null with no errors entry: genuinely empty, not hidden. Pure. */
export function absent(body) {
if (!body || typeof body !== 'object') return [];
const named = new Set(Object.keys(errorPaths(body)));
return nullPaths(body.data).filter((p) => !named.has(p)).sort();
}
/** Error paths that do not resolve to a null in data. Pure. */
export function orphanErrorPaths(body) {
if (!body || typeof body !== 'object') return [];
const nulls = new Set(nullPaths(body.data));
return Object.keys(errorPaths(body)).filter((p) => !nulls.has(p)).sort();
}
/** The permission a withheld field would want. Pure. */
export function permissionHint(dotted) {
const leaf = String(dotted).split('.').pop();
return Object.prototype.hasOwnProperty.call(PERMISSION_HINT, leaf)
? PERMISSION_HINT[leaf]
: 'the permission that covers this field';
}
/** Counts for one response. Pure. */
export function tally(body) {
return {
withheld: withheld(body).length,
absent: absent(body).length,
orphaned: orphanErrorPaths(body).length,
unpathed_errors: unpathedErrors(body),
};
}
/** Data survived and errors arrived beside it. Pure. */
export function isPartialSuccess(body) {
if (!body || typeof body !== 'object') return false;
return Array.isArray(body.errors) && body.errors.length > 0 && hasUsableData(body);
}
/** Whether a sum under this root is honest. Pure. Returns [bool, sentence]. */
export function safeToAggregate(body, root) {
const under = withheld(body).filter(
(p) => !root || p === root || p.startsWith(`${root}.`),
);
if (under.length === 0) {
return [true, `no withheld fields under '${root}', so a total over it is a total.`];
}
return [false, `${under.length} withheld field(s) under '${root}', so a total `
+ 'over it is a lower bound and has to be labelled as one.'];
}
/** Classify one response. Pure. Returns [state, detail]. */
export function classify(body) {
if (!body || typeof body !== 'object' || Array.isArray(body)) {
return ['unreadable', 'the response was not a JSON object, so nothing can '
+ 'be counted in it.'];
}
const errs = Array.isArray(body.errors) ? body.errors : [];
const hidden = withheld(body);
const empty = absent(body);
if (errs.length > 0 && !hasUsableData(body)) {
return ['total-failure', `${errs.length} error(s) arrived and no field `
+ 'resolved, so this is a failed query wearing a 200 rather than a '
+ 'partial one.'];
}
if (errs.length > 0 && hidden.length === 0 && unpathedErrors(body) > 0) {
return ['errors-without-path', `${unpathedErrors(body)} error(s) arrived `
+ 'beside usable data but none of them names a field, so nothing can be '
+ 'attributed to a column.'];
}
if (hidden.length > 0) {
return ['partial-withheld', `${hidden.length} field(s) resolved to null and `
+ `errors[].path explains ${hidden.length === 2 ? 'both' : 'each of them'}.`];
}
if (empty.length > 0) {
return ['nulls-unexplained', `${empty.length} null(s) in the data and no `
+ 'errors entry for any of them, so they are genuinely empty rather than '
+ 'withheld.'];
}
return ['complete', 'every requested field resolved and the errors array is empty.'];
}
/** The sentence a reader has to act on. Pure. */
export function repair(state) {
if (state === 'partial-withheld') {
return 'record the withheld paths as unknown, not zero, and label the total '
+ 'a lower bound. Do not retry: this token returns the same nulls every time.';
}
if (state === 'nulls-unexplained') {
return 'nothing on the nulls: with no errors entry beside them they are real '
+ 'answers. Keep reading errors[].path anyway, because that is what will '
+ 'tell you when one of them stops being real.';
}
if (state === 'total-failure') {
return 'see /github/graphql-200-with-errors/ -- nothing resolved here, so '
+ 'this is the total-failure case and partial-response handling does not apply.';
}
if (state === 'errors-without-path') {
return 'log these errors verbatim and treat the whole response as suspect. '
+ 'An error with no path cannot be attributed to a column, so no per-field '
+ 'repair is available.';
}
if (state === 'complete') return 'nothing.';
return 'point the check at a document this endpoint can answer.';
}
/** Points this run will spend against the GraphQL budget. Pure. */
export function pointCost(queries) {
const n = Number(queries);
if (!Number.isFinite(n) || n < 0) return 0;
return Math.trunc(n) * POINTS_PER_QUERY;
}
function headers(token) {
return {
Authorization: `Bearer ${token}`,
Accept: 'application/vnd.github+json',
'Content-Type': 'application/json',
'User-Agent': UA,
};
}
async function runQuery(token, document, variables) {
const res = await fetch(`${API}/graphql`, {
// A GraphQL query is a read. POST is only how the document reaches the
// endpoint, and refusal() has already rejected anything that is not a read.
method: 'POST',
headers: headers(token),
body: JSON.stringify({ query: document, variables: variables || {} }),
});
let body = null;
try { body = await res.json(); } catch { body = null; }
return { status: res.status, body };
}
async function main() {
const token = process.env.GITHUB_TOKEN;
const repo = process.env.GITHUB_REPO;
if (!token || !repo) {
console.error('set GITHUB_TOKEN (read-only is enough) and GITHUB_REPO=owner/name');
process.exitCode = 2;
return;
}
const [owner, name] = repo.split('/');
if (!owner || !name) {
console.error('GITHUB_REPO takes owner/name');
process.exitCode = 2;
return;
}
const document = process.env.GITHUB_QUERY || DEFAULT_QUERY;
const root = process.env.GITHUB_ROOT || 'repository';
const whyNot = refusal(document);
if (whyNot) {
console.error(`refusing to send: ${whyNot}`);
process.exitCode = 2;
return;
}
console.log(`point cost: ${pointCost(1)} point(s) against the 5,000/hour GraphQL budget`);
const { status, body } = await runQuery(token, document, { owner, name });
const [state, detail] = classify(body);
const named = errorPaths(body);
const errs = (body && Array.isArray(body.errors)) ? body.errors.length : 0;
console.log(`HTTP ${status}, errors=${errs}, data usable=${hasUsableData(body) ? 'yes' : 'no'}`);
console.log(`${state}: ${detail}`);
for (const p of withheld(body)) {
console.log(` ${p.padEnd(34)} withheld ${(named[p] || 'UNTYPED').padEnd(11)} wants: ${permissionHint(p)}`);
}
for (const p of absent(body)) {
console.log(` ${p.padEnd(34)} absent ${'-'.padEnd(11)} genuinely empty, safe to read as none`);
}
for (const p of orphanErrorPaths(body)) {
console.log(` ${p.padEnd(34)} orphaned ${(named[p] || 'UNTYPED').padEnd(11)} named by errors but not null in data`);
}
const [ok, sentence] = safeToAggregate(body, root);
console.log(`aggregation over '${root}' is ${ok ? 'safe' : 'NOT safe'}: ${sentence}`);
console.log(`repair: ${repair(state)}`);
console.log(JSON.stringify({
points_spent: pointCost(1),
status,
state,
detail,
partial_success: isPartialSuccess(body),
withheld: withheld(body),
absent: absent(body),
orphan_error_paths: orphanErrorPaths(body),
tally: tally(body),
aggregation_root: root,
aggregation_safe: ok,
}, null, 2));
process.exitCode = ['partial-withheld', 'errors-without-path'].includes(state) ? 1 : 0;
}
// Guarded so importing this file from the test runner does not start a run and
// fail the suite even as every test passes.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The suite is built around one envelope that is deliberately awkward: a repository object with three nulls in it, two of which are named in errors[].path and one of which is not. Everything important falls out of that. The path resolver is tested against list indices, because real error paths contain them and a resolver that only walks objects would silently report the interesting cases as unexplained. Then the set difference in both directions, an error path that resolves to nothing at all, the aggregation verdict under a named root, the boundary against the total-failure case that belongs to the other note, and the refusal that keeps a mutation off the wire.
from github_graphql_partial import (
MISSING, absent, classify, error_paths, is_partial_success, null_paths, operations,
orphan_error_paths, path_key, permission_hint, point_cost, refusal, repair,
safe_to_aggregate, tally, unpathed_errors, value_at, withheld,
)
PARTIAL = {
"data": {"repository": {
"name": "monorepo",
"isPrivate": True,
"diskUsage": None,
"licenseInfo": None,
"collaborators": None,
}},
"errors": [
{"type": "FORBIDDEN", "path": ["repository", "diskUsage"],
"message": "Resource not accessible by personal access token"},
{"type": "FORBIDDEN", "path": ["repository", "collaborators"],
"message": "Must have push access to view repository collaborators."},
],
}
IN_A_LIST = {
"data": {"repository": {"pullRequests": {"nodes": [
{"number": 1, "author": {"login": "ada"}},
{"number": 2, "author": None},
]}}},
"errors": [{"type": "FORBIDDEN",
"path": ["repository", "pullRequests", "nodes", 1, "author"]}],
}
TOTAL_FAILURE = {"data": {"repository": None},
"errors": [{"type": "NOT_FOUND", "path": ["repository"]}]}
CLEAN = {"data": {"repository": {"name": "monorepo", "isPrivate": False}}}
def test_a_partial_response_is_a_third_outcome_not_a_failure():
assert is_partial_success(PARTIAL)
assert not is_partial_success(TOTAL_FAILURE)
assert not is_partial_success(CLEAN)
def test_withheld_and_absent_are_the_two_kinds_of_null():
assert withheld(PARTIAL) == ["repository.collaborators", "repository.diskUsage"]
assert absent(PARTIAL) == ["repository.licenseInfo"]
def test_a_null_with_no_errors_entry_is_a_real_answer():
body = {"data": {"repository": {"name": "x", "licenseInfo": None}}}
assert withheld(body) == []
assert absent(body) == ["repository.licenseInfo"]
state, detail = classify(body)
assert state == "nulls-unexplained"
assert "genuinely empty" in detail
def test_error_paths_survive_a_list_index():
assert path_key(["repository", "pullRequests", "nodes", 1, "author"]) == \
"repository.pullRequests.nodes.1.author"
assert withheld(IN_A_LIST) == ["repository.pullRequests.nodes.1.author"]
assert absent(IN_A_LIST) == []
def test_the_path_resolver_walks_lists_as_well_as_objects():
data = IN_A_LIST["data"]
assert value_at(data, "repository.pullRequests.nodes.0.number") == 1
assert value_at(data, "repository.pullRequests.nodes.1.author") is None
assert value_at(data, "repository.pullRequests.nodes.9") is MISSING
assert value_at(data, "repository.nothingLikeThis") is MISSING
assert null_paths({"a": None, "b": {"c": None, "d": 1}}) == ["a", "b.c"]
def test_an_error_path_that_matches_no_null_is_reported_not_swallowed():
body = {"data": {"repository": {"name": "x"}},
"errors": [{"type": "FORBIDDEN", "path": ["repository", "gone"]}]}
assert orphan_error_paths(body) == ["repository.gone"]
assert withheld(body) == []
def test_an_error_with_no_path_cannot_be_attributed():
body = {"data": {"repository": {"name": "x"}},
"errors": [{"type": "INTERNAL", "message": "something broke"}]}
assert unpathed_errors(body) == 1
assert error_paths(body) == {}
state, _ = classify(body)
assert state == "errors-without-path"
assert "verbatim" in repair(state)
def test_a_query_where_nothing_resolved_belongs_to_the_other_note():
state, detail = classify(TOTAL_FAILURE)
assert state == "total-failure"
assert "failed query wearing a 200" in detail
assert "graphql-200-with-errors" in repair(state)
def test_the_finding_names_the_paths_rather_than_counting_errors():
state, detail = classify(PARTIAL)
assert state == "partial-withheld"
assert "errors[].path" in detail
assert "unknown, not zero" in repair(state)
assert "Do not retry" in repair(state)
def test_an_aggregate_over_a_root_with_withheld_fields_is_a_lower_bound():
ok, sentence = safe_to_aggregate(PARTIAL, "repository")
assert not ok
assert "lower bound" in sentence
ok2, sentence2 = safe_to_aggregate(PARTIAL, "viewer")
assert ok2
assert "is a total" in sentence2
def test_the_aggregation_root_is_matched_on_a_boundary_not_a_prefix():
body = {"data": {"repo": {"a": None}, "repository": {"b": 1}},
"errors": [{"type": "FORBIDDEN", "path": ["repo", "a"]}]}
ok, _ = safe_to_aggregate(body, "repository")
assert ok
def test_each_withheld_field_names_the_permission_it_would_want():
assert "admin" in permission_hint("repository.diskUsage")
assert "members" in permission_hint("repository.collaborators")
assert permission_hint("repository.somethingNew") == \
"the permission that covers this field"
def test_the_tally_counts_all_four_kinds_of_thing():
assert tally(PARTIAL) == {"withheld": 2, "absent": 1, "orphaned": 0,
"unpathed_errors": 0}
assert tally(CLEAN) == {"withheld": 0, "absent": 0, "orphaned": 0,
"unpathed_errors": 0}
def test_a_clean_response_says_so_plainly():
state, _ = classify(CLEAN)
assert state == "complete"
assert repair(state) == "nothing."
assert classify(None)[0] == "unreadable"
def test_the_script_refuses_to_send_a_mutation():
assert operations("query Q { viewer { login } }") == ["query"]
assert refusal("mutation M { addStar(input: {}) { clientMutationId } }")
assert refusal("subscription S { thing { id } }")
assert refusal("query Q { repository(owner: \"o\", name: \"n\") { name } }") is None
def test_the_run_says_what_it_will_spend():
assert point_cost(1) == 1
assert point_cost(0) == 0
assert point_cost(None) == 0
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
absent, classify, errorPaths, isPartialSuccess, nullPaths, operations,
orphanErrorPaths, pathKey, permissionHint, pointCost, refusal, repair,
safeToAggregate, tally, unpathedErrors, valueAt, withheld,
} from './github-graphql-partial.mjs';
const PARTIAL = {
data: {
repository: {
name: 'monorepo',
isPrivate: true,
diskUsage: null,
licenseInfo: null,
collaborators: null,
},
},
errors: [
{ type: 'FORBIDDEN', path: ['repository', 'diskUsage'] },
{ type: 'FORBIDDEN', path: ['repository', 'collaborators'] },
],
};
const IN_A_LIST = {
data: {
repository: {
pullRequests: {
nodes: [
{ number: 1, author: { login: 'ada' } },
{ number: 2, author: null },
],
},
},
},
errors: [{ type: 'FORBIDDEN', path: ['repository', 'pullRequests', 'nodes', 1, 'author'] }],
};
const TOTAL_FAILURE = {
data: { repository: null },
errors: [{ type: 'NOT_FOUND', path: ['repository'] }],
};
const CLEAN = { data: { repository: { name: 'monorepo', isPrivate: false } } };
test('a partial response is a third outcome, not a failure', () => {
assert.ok(isPartialSuccess(PARTIAL));
assert.ok(!isPartialSuccess(TOTAL_FAILURE));
assert.ok(!isPartialSuccess(CLEAN));
});
test('withheld and absent are the two kinds of null', () => {
assert.deepEqual(withheld(PARTIAL), ['repository.collaborators', 'repository.diskUsage']);
assert.deepEqual(absent(PARTIAL), ['repository.licenseInfo']);
});
test('a null with no errors entry is a real answer', () => {
const body = { data: { repository: { name: 'x', licenseInfo: null } } };
assert.deepEqual(withheld(body), []);
assert.deepEqual(absent(body), ['repository.licenseInfo']);
const [state, detail] = classify(body);
assert.equal(state, 'nulls-unexplained');
assert.match(detail, /genuinely empty/);
});
test('error paths survive a list index', () => {
assert.equal(
pathKey(['repository', 'pullRequests', 'nodes', 1, 'author']),
'repository.pullRequests.nodes.1.author',
);
assert.deepEqual(withheld(IN_A_LIST), ['repository.pullRequests.nodes.1.author']);
assert.deepEqual(absent(IN_A_LIST), []);
});
test('the path resolver walks lists as well as objects', () => {
const data = IN_A_LIST.data;
assert.equal(valueAt(data, 'repository.pullRequests.nodes.0.number'), 1);
assert.equal(valueAt(data, 'repository.pullRequests.nodes.1.author'), null);
assert.deepEqual(nullPaths({ a: null, b: { c: null, d: 1 } }), ['a', 'b.c']);
});
test('an error path that matches no null is reported, not swallowed', () => {
const body = {
data: { repository: { name: 'x' } },
errors: [{ type: 'FORBIDDEN', path: ['repository', 'gone'] }],
};
assert.deepEqual(orphanErrorPaths(body), ['repository.gone']);
assert.deepEqual(withheld(body), []);
});
test('an error with no path cannot be attributed', () => {
const body = {
data: { repository: { name: 'x' } },
errors: [{ type: 'INTERNAL', message: 'something broke' }],
};
assert.equal(unpathedErrors(body), 1);
assert.deepEqual(errorPaths(body), {});
const [state] = classify(body);
assert.equal(state, 'errors-without-path');
assert.match(repair(state), /verbatim/);
});
test('a query where nothing resolved belongs to the other note', () => {
const [state, detail] = classify(TOTAL_FAILURE);
assert.equal(state, 'total-failure');
assert.match(detail, /failed query wearing a 200/);
assert.match(repair(state), /graphql-200-with-errors/);
});
test('the finding names the paths rather than counting errors', () => {
const [state, detail] = classify(PARTIAL);
assert.equal(state, 'partial-withheld');
assert.match(detail, /errors\[\]\.path/);
assert.match(repair(state), /unknown, not zero/);
assert.match(repair(state), /Do not retry/);
});
test('an aggregate over a root with withheld fields is a lower bound', () => {
const [ok, sentence] = safeToAggregate(PARTIAL, 'repository');
assert.ok(!ok);
assert.match(sentence, /lower bound/);
const [ok2, sentence2] = safeToAggregate(PARTIAL, 'viewer');
assert.ok(ok2);
assert.match(sentence2, /is a total/);
});
test('the aggregation root is matched on a boundary, not a prefix', () => {
const body = {
data: { repo: { a: null }, repository: { b: 1 } },
errors: [{ type: 'FORBIDDEN', path: ['repo', 'a'] }],
};
const [ok] = safeToAggregate(body, 'repository');
assert.ok(ok);
});
test('each withheld field names the permission it would want', () => {
assert.match(permissionHint('repository.diskUsage'), /admin/);
assert.match(permissionHint('repository.collaborators'), /members/);
assert.equal(
permissionHint('repository.somethingNew'),
'the permission that covers this field',
);
});
test('the tally counts all four kinds of thing', () => {
assert.deepEqual(tally(PARTIAL),
{ withheld: 2, absent: 1, orphaned: 0, unpathed_errors: 0 });
assert.deepEqual(tally(CLEAN),
{ withheld: 0, absent: 0, orphaned: 0, unpathed_errors: 0 });
});
test('a clean response says so plainly', () => {
const [state] = classify(CLEAN);
assert.equal(state, 'complete');
assert.equal(repair(state), 'nothing.');
assert.equal(classify(null)[0], 'unreadable');
});
test('the script refuses to send a mutation', () => {
assert.deepEqual(operations('query Q { viewer { login } }'), ['query']);
assert.ok(refusal('mutation M { addStar(input: {}) { clientMutationId } }'));
assert.ok(refusal('subscription S { thing { id } }'));
assert.equal(refusal('query Q { repository(owner: "o", name: "n") { name } }'), null);
});
test('the run says what it will spend', () => {
assert.equal(pointCost(1), 1);
assert.equal(pointCost(0), 0);
assert.equal(pointCost(null), 0);
});
FAQ
Why does a forbidden field not fail the whole query?
Because GraphQL resolves fields independently, and failing the document because one field out of forty was refused would make the API almost unusable for anyone holding a narrow token. The design decision is deliberate: you get everything you were allowed to have, plus an exact statement of what you were not. It is a better contract than the REST equivalent, which is a 404 that will not say whether the thing is missing or hidden. It only helps if the errors array is read.
How do I tell a withheld null from a genuinely empty field?
By whether a path in errors[].path points at it. That is the only signal, and it is reliable: a field that resolved to null because there is nothing there produces no error entry at all, while a field the server refused produces one naming the exact route to it, list indices included. The script does this comparison as a set difference in both directions, and it also reports the leftovers, meaning error paths that do not correspond to any null, because those usually mean an element was dropped from a list rather than nulled in place.
Should I retry a partial response?
No. The nulls come from what this token is permitted to read, and that does not change between one request and the next, so a retry costs another point and returns the same shape. Retrying is the right reflex for RATE_LIMITED and sometimes for INTERNAL; it is exactly wrong here. The two available moves are to widen the token, which is a decision about access rather than a bug fix, and to record the withheld fields as unknown so nothing downstream sums across them.
Can I just filter out the objects that have nulls?
Only if you also report how many you dropped. Dropping the eight private repositories from a fifty-repository storage report and publishing the total for forty-two as though it were the total for fifty is the same error the nulls caused, arrived at more deliberately. The honest shape is a number plus a count of unknowns, and the script prints exactly that so it can be carried through to whatever consumes the result.
Does this need a token with more permissions to run?
It needs less, not more, and that is rather the point: run it with the same read-only token your integration uses and it will show you the gaps that token has. Running it as an administrator would report a clean response and tell you nothing, because the nulls are a property of the credential rather than of the repository. One query, one point out of 5,000 an hour, printed before it is spent.
Related field notes
- A GraphQL 200 that carries an errors array
- An App permission the integration never asked for
- SSO quietly removes rows from a list
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- GraphQL specification: Errors
- Using the GraphQL API — GitHub Docs
- Permissions required for fine-grained personal access tokens — GitHub Docs
- Introduction to GraphQL — GitHub Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.