Diagnostic GitHub API
The org's base permission dropped and repos vanished
The inventory job used to report on four hundred repositories. This morning it reported on nine, and reported them cheerfully: no errors, no refusals, no retries, a clean run in a tenth of the usual time. Nothing was revoked from the account — it is still a member of the organization, its token is unchanged, and the nine repositories it can see are ones somebody added it to explicitly, years ago. What changed is one field on the organization. Somebody moved the base permission from read to none, which is a good and normal thing to do, and every member's implicit access to every repository they were never explicitly added to ended in the same instant.
An organization's default_repository_permission is the role every member gets on repositories they have not been added to individually or through a team. Tightening it from read to none is a standard hardening step and it takes effect everywhere at once.
Read the field and then measure the damage. GET /orgs/{org} returns default_repository_permission alongside public_repos and total_private_repos. Then count what your account can actually see: request GET /user/repos?affiliation=organization_member&per_page=1 and read the page number out of the rel="last" link — with one item per page, the last page number is the count, and it costs one request rather than four hundred. The gap between the two numbers is the finding.
The problem in plain words
The expensive part is that nothing fails. A permission problem that produces a 403 gets fixed, because a 403 is loud, has a stack trace attached to it and lands in an alert. This produces shorter lists. The job succeeds, the report is well formed, the dashboard is green, and the number on it is wrong in a direction nobody looks at. Coverage loss is the failure mode that survives longest in production, because every part of the system agrees that it went fine.
When somebody does notice, the search starts in the wrong place, because the natural question is "which repositories are missing" and the natural next step is to check one of them. That check is about a repository: does it exist, is it private, was it archived, was it renamed. Every one of those answers comes back normal, because nothing is wrong with the repositories. The subject of this failure is the organization, and it is one field wide.
The other trap is that it does not look org-wide from the inside, because it is not uniform. Repositories the account was explicitly added to keep working perfectly. So the surviving nine are a distraction: they look like proof that access is basically fine and the problem must be with the other four hundred, when they are actually the only ones whose access never depended on the default in the first place.
Why it happens
Base permission is a default, and defaults are invisible until they move. Every member of an organization has a role on every repository, and for most people on most repositories that role was never granted to them by name — it came from default_repository_permission. That is what makes a change to it feel like something else entirely: there is no grant to look for and no revocation event, because nothing was ever granted or revoked at the repository level. The floor moved.
This is an organization-wide object, which is what separates it from a role on one repository. The section already publishes the case where an account's role on a single repository is too low for the write it is attempting. That note reads a permissions object per repository and answers about one of them. This one reads a single field that re-grades every repository in the organization simultaneously, and its symptom is not a refused write at all — it is a list that got shorter while every call in it succeeded.
Counting with rel="last" makes the measurement affordable. Asking for one item per page and reading the last page number out of the Link header gives you the size of a collection in a single request. That is the difference between a check you run on a schedule and one you never run at all. It has one failure mode worth knowing: when the result fits on a single page there is no rel="last", which is its own note, and the script reports the count as coming from a single page rather than pretending the collection is empty.
The two numbers you are comparing are not the same kind of number, and the script says so. public_repos plus total_private_repos on the organization is what the organization contains. The affiliation count is what your account can reach through membership. A gap is expected the moment base permission is none; the question is whether the gap is the one you intended. A drop from four hundred to nine with base permission at none and nine explicit grants is a correctly-configured organization and a badly-configured integration.
The repair is not to put the default back. Base permissions are a security control, and a member of an organization that sets it to none should be given access to what it needs by name. Reversing the hardening to fix one integration re-grants implicit access to everybody, which is a much larger change than the one that broke you. The script prints the narrow repair: add the account, or the team it belongs to, to the repositories the job is supposed to cover.
The fix, as a flow
This one is measured rather than caught, because nothing fails. The organization read gives the base permission and the size of the organization; one more request, asking for a single item per page, gives the number of repositories the account can actually reach, because at one per page the last page number is the count. Two numbers, one comparison, and the grade is a word rather than a ratio so it can be alerted on. The script will not blame the field it just read: a coverage that collapsed while the base permission still grants implicit access is somebody else's note, and it says so.
How to fix it
Read the field that moved
One call to GET /orgs/{org} gives default_repository_permission. The script prints what that value implies for a member with no explicit grants: at none, private repositories are invisible; at read, all of them are readable. Reading the field needs organization access, so an unreadable value is reported as unreadable rather than assumed.
Count what the account can actually see, in one request
GET /user/repos?affiliation=organization_member&per_page=1 with the rel="last" page number gives the count without paging through it. The script reports where the number came from — the Link header, or a single page with no rel="last" — because those two are the same number for different reasons and only one of them is a count.
Count what the organization holds
public_repos plus total_private_repos from the same organization read is the size of the organization. The script compares the two numbers and grades the coverage: full, partial, shrunken or collapsed. A collapsed coverage beside a base permission of none is this note; a collapsed coverage beside a base permission of read is something else and the script says so rather than blaming the field it just read.
Alert on drift rather than rediscovering it
Pass --expect read with the base permission your integration was designed against. The script compares and reports tightened, loosened or unchanged. That turns this from a thing you work out during an incident into a line in a scheduled check, which is the only form in which a coverage problem gets caught early.
Take the repair to the account, not to the default
The printed repair adds the account or its team to the repositories it is supposed to cover. It explicitly does not suggest raising the base permission back, because that re-grants implicit access to every member of the organization to fix one integration. The script changes nothing: it reads three things, prints a comparison and exits.
How to check it worked
Once the account or its team is added to the repositories it should cover, the affiliation count rises to meet them and the coverage stops being collapsed.
GITHUB_TOKEN=$READ_ONLY_TOKEN python3 github_org_base_permission.py acme --expect read
# read cost: 3 request(s) against the core hourly quota
# base permission: none — members get no role on repositories they were not
# added to individually or through a team. Every private repository in the
# organization is invisible to a member with no explicit grants.
# drift: base-tightened — configured for 'read', the organization now says
# 'none'. That is one field and it re-graded every repository at once.
# visible through membership: 9 (from rel="last" with per_page=1)
# organization holds: 412 repositories (public 12 + private 400)
# coverage: collapsed — 9 of 412
# state: base-none-implicit-access-gone
# repair: add this account, or a team it belongs to, to the repositories the
# job is meant to cover. Do not raise the base permission back: that
# re-grants implicit access to every member to fix one integration.The full code
Three reads, and the only clever one is a counting trick: ask for a single item per page and the last page number is the size of the collection, so a four-hundred-repository organization is measured in one request rather than four hundred. Everything else is pure — the Link header parse, which is written without a regular expression because a header with a comma inside a URL is exactly the input a hasty pattern gets wrong; the grading of coverage; the drift comparison against the base permission you configured for; and a repair that deliberately refuses to recommend the easy fix.
"""Say whether an organization's base permission is why the repository list shrank.
Read only. Three GETs. Granting an account access to a repository is a write
and a decision somebody with admin has to make, so this script measures the
loss and prints the narrow repair.
The point of the note: default_repository_permission is the role every member
holds on repositories they were never explicitly added to. Moving it from read
to none is ordinary hardening and it removes implicit access everywhere at
once, so a read-only integration keeps succeeding and covers a tenth of what
it did yesterday.
What this can and cannot see: the field is readable with organization access,
and the account's reachable repositories are countable in one request. What is
not visible is which repositories were reachable yesterday. This measures the
gap now and compares it against the base permission you say you configured
for; it cannot replay history.
Environment:
GITHUB_TOKEN a read-only token for the account whose coverage shrank
"""
import argparse
import json
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("github_org_base_permission")
API = "https://api.github.com"
UA = "github-org-base-permission/1.0"
# Weakest first. The documented values of default_repository_permission.
BASE_PERMISSIONS = ("none", "read", "write", "admin")
# What a member with no explicit grants gets, per base permission.
IMPLIES = {
"none": "members get no role on repositories they were not added to "
"individually or through a team. Every private repository in the "
"organization is invisible to a member with no explicit grants.",
"read": "every member can read every repository in the organization "
"without being added to it.",
"write": "every member can push to every repository in the organization "
"without being added to it.",
"admin": "every member administers every repository in the organization. "
"This is rare and worth questioning on its own.",
}
def read_cost():
"""Requests this run will spend against the core quota. Pure."""
return 3
def base_rank(value):
"""Position in the hierarchy, or -1 for something unrecognised. Pure."""
try:
return BASE_PERMISSIONS.index(str(value or "").strip().lower())
except ValueError:
return -1
def base_state(org_payload):
"""The organization's base permission. Pure. (value, detail).
An absent field is "unreadable" rather than a default. The field is
returned to callers with organization access, and a token without it would
otherwise be reported as an organization that grants nothing.
"""
if not isinstance(org_payload, dict):
return (None, "no organization payload was read.")
if "default_repository_permission" not in org_payload:
return (None, "default_repository_permission was not returned. Reading "
"it needs organization access, so this is unreadable "
"rather than absent.")
value = str(org_payload.get("default_repository_permission") or "").strip().lower()
if base_rank(value) < 0:
return (value or None, "the value %r is not one of the four documented "
"base permissions." % value)
return (value, IMPLIES[value])
def link_parts(link_header):
"""Split a Link header into its entries. Pure. No regular expression.
Split on the commas that separate entries and not on the ones inside a
URL, because a URL in a Link header can carry commas of its own -- a
search query, a list of fields -- and splitting the whole header throws
away the half that holds the page number.
"""
parts, current, depth = [], "", 0
for ch in str(link_header or ""):
if ch == "<":
depth += 1
elif ch == ">":
depth = max(0, depth - 1)
if ch == "," and depth == 0:
parts.append(current)
current = ""
else:
current += ch
if current.strip():
parts.append(current)
return parts
def last_page_from_link(link_header):
"""The page number of rel="last", or None. Pure. No regular expression."""
if not link_header:
return None
for part in link_parts(link_header):
if 'rel="last"' not in part and "rel=last" not in part:
continue
start = part.find("<")
end = part.find(">", start + 1)
if start < 0 or end < 0:
continue
url = part[start + 1:end]
query = url.partition("?")[2]
for field in query.split("&"):
name, _, value = field.partition("=")
if name == "page" and value.isdigit():
return int(value)
return None
def count_from_link(link_header, returned):
"""How many items the collection holds, at per_page=1. Pure. (count, how).
With one item per page the last page number is the count. When everything
fits on one page there is no rel="last" at all, and that case is reported
as what it is rather than as a count of zero.
"""
last = last_page_from_link(link_header)
if last is not None:
return (last, 'from rel="last" with per_page=1')
if not returned:
return (0, "the first page came back empty and carried no rel=\"last\"")
return (int(returned), 'a single page with no rel="last", so this is what '
'came back rather than a measured count')
def org_total(org_payload):
"""How many repositories the organization holds. Pure. (count, detail)."""
if not isinstance(org_payload, dict):
return (None, "no organization payload was read.")
public = org_payload.get("public_repos")
private = org_payload.get("total_private_repos")
if public is None and private is None:
return (None, "neither repository count was returned, which needs "
"organization access.")
total = int(public or 0) + int(private or 0)
return (total, "public %s + private %s"
% ("unreadable" if public is None else public,
"unreadable" if private is None else private))
def coverage_state(visible, total):
"""Grade what the account can see against what the org holds. Pure."""
if total is None or visible is None:
return "unknown"
if total <= 0:
return "nothing-to-cover"
if visible >= total:
return "full"
if visible == 0 or visible * 20 < total:
return "collapsed"
if visible * 2 < total:
return "shrunken"
return "partial"
def drift(expected, actual):
"""Compare the configured base permission against the live one. Pure."""
if not expected or actual is None:
return ("drift-unknown",
"no expected base permission was supplied, or the live one "
"could not be read, so there is nothing to compare.")
want, have = base_rank(expected), base_rank(actual)
if want < 0 or have < 0:
return ("drift-unknown",
"one of the two values is not a documented base permission.")
if want == have:
return ("base-unchanged",
"the organization still reports the base permission this "
"integration was configured against.")
if have < want:
return ("base-tightened",
"configured for %r, the organization now says %r. That is one "
"field and it re-graded every repository at once."
% (expected, actual))
return ("base-loosened",
"configured for %r, the organization now says %r, which grants "
"more implicit access than you expected rather than less."
% (expected, actual))
def verdict(base, coverage):
"""The finding, in one state. Pure. (state, detail)."""
if base is None:
return ("base-unreadable",
"the base permission could not be read, so the coverage number "
"stands on its own. Read it with a token that has organization "
"access before concluding anything about the default.")
if base == "none" and coverage in ("collapsed", "shrunken"):
return ("base-none-implicit-access-gone",
"base permission is none and this account reaches a fraction "
"of the organization. The repositories it still reaches are "
"the ones it was added to explicitly; the rest were never "
"granted, only defaulted.")
if base == "none" and coverage in ("full", "partial"):
return ("base-none-explicit-grants-hold",
"base permission is none and coverage is largely intact, which "
"means this account's access is explicit. It is not exposed to "
"this change.")
if base != "none" and coverage in ("collapsed", "shrunken"):
return ("coverage-lost-elsewhere",
"the base permission still grants implicit access and the "
"coverage is short anyway, so the loss is not this field. "
"Membership, SSO authorization and an App's repository "
"selection are the other ways a list gets shorter.")
if coverage == "nothing-to-cover":
return ("nothing-to-cover",
"the organization reports no repositories, so there is no "
"coverage question to answer.")
return ("coverage-as-expected",
"the account reaches what the base permission implies it should. "
"Nothing here explains a shorter list.")
def repair(state, org):
"""The narrow repair. Pure. Nothing here is executed."""
if state == "base-none-implicit-access-gone":
return ("add this account, or a team it belongs to, to the "
"repositories the job is meant to cover in %s. Do not raise "
"the base permission back: that re-grants implicit access to "
"every member of the organization to fix one integration."
% org)
if state == "coverage-lost-elsewhere":
return ("look past the base permission. Check that the account is "
"still a member, that the token is SSO-authorized where that "
"applies, and, for a GitHub App, that the installation covers "
"the repositories you expect.")
if state == "base-unreadable":
return ("re-read the organization with a token that has organization "
"access. Until then the coverage number is a measurement "
"without an explanation.")
if state == "base-none-explicit-grants-hold":
return ("nothing. Keep it that way: explicit grants are what makes "
"this account immune to the next change to the default.")
return ("nothing on the base permission. The shorter list, if there is "
"one, has another cause.")
def get(session, path):
"""One GET. Returns the response object."""
return session.get(API + path, timeout=30)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("org", help="the organization whose repositories shrank")
parser.add_argument("--expect",
help="the base permission this integration was "
"configured against, e.g. read")
args = parser.parse_args()
token = os.environ.get("GITHUB_TOKEN")
if not token:
log.error("set GITHUB_TOKEN (a read-only token is enough)")
return 2
log.info("read cost: %d request(s) against the core hourly quota", read_cost())
session = requests.Session()
session.headers.update({
"Authorization": "Bearer " + token,
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": UA,
})
org_response = get(session, "/orgs/" + args.org)
org_payload = org_response.json() if org_response.status_code == 200 else {}
base, base_detail = base_state(org_payload)
log.info("base permission: %s — %s", base or "unreadable", base_detail)
drift_state, drift_detail = drift(args.expect, base)
log.info("drift: %s — %s", drift_state, drift_detail)
mine = get(session, "/user/repos?affiliation=organization_member&per_page=1")
body = mine.json() if mine.status_code == 200 else []
visible, how = count_from_link(mine.headers.get("Link"),
len(body) if isinstance(body, list) else 0)
log.info("visible through membership: %s (%s)", visible, how)
total, total_detail = org_total(org_payload)
log.info("organization holds: %s repositories (%s)",
"unreadable" if total is None else total, total_detail)
also = get(session, "/orgs/%s/repos?per_page=1" % args.org)
also_body = also.json() if also.status_code == 200 else []
listed, listed_how = count_from_link(
also.headers.get("Link"), len(also_body) if isinstance(also_body, list) else 0)
log.info("listed by /orgs/%s/repos: %s (%s)", args.org, listed, listed_how)
coverage = coverage_state(visible, total)
log.info("coverage: %s — %s of %s", coverage, visible,
"unreadable" if total is None else total)
state, detail = verdict(base, coverage)
log.info("state: %s — %s", state, detail)
log.info("repair: %s", repair(state, args.org))
print(json.dumps({
"organization": args.org,
"default_repository_permission": base,
"expected_base_permission": args.expect,
"drift_state": drift_state,
"visible_through_membership": visible,
"visible_source": how,
"listed_by_org_repos": listed,
"organization_total": total,
"coverage": coverage,
"state": state,
"detail": detail,
"repair": repair(state, args.org),
}, indent=2, default=str))
return 1 if state in ("base-none-implicit-access-gone",
"coverage-lost-elsewhere") else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Say whether an organization's base permission is why the repository list shrank.
*
* Read only. Three GETs. Granting access to a repository is a write and
* somebody with admin has to make it, so this script measures the loss and
* prints the narrow repair.
*
* default_repository_permission is the role every member holds on
* repositories they were never explicitly added to. Moving it from read to
* none removes implicit access everywhere at once, so the integration keeps
* succeeding while covering a fraction of what it did yesterday.
*
* Environment:
* GITHUB_TOKEN a read-only token for the account whose coverage shrank
* GITHUB_ORG the organization whose repositories shrank
* GITHUB_EXPECT optional base permission you configured against
*/
const API = 'https://api.github.com';
const UA = 'github-org-base-permission/1.0';
/** Weakest first. The documented values of default_repository_permission. */
export const BASE_PERMISSIONS = ['none', 'read', 'write', 'admin'];
/** What a member with no explicit grants gets, per base permission. */
export const IMPLIES = {
none: 'members get no role on repositories they were not added to '
+ 'individually or through a team. Every private repository in the '
+ 'organization is invisible to a member with no explicit grants.',
read: 'every member can read every repository in the organization without '
+ 'being added to it.',
write: 'every member can push to every repository in the organization '
+ 'without being added to it.',
admin: 'every member administers every repository in the organization. This '
+ 'is rare and worth questioning on its own.',
};
/** Requests this run will spend against the core quota. Pure. */
export function readCost() {
return 3;
}
/** Position in the hierarchy, or -1 for something unrecognised. Pure. */
export function baseRank(value) {
return BASE_PERMISSIONS.indexOf(String(value ?? '').trim().toLowerCase());
}
/** The organization's base permission. Pure. [value, detail]. */
export function baseState(orgPayload) {
if (!orgPayload || typeof orgPayload !== 'object') {
return [null, 'no organization payload was read.'];
}
if (!Object.prototype.hasOwnProperty.call(orgPayload, 'default_repository_permission')) {
return [null, 'default_repository_permission was not returned. Reading it '
+ 'needs organization access, so this is unreadable rather than absent.'];
}
const value = String(orgPayload.default_repository_permission ?? '')
.trim().toLowerCase();
if (baseRank(value) < 0) {
return [value || null, `the value '${value}' is not one of the four `
+ 'documented base permissions.'];
}
return [value, IMPLIES[value]];
}
/** Split a Link header into its entries. Pure. No regular expression.
*
* Split on the commas that separate entries and not on the ones inside a URL,
* because a URL in a Link header can carry commas of its own.
*/
export function linkParts(linkHeader) {
const parts = [];
let current = '';
let depth = 0;
for (const ch of String(linkHeader ?? '')) {
if (ch === '<') depth += 1;
else if (ch === '>') depth = Math.max(0, depth - 1);
if (ch === ',' && depth === 0) { parts.push(current); current = ''; } else current += ch;
}
if (current.trim()) parts.push(current);
return parts;
}
/** The page number of rel="last", or null. Pure. No regular expression. */
export function lastPageFromLink(linkHeader) {
if (!linkHeader) return null;
for (const part of linkParts(linkHeader)) {
if (!part.includes('rel="last"') && !part.includes('rel=last')) continue;
const start = part.indexOf('<');
const end = part.indexOf('>', start + 1);
if (start < 0 || end < 0) continue;
const url = part.slice(start + 1, end);
const query = url.includes('?') ? url.slice(url.indexOf('?') + 1) : '';
for (const field of query.split('&')) {
const cut = field.indexOf('=');
const name = cut < 0 ? field : field.slice(0, cut);
const value = cut < 0 ? '' : field.slice(cut + 1);
if (name === 'page' && value.length && [...value].every((c) => c >= '0' && c <= '9')) {
return Number(value);
}
}
}
return null;
}
/** How many items the collection holds, at per_page=1. Pure. [count, how]. */
export function countFromLink(linkHeader, returned) {
const last = lastPageFromLink(linkHeader);
if (last !== null) return [last, 'from rel="last" with per_page=1'];
if (!returned) return [0, 'the first page came back empty and carried no rel="last"'];
return [Number(returned), 'a single page with no rel="last", so this is what '
+ 'came back rather than a measured count'];
}
/** How many repositories the organization holds. Pure. [count, detail]. */
export function orgTotal(orgPayload) {
if (!orgPayload || typeof orgPayload !== 'object') {
return [null, 'no organization payload was read.'];
}
const pub = orgPayload.public_repos;
const priv = orgPayload.total_private_repos;
if ((pub === null || pub === undefined) && (priv === null || priv === undefined)) {
return [null, 'neither repository count was returned, which needs '
+ 'organization access.'];
}
const total = Number(pub || 0) + Number(priv || 0);
return [total, `public ${pub ?? 'unreadable'} + private ${priv ?? 'unreadable'}`];
}
/** Grade what the account can see against what the org holds. Pure. */
export function coverageState(visible, total) {
if (total === null || total === undefined || visible === null
|| visible === undefined) {
return 'unknown';
}
if (total <= 0) return 'nothing-to-cover';
if (visible >= total) return 'full';
if (visible === 0 || visible * 20 < total) return 'collapsed';
if (visible * 2 < total) return 'shrunken';
return 'partial';
}
/** Compare the configured base permission against the live one. Pure. */
export function drift(expected, actual) {
if (!expected || actual === null || actual === undefined) {
return ['drift-unknown', 'no expected base permission was supplied, or the '
+ 'live one could not be read, so there is nothing to compare.'];
}
const want = baseRank(expected);
const have = baseRank(actual);
if (want < 0 || have < 0) {
return ['drift-unknown', 'one of the two values is not a documented base '
+ 'permission.'];
}
if (want === have) {
return ['base-unchanged', 'the organization still reports the base '
+ 'permission this integration was configured against.'];
}
if (have < want) {
return ['base-tightened', `configured for '${expected}', the organization `
+ `now says '${actual}'. That is one field and it re-graded every `
+ 'repository at once.'];
}
return ['base-loosened', `configured for '${expected}', the organization now `
+ `says '${actual}', which grants more implicit access than you expected `
+ 'rather than less.'];
}
/** The finding, in one state. Pure. [state, detail]. */
export function verdict(base, coverage) {
if (base === null || base === undefined) {
return ['base-unreadable', 'the base permission could not be read, so the '
+ 'coverage number stands on its own. Read it with a token that has '
+ 'organization access before concluding anything about the default.'];
}
if (base === 'none' && ['collapsed', 'shrunken'].includes(coverage)) {
return ['base-none-implicit-access-gone', 'base permission is none and this '
+ 'account reaches a fraction of the organization. The repositories it '
+ 'still reaches are the ones it was added to explicitly; the rest were '
+ 'never granted, only defaulted.'];
}
if (base === 'none' && ['full', 'partial'].includes(coverage)) {
return ['base-none-explicit-grants-hold', 'base permission is none and '
+ "coverage is largely intact, which means this account's access is "
+ 'explicit. It is not exposed to this change.'];
}
if (base !== 'none' && ['collapsed', 'shrunken'].includes(coverage)) {
return ['coverage-lost-elsewhere', 'the base permission still grants '
+ 'implicit access and the coverage is short anyway, so the loss is not '
+ "this field. Membership, SSO authorization and an App's repository "
+ 'selection are the other ways a list gets shorter.'];
}
if (coverage === 'nothing-to-cover') {
return ['nothing-to-cover', 'the organization reports no repositories, so '
+ 'there is no coverage question to answer.'];
}
return ['coverage-as-expected', 'the account reaches what the base permission '
+ 'implies it should. Nothing here explains a shorter list.'];
}
/** The narrow repair. Pure. Nothing here is executed. */
export function repair(state, org) {
if (state === 'base-none-implicit-access-gone') {
return `add this account, or a team it belongs to, to the repositories the `
+ `job is meant to cover in ${org}. Do not raise the base permission `
+ 'back: that re-grants implicit access to every member of the '
+ 'organization to fix one integration.';
}
if (state === 'coverage-lost-elsewhere') {
return 'look past the base permission. Check that the account is still a '
+ 'member, that the token is SSO-authorized where that applies, and, for '
+ 'a GitHub App, that the installation covers the repositories you expect.';
}
if (state === 'base-unreadable') {
return 're-read the organization with a token that has organization access. '
+ 'Until then the coverage number is a measurement without an explanation.';
}
if (state === 'base-none-explicit-grants-hold') {
return 'nothing. Keep it that way: explicit grants are what makes this '
+ 'account immune to the next change to the default.';
}
return 'nothing on the base permission. The shorter list, if there is one, '
+ 'has another cause.';
}
function headers(token) {
return {
Authorization: `Bearer ${token}`,
Accept: 'application/vnd.github+json',
'X-GitHub-Api-Version': '2022-11-28',
'User-Agent': UA,
};
}
async function main() {
const token = process.env.GITHUB_TOKEN;
const org = process.env.GITHUB_ORG;
if (!token || !org) {
console.error('set GITHUB_TOKEN (read-only is enough) and GITHUB_ORG');
process.exitCode = 2;
return;
}
const expect = process.env.GITHUB_EXPECT || '';
console.log(`read cost: ${readCost()} request(s) against the core hourly quota`);
const orgResponse = await fetch(`${API}/orgs/${org}`, { headers: headers(token) });
const orgPayload = orgResponse.status === 200 ? await orgResponse.json() : {};
const [base, baseDetail] = baseState(orgPayload);
console.log(`base permission: ${base || 'unreadable'} — ${baseDetail}`);
const [driftState, driftDetail] = drift(expect, base);
console.log(`drift: ${driftState} — ${driftDetail}`);
const mine = await fetch(
`${API}/user/repos?affiliation=organization_member&per_page=1`,
{ headers: headers(token) },
);
const body = mine.status === 200 ? await mine.json() : [];
const [visible, how] = countFromLink(mine.headers.get('link'),
Array.isArray(body) ? body.length : 0);
console.log(`visible through membership: ${visible} (${how})`);
const [total, totalDetail] = orgTotal(orgPayload);
console.log(`organization holds: ${total ?? 'unreadable'} repositories (${totalDetail})`);
const coverage = coverageState(visible, total);
console.log(`coverage: ${coverage} — ${visible} of ${total ?? 'unreadable'}`);
const [state, detail] = verdict(base, coverage);
console.log(`state: ${state} — ${detail}`);
console.log(`repair: ${repair(state, org)}`);
console.log(JSON.stringify({
organization: org,
default_repository_permission: base,
expected_base_permission: expect || null,
drift_state: driftState,
visible_through_membership: visible,
visible_source: how,
organization_total: total,
coverage,
state,
detail,
repair: repair(state, org),
}, null, 2));
process.exitCode = ['base-none-implicit-access-gone',
'coverage-lost-elsewhere'].includes(state) ? 1 : 0;
}
// Guarded so importing this file from the test runner does not start a run.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
Two things are worth testing hard here and the suite spends most of its length on them. The first is the counting trick, because it is the thing that makes the check affordable and the thing that is quietly wrong when a collection fits on one page: a missing rel="last" has to be reported as a single page rather than as a count of zero, and the parse has to survive a URL with commas in it. The second is the refusal to blame the field the script happens to have read: a collapsed coverage beside a base permission of read is asserted to come back as somebody else's problem, with the alternatives named. The drift comparison is asserted in both directions, because a base permission that got looser than your configuration is also worth an alert.
from github_org_base_permission import (
base_rank, base_state, count_from_link, coverage_state, drift,
last_page_from_link, org_total, read_cost, repair, verdict,
)
LINK = ('<https://api.github.com/user/repos?per_page=1&page=2>; rel="next", '
'<https://api.github.com/user/repos?per_page=1&page=9>; rel="last"')
ORG = {"default_repository_permission": "none",
"public_repos": 12, "total_private_repos": 400}
def test_the_last_page_number_is_the_count_at_one_per_page():
assert last_page_from_link(LINK) == 9
count, how = count_from_link(LINK, 1)
assert count == 9
assert 'rel="last"' in how
def test_a_single_page_is_not_a_count_of_zero():
# The collection fits on one page, so there is no rel="last" at all. Saying
# zero here would report a working integration as having lost everything.
count, how = count_from_link(None, 1)
assert count == 1
assert "single page" in how
assert count_from_link("", 0) == (0, 'the first page came back empty and '
'carried no rel="last"')
def test_the_link_parse_survives_a_url_with_commas_in_it():
header = ('<https://api.github.com/search?q=a,b&per_page=1&page=3>; rel="last"')
assert last_page_from_link(header) == 3
assert last_page_from_link('<https://x/?page=notanumber>; rel="last"') is None
assert last_page_from_link('<https://x/?page=2>; rel="next"') is None
def test_the_base_permission_says_what_it_implies():
value, detail = base_state(ORG)
assert value == "none"
assert "were not added to" in detail
assert base_state({"default_repository_permission": "read"})[1].startswith(
"every member can read")
assert base_rank("none") < base_rank("read") < base_rank("write")
def test_an_absent_base_permission_is_unreadable_not_none():
value, detail = base_state({"login": "acme"})
assert value is None
assert "unreadable rather than absent" in detail
assert verdict(None, "collapsed")[0] == "base-unreadable"
def test_the_organization_total_adds_both_halves():
total, detail = org_total(ORG)
assert total == 412
assert "public 12" in detail
assert org_total({"login": "acme"})[0] is None
def test_coverage_is_graded_rather_than_reported_as_a_ratio():
assert coverage_state(9, 412) == "collapsed"
assert coverage_state(0, 412) == "collapsed"
assert coverage_state(150, 412) == "shrunken"
assert coverage_state(300, 412) == "partial"
assert coverage_state(412, 412) == "full"
assert coverage_state(5, None) == "unknown"
assert coverage_state(0, 0) == "nothing-to-cover"
def test_the_finding_names_the_field_only_when_the_field_fits():
state, detail = verdict("none", "collapsed")
assert state == "base-none-implicit-access-gone"
assert "never granted, only defaulted" in detail
def test_a_collapsed_coverage_under_read_is_somebody_elses_problem():
# The script has just read the base permission, which makes it the easiest
# thing in the room to blame. It refuses.
state, detail = verdict("read", "collapsed")
assert state == "coverage-lost-elsewhere"
assert "not this field" in detail
assert "repository selection" in detail
def test_explicit_grants_are_reported_as_immunity():
state, detail = verdict("none", "full")
assert state == "base-none-explicit-grants-hold"
assert "not exposed to this change" in detail
def test_drift_is_reported_in_both_directions():
state, detail = drift("read", "none")
assert state == "base-tightened"
assert "re-graded every repository at once" in detail
assert drift("read", "write")[0] == "base-loosened"
assert drift("read", "read")[0] == "base-unchanged"
assert drift(None, "read")[0] == "drift-unknown"
assert drift("read", None)[0] == "drift-unknown"
def test_the_repair_refuses_to_recommend_the_easy_fix():
fix = repair("base-none-implicit-access-gone", "acme")
assert "add this account" in fix and "acme" in fix
assert "Do not raise the base permission back" in fix
assert "still a member" in repair("coverage-lost-elsewhere", "acme")
def test_the_run_costs_three_reads():
assert read_cost() == 3
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
baseRank, baseState, countFromLink, coverageState, drift, lastPageFromLink,
orgTotal, readCost, repair, verdict,
} from './github-org-base-permission.mjs';
const LINK = '<https://api.github.com/user/repos?per_page=1&page=2>; rel="next", '
+ '<https://api.github.com/user/repos?per_page=1&page=9>; rel="last"';
const ORG = {
default_repository_permission: 'none',
public_repos: 12,
total_private_repos: 400,
};
test('the last page number is the count at one per page', () => {
assert.equal(lastPageFromLink(LINK), 9);
const [count, how] = countFromLink(LINK, 1);
assert.equal(count, 9);
assert.match(how, /rel="last"/);
});
test('a single page is not a count of zero', () => {
const [count, how] = countFromLink(null, 1);
assert.equal(count, 1);
assert.match(how, /single page/);
assert.equal(countFromLink('', 0)[0], 0);
});
test('the link parse survives a url with commas in it', () => {
const header = '<https://api.github.com/search?q=a,b&per_page=1&page=3>; rel="last"';
assert.equal(lastPageFromLink(header), 3);
assert.equal(lastPageFromLink('<https://x/?page=notanumber>; rel="last"'), null);
assert.equal(lastPageFromLink('<https://x/?page=2>; rel="next"'), null);
});
test('the base permission says what it implies', () => {
const [value, detail] = baseState(ORG);
assert.equal(value, 'none');
assert.match(detail, /were not added to/);
assert.ok(baseRank('none') < baseRank('read'));
assert.ok(baseRank('read') < baseRank('write'));
});
test('an absent base permission is unreadable not none', () => {
const [value, detail] = baseState({ login: 'acme' });
assert.equal(value, null);
assert.match(detail, /unreadable rather than absent/);
assert.equal(verdict(null, 'collapsed')[0], 'base-unreadable');
});
test('the organization total adds both halves', () => {
const [total, detail] = orgTotal(ORG);
assert.equal(total, 412);
assert.match(detail, /public 12/);
assert.equal(orgTotal({ login: 'acme' })[0], null);
});
test('coverage is graded rather than reported as a ratio', () => {
assert.equal(coverageState(9, 412), 'collapsed');
assert.equal(coverageState(0, 412), 'collapsed');
assert.equal(coverageState(150, 412), 'shrunken');
assert.equal(coverageState(300, 412), 'partial');
assert.equal(coverageState(412, 412), 'full');
assert.equal(coverageState(5, null), 'unknown');
assert.equal(coverageState(0, 0), 'nothing-to-cover');
});
test('the finding names the field only when the field fits', () => {
const [state, detail] = verdict('none', 'collapsed');
assert.equal(state, 'base-none-implicit-access-gone');
assert.match(detail, /never granted, only defaulted/);
});
test('a collapsed coverage under read is somebody elses problem', () => {
const [state, detail] = verdict('read', 'collapsed');
assert.equal(state, 'coverage-lost-elsewhere');
assert.match(detail, /not this field/);
assert.match(detail, /repository selection/);
});
test('explicit grants are reported as immunity', () => {
const [state, detail] = verdict('none', 'full');
assert.equal(state, 'base-none-explicit-grants-hold');
assert.match(detail, /not exposed to this change/);
});
test('drift is reported in both directions', () => {
const [state, detail] = drift('read', 'none');
assert.equal(state, 'base-tightened');
assert.match(detail, /re-graded every repository at once/);
assert.equal(drift('read', 'write')[0], 'base-loosened');
assert.equal(drift('read', 'read')[0], 'base-unchanged');
assert.equal(drift(null, 'read')[0], 'drift-unknown');
assert.equal(drift('read', null)[0], 'drift-unknown');
});
test('the repair refuses to recommend the easy fix', () => {
const fix = repair('base-none-implicit-access-gone', 'acme');
assert.match(fix, /add this account/);
assert.match(fix, /Do not raise the base permission back/);
assert.match(repair('coverage-lost-elsewhere', 'acme'), /still a member/);
});
test('the run costs three reads', () => {
assert.equal(readCost(), 3);
});
FAQ
Nothing was revoked from our account. How can it have lost access?
Because most of that access was never granted to the account in the first place. A member's role on a repository they were not added to comes from the organization's default, and a default is not a grant — there is no record on the repository naming your account, nothing to revoke and nothing in an audit trail about your account at all. The change is one field on the organization, and it applies to every member and every repository the moment it is saved.
How is this different from a role being too low on one repository?
Different object, different symptom. A role that is too low on one repository refuses one action on one repository, loudly, with a 403 you can catch. A base permission is organization-wide and re-grades every repository at once, and its symptom is that a list came back shorter while every call in it returned 200. That is why the diagnostic here is a count rather than a permissions object: there is no single failing call to inspect.
Why count with rel="last" instead of listing everything?
Because the affordable version of a check is the one that gets run. Asking for one item per page and reading the last page number gives you the size of a collection in a single request, so a four-hundred-repository organization costs one unit of quota to measure rather than four hundred. The catch is that a collection which fits on one page has no <code>rel="last"</code> at all, and a script that reads that as zero has just invented a catastrophe. The count is reported with its source for exactly that reason.
Should we set the base permission back to read?
Almost certainly not. Base permission at <code>read</code> means every member of the organization can read every repository in it, which is a much bigger grant than the one your integration needs, and moving it back to fix one job undoes a deliberate security decision for everybody. Add the account, or a team it belongs to, to the repositories the job covers. Explicit grants also make the integration immune to the next change to the default, which is the real win.
Our coverage collapsed and the base permission is still read. Then what?
Then it is not this. The script says so rather than blaming the field it happens to have read. A list gets shorter for several other reasons: the account may no longer be a member of the organization, the token may not be SSO-authorized for it, or — for a GitHub App — the installation may cover only selected repositories. Each of those is its own note, and each has a different single reading that confirms it.
Related field notes
- The same question about one repository and one account
- When the counting trick has no last page to read
- The App version of silently partial coverage
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Setting base permissions for an organization — GitHub Docs
- Organizations — GitHub REST API
- Repositories — GitHub REST API
- Using pagination in the REST API — GitHub Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.