Diagnostic GitHub API
A repository went private and anonymous callers now see 404
The dependency dashboard has read one repository's tags and release notes since 2019, anonymously, because it is a public repository and there was never any reason to authenticate. On a Tuesday it starts returning 404. Nothing was deployed, no code changed, the URL is character-for-character the one that worked on Monday. Somebody checks whether the repository was deleted, finds it in their browser because their browser is logged in, and now has two contradictory facts and no idea which to believe.
Take two readings of the same URL and compare them. GET /repos/{owner}/{repo} with a token that still has access, and the same call with no credential at all. A 200 authenticated beside a 404 anonymous is a repository that is no longer public. Neither half proves it alone, which is why one reading has kept people arguing.
The authenticated response then says which state it moved to: private: true with visibility of private or internal. Those two are not the same and the boolean cannot tell them apart — an internal repository is readable by every member of the enterprise and reports private: true exactly like a private one.
The repair is a credential, not a URL. A classic token needs the repo scope, because public_repo covers precisely what has just stopped being true; a fine-grained token needs Metadata: Read plus Contents: Read on that repository, granted by its owner. And GET /rate_limit, which is free, tells you whether your client was ever authenticated at all: a core.limit of 60 means it was not.
The problem in plain words
The status code is the problem. GitHub answers 404 rather than 403 for a private resource a caller cannot see, deliberately, so that error codes cannot be used to enumerate repositories that exist. The consequence is that "this repository is now private" and "this repository was deleted" arrive as the same bytes, and a client has no way to tell them apart from its own side of the call.
What follows is a specific and unproductive argument. One person, logged in, opens the URL in a browser and sees the repository. The dashboard, which is not logged in, insists it is gone. Both are right. The disagreement usually resolves as "the API is flaky" or "GitHub deleted it and restored it", and the integration gets a retry loop, or gets pointed at a mirror, or gets quietly disabled.
The forks make it worse. When a public repository becomes private, the forks that already existed are split off into a network of their own and stay public, so search results and vendored links still find something that looks like the repository. A dependency tool that follows one of those is now tracking a copy that has stopped receiving upstream commits, which produces a second, slower failure a month later.
And the anonymity is often accidental. Plenty of clients were written against public repositories, never given a token, and then inherited by somebody who assumes there is one, because there is a GITHUB_TOKEN in the environment somewhere. Checking that assumption is one free call and it is almost never made.
Why it happens
Going private removes anonymous access completely. Not partially, not for some endpoints. An unauthenticated client sees the same 404 for a private repository as it does for one that never existed. That is a design decision rather than an oversight, and it means the only way to distinguish the cases is to bring a credential that does have access and take a second reading.
Two readings are the evidence, and each alone is ambiguous. An anonymous 404 by itself is consistent with deletion, with a typo, with a rename, and with going private. An authenticated 200 by itself just says you can read it. Together they say: this exists, and it is not public. That pair is what this note is for, and it is the reading the general 404 triage does not take, because that note is answering a wider question for a caller who may never have had access at all.
private and visibility are not the same field. An internal repository — visible to every member of an enterprise, invisible to the public — reports private: true. A client keying on the boolean sees internal and private as identical, and they have different repairs: internal access is granted by enterprise membership, private access by a per-repository grant. Read visibility, which has three values, and keep the boolean for backwards compatibility only.
public_repo is the scope that describes what just changed. A classic token holding public_repo can read every public repository and no private one, so it is exactly as blind as no token at all here. The upgrade is repo, which is read and write together and is more authority than a reader needs; a fine-grained token with Metadata: Read and Contents: Read on that one repository is the narrower credential and the better answer.
The moment of the change is not readable, and pretending otherwise would be worse than not knowing. There is no visibility-changed timestamp on the repository object. updated_at moves for a dozen unrelated reasons, and the organization audit log, which does record it, needs admin:org-class access that a repository-scoped reader does not have. So the honest finding is the current asymmetry, plus whatever your own logs say about when the 404s started. The script reports the state, not a date it cannot see.
Every part of this is a read, including the anonymous one. Two GETs against the repository and one free GET /rate_limit. The anonymous call is billed to the unauthenticated bucket for your IP address, which is 60 an hour and shared with anything else on that address, so the script says so before spending one.
The fix, as a flow
One reading cannot answer this, so the script takes two of the same URL: once with the token and once with no credential at all. The pair is the finding. A token that sees 200 beside an anonymous request that sees 404 is a repository that is no longer public, which is a different sentence from the repository is gone, and the two are indistinguishable from either half alone. The unauthenticated rate limit is read for free alongside, because a limit of 60 is proof that the caller really was anonymous rather than quietly authenticated by an environment variable somebody forgot about.
How to fix it
Read the URL twice, once with the token and once with nothing
The script sends GET /repos/{owner}/{repo} authenticated, then sends the identical call from a session carrying no Authorization header at all. Two reads, and the pair of status codes is the finding. It prints both before interpreting either, so you can see the asymmetry yourself rather than take the verdict on trust.
Confirm your client really is anonymous
GET /rate_limit costs nothing and reports core.limit as 60 for an unauthenticated caller and 5,000 for an authenticated one. That single number settles the question people spend an afternoon on: whether the failing integration was ever sending a credential. It is checked for both sessions, because a token that has expired authenticates as nobody and produces the anonymous limit.
Read the state it moved to, not just the boolean
The authenticated response carries visibility with three possible values. The script reports it beside private and names them separately, because internal is private: true with a completely different repair: enterprise membership rather than a repository grant.
Rule out deletion and renaming explicitly
404 on both readings is not this note, and the script says so rather than guessing: it hands you to the wider 404 triage, because a repository you cannot see with any credential is either gone or was never granted to you. A 301 on either reading is a rename and belongs somewhere else again. Only the asymmetric pair is reported as a visibility change.
Take the repair to the credential, and pick the narrow one
The printed repair names what the client needs: repo for a classic token, or Metadata: Read and Contents: Read on that repository for a fine-grained one, with the owner granting access. If the client is holding public_repo the script names that specifically, because it is the scope that looks like it should work and cannot. Nothing is changed by the script.
How to check it worked
Once the client carries a credential with access, the two readings stop disagreeing in the way that matters and the state is reported rather than inferred.
GITHUB_TOKEN=$READ_ONLY_TOKEN python3 github_visibility_change.py \
octo-org/schema-registry
# read cost: 2 request(s) against the core hourly quota, plus 2 free
# /rate_limit calls. One of the reads is anonymous and is billed to the
# unauthenticated bucket for this IP address, which is 60 an hour.
# authenticated: HTTP 200 core.limit=5000
# anonymous: HTTP 404 core.limit=60
# octo-org/schema-registry: private=True visibility=private
# went-private: the repository is readable with a token and invisible without
# one, so it exists and is no longer public. Deletion would answer 404 to
# both readings.
# forks-note: forks that existed while it was public were split into their own
# network and are still public, so a link that still resolves may be a copy
# that stopped receiving commits.
# blind-spot: no visibility-changed timestamp is exposed to a reader, and the
# audit log that records it needs org-level access. When it happened is in
# your own logs, not in this response.
# repair: give the client the repo scope (classic) or Metadata: Read and
# Contents: Read on this repository (fine-grained), granted by the owner.
# public_repo covers exactly what has stopped being true.The full code
The unusual part of this script is the session that carries no credential. It is built deliberately and kept separate from the authenticated one, because the whole method is a comparison between two callers and mixing them would silently destroy the finding. Everything after the two reads is a small classifier over a pair of status codes plus the visibility field, with the ambiguous combinations named as ambiguous rather than resolved: 404 to both readings is handed to the wider triage, and a 301 anywhere is a rename and somebody else's note.
"""Tell a repository that went private apart from one that was deleted.
Read only. GET requests and nothing else, and one of them carries no credential
at all: the method here is a comparison between two callers reading the same
URL, so the anonymous session is built on purpose and kept separate from the
authenticated one.
The point of the note: making a repository private removes anonymous access
entirely, and GitHub answers 404 rather than 403 so that error codes cannot be
used to enumerate resources. A client that read the repository anonymously for
years therefore sees exactly what it would see if the repository had been
deleted. One reading cannot separate those. Two can.
What this can and cannot see: there is no visibility-changed timestamp on the
repository object. updated_at moves for unrelated reasons and the audit log
that records the change needs organization-level access. So this reports the
current asymmetry and never a date it cannot read.
Environment:
GITHUB_TOKEN a token that still has access to the repository
"""
import argparse
import json
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("github_visibility_change")
API = "https://api.github.com"
UA = "github-visibility-change/1.0"
# The unauthenticated core quota, per IP address. Reading this number is how a
# client proves whether it is authenticated at all, and the read is free.
ANONYMOUS_CORE_LIMIT = 60
# The three values of `visibility`. `private` the boolean is true for two of
# them, which is why a client keying on the boolean cannot tell them apart.
VISIBILITIES = ("public", "private", "internal")
# The classic scope that describes exactly what has stopped being true.
BLIND_SCOPE = "public_repo"
PRIVATE_SCOPE = "repo"
# What a fine-grained token needs instead, on that one repository.
FINE_GRAINED_PERMISSIONS = ("Metadata: Read", "Contents: Read")
def read_cost():
"""Billable requests this run spends against the core quota. Pure.
Two repository reads. The two /rate_limit calls are free and one of the
repository reads is billed to the unauthenticated bucket for this IP
address rather than to the token.
"""
return 2
def client_is_anonymous(core_limit):
"""Was this caller authenticated. Pure.
A limit of 60 is the unauthenticated bucket. An expired or revoked token
authenticates as nobody and produces the same number, which is why this is
a better question than "is a token set in the environment".
"""
try:
return int(core_limit) <= ANONYMOUS_CORE_LIMIT
except (TypeError, ValueError):
return None
def visibility_of(repo):
"""The three-valued visibility, falling back to the boolean. Pure."""
repo = repo or {}
value = str(repo.get("visibility") or "").strip().lower()
if value in VISIBILITIES:
return value
if repo.get("private") is True:
return "private"
if repo.get("private") is False:
return "public"
return "unreported"
def scope_list(header_value):
"""Read x-oauth-scopes into a list, keeping absent and empty apart. Pure."""
if header_value is None:
return None
return [s.strip() for s in header_value.split(",") if s.strip()]
def scope_gap(scopes, visibility):
"""Is the client's scope set exactly the wrong shape for this. Pure.
Returns (state, detail). public_repo is the interesting one: it grants
every public repository and no private one, so it is precisely as useful
as no token at all once the repository stops being public.
"""
if visibility == "public":
return ("not-applicable",
"the repository is public, so no scope is required to read it.")
if scopes is None:
return ("no-scopes-reported",
"this credential reports no OAuth scopes, so it is a "
"fine-grained or App token. It needs %s on this repository, "
"granted by the owner." % ", ".join(FINE_GRAINED_PERMISSIONS))
if PRIVATE_SCOPE in scopes:
return ("scope-sufficient",
"the token carries '%s', which covers a private repository. If "
"it still cannot read this one, the account behind it has no "
"grant on the repository." % PRIVATE_SCOPE)
if BLIND_SCOPE in scopes:
return ("blind-scope",
"the token carries '%s' and not '%s'. That scope grants every "
"public repository and no private one, so it is exactly as "
"blind here as sending no token at all."
% (BLIND_SCOPE, PRIVATE_SCOPE))
return ("scope-insufficient",
"the token carries %s, none of which reaches a private repository. "
"It needs '%s'." % (", ".join(scopes) or "no scopes at all",
PRIVATE_SCOPE))
def classify(anon_status, auth_status, repo=None):
"""Sort a pair of readings of one URL. Pure. Returns (state, detail).
The ambiguous combinations are named as ambiguous. Resolving them here
would be the same mistake the 404 makes, committed on purpose.
"""
visibility = visibility_of(repo)
if str(anon_status) == "301" or str(auth_status) == "301":
return ("moved",
"a 301 means the repository was renamed or transferred and a "
"redirect was left behind. That is a different note; follow it "
"once and rewrite your configuration.")
if auth_status == 200 and anon_status == 404:
if visibility == "internal":
return ("internal-visibility",
"the repository is internal: private=true, but readable by "
"every member of the enterprise rather than by a named list. "
"A client keying on the private boolean cannot see that "
"difference, and the repair for it is membership rather "
"than a repository grant.")
return ("went-private",
"the repository is readable with a token and invisible without "
"one, so it exists and is no longer public. Deletion would "
"answer 404 to both readings.")
if auth_status == 200 and anon_status == 200:
return ("still-public",
"both readings succeeded, so visibility is not what broke. The "
"404 your client recorded has another cause.")
if auth_status == 404 and anon_status == 404:
return ("invisible-to-both",
"neither reading can see it, so this is deletion or an account "
"that was never granted access. That is the wider 404 triage "
"and not this note.")
if auth_status != 200 and anon_status == 200:
return ("token-is-the-problem",
"the anonymous read succeeded and the authenticated one did "
"not, so the repository is public and the credential is "
"failing. Check whether the token is expired or revoked.")
return ("unclassified",
"authenticated %s and anonymous %s is not a combination this sorts. "
"Report both codes before drawing a conclusion."
% (auth_status, anon_status))
def fork_fallout(repo):
"""The second, slower failure this change produces. Pure, or None.
Forks that existed while the repository was public are split into their own
network and stay public. Something that looks like the repository therefore
still resolves, and a tool that follows it starts tracking a copy that no
longer receives commits.
"""
repo = repo or {}
if visibility_of(repo) == "public":
return None
if (repo.get("forks_count") or 0) <= 0:
return None
return ("forks that existed while it was public were split into their own "
"network and are still public, so a link that still resolves may be "
"a copy that stopped receiving commits.")
def blind_spot():
"""What this cannot establish, said out loud. Pure."""
return ("no visibility-changed timestamp is exposed to a reader, and the "
"audit log that records it needs organization-level access. When it "
"happened is in your own logs, not in this response.")
def repair(state, scope_state=None):
"""The sentence a reader has to act on. Pure."""
credential = ("give the client the '%s' scope (classic) or %s on this "
"repository (fine-grained), granted by the owner."
% (PRIVATE_SCOPE, " and ".join(FINE_GRAINED_PERMISSIONS)))
if state == "went-private":
text = credential
if scope_state == "blind-scope":
text += (" The scope it holds now, '%s', covers exactly what has "
"stopped being true." % BLIND_SCOPE)
return text
if state == "internal-visibility":
return ("the repository is internal, so access follows enterprise "
"membership. A machine account has to be a member of the "
"enterprise, and after that " + credential)
if state == "invisible-to-both":
return ("stop here and run the wider 404 triage. Nothing about "
"visibility can be established when no credential can see it.")
if state == "still-public":
return ("look elsewhere. The repository is public and readable "
"anonymously, so the 404 came from something other than "
"visibility.")
if state == "token-is-the-problem":
return ("check the credential rather than the repository. An expired "
"or revoked token authenticates as nobody.")
if state == "moved":
return ("follow the redirect once, take full_name from the response, "
"and store the repository id so the next rename is not a "
"surprise either.")
return "report both status codes before drawing a conclusion."
def core_limit(session):
"""core.limit for whichever caller this session is. Free to read."""
r = session.get(API + "/rate_limit", timeout=30)
if r.status_code != 200:
return None
try:
return ((r.json().get("resources") or {}).get("core") or {}).get("limit")
except ValueError:
return None
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("repo", help="owner/name of the repository that started 404ing")
args = ap.parse_args()
token = os.environ.get("GITHUB_TOKEN")
if not token:
log.error("set GITHUB_TOKEN (a read-only token with access is enough)")
return 2
if "/" not in args.repo:
log.error("repo should be owner/name")
return 2
log.info("read cost: %d request(s) against the core hourly quota, plus 2 "
"free /rate_limit calls. One of the reads is anonymous and is "
"billed to the unauthenticated bucket for this IP address, which "
"is %d an hour.", read_cost(), ANONYMOUS_CORE_LIMIT)
common = {
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
# GitHub rejects requests with no User-Agent outright.
"User-Agent": UA,
}
authed = requests.Session()
authed.headers.update(dict(common, Authorization="Bearer " + token))
# Deliberately credential-free. The whole method is a comparison between
# two callers, so this session must never acquire an Authorization header.
anon = requests.Session()
anon.headers.update(common)
auth_limit = core_limit(authed)
anon_limit = core_limit(anon)
auth_response = authed.get(API + "/repos/" + args.repo, timeout=30,
allow_redirects=False)
anon_response = anon.get(API + "/repos/" + args.repo, timeout=30,
allow_redirects=False)
log.info("authenticated: HTTP %s core.limit=%s", auth_response.status_code,
auth_limit)
log.info("anonymous: HTTP %s core.limit=%s", anon_response.status_code,
anon_limit)
if client_is_anonymous(auth_limit):
log.warning("the authenticated session reports the unauthenticated "
"limit, so GITHUB_TOKEN is expired, revoked or not being "
"sent. Fix that before reading anything else here.")
repo = None
scopes = scope_list(auth_response.headers.get("x-oauth-scopes"))
if auth_response.status_code == 200:
repo = auth_response.json()
log.info("%s: private=%s visibility=%s", args.repo, repo.get("private"),
visibility_of(repo))
state, detail = classify(anon_response.status_code,
auth_response.status_code, repo)
log.info("%s: %s", state, detail)
scope_state, scope_detail = scope_gap(scopes, visibility_of(repo))
log.info("%s: %s", scope_state, scope_detail)
fallout = fork_fallout(repo)
if fallout:
log.info("forks-note: %s", fallout)
log.info("blind-spot: %s", blind_spot())
log.info("repair: %s", repair(state, scope_state))
print(json.dumps({
"repository": args.repo,
"authenticated_status": auth_response.status_code,
"anonymous_status": anon_response.status_code,
"authenticated_core_limit": auth_limit,
"anonymous_core_limit": anon_limit,
"client_was_anonymous": client_is_anonymous(auth_limit),
"private": (repo or {}).get("private"),
"visibility": visibility_of(repo),
"scopes": scopes,
"state": state,
"detail": detail,
"scope_state": scope_state,
"forks_note": fallout,
"blind_spot": blind_spot(),
"repair": repair(state, scope_state),
}, indent=2, default=str))
return 1 if state in ("went-private", "internal-visibility") else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Tell a repository that went private apart from one that was deleted.
*
* Read only. GET requests and nothing else, and one of them carries no
* credential at all: the method is a comparison between two callers reading the
* same URL, so the anonymous request is made on purpose and never given an
* Authorization header.
*
* Making a repository private removes anonymous access entirely, and GitHub
* answers 404 rather than 403 so error codes cannot enumerate resources. A
* client that read it anonymously for years sees exactly what it would see if
* the repository had been deleted. One reading cannot separate those; two can.
*
* Environment:
* GITHUB_TOKEN a token that still has access to the repository
* GITHUB_REPO owner/name of the repository that started 404ing
*/
const API = 'https://api.github.com';
const UA = 'github-visibility-change/1.0';
/** The unauthenticated core quota, per IP address. Free to read. */
export const ANONYMOUS_CORE_LIMIT = 60;
/** The three values of visibility. The boolean is true for two of them. */
export const VISIBILITIES = ['public', 'private', 'internal'];
/** The classic scope that describes exactly what has stopped being true. */
export const BLIND_SCOPE = 'public_repo';
export const PRIVATE_SCOPE = 'repo';
/** What a fine-grained token needs instead, on that one repository. */
export const FINE_GRAINED_PERMISSIONS = ['Metadata: Read', 'Contents: Read'];
/** Billable requests this run spends against the core quota. Pure. */
export function readCost() {
return 2;
}
/** Was this caller authenticated. Pure. */
export function clientIsAnonymous(coreLimit) {
if (coreLimit === null || coreLimit === undefined || coreLimit === '') return null;
const n = Number(coreLimit);
if (!Number.isFinite(n)) return null;
return n <= ANONYMOUS_CORE_LIMIT;
}
/** The three-valued visibility, falling back to the boolean. Pure. */
export function visibilityOf(repo) {
const r = repo || {};
const value = String(r.visibility ?? '').trim().toLowerCase();
if (VISIBILITIES.includes(value)) return value;
if (r.private === true) return 'private';
if (r.private === false) return 'public';
return 'unreported';
}
/** Read x-oauth-scopes into a list, keeping absent and empty apart. Pure. */
export function scopeList(headerValue) {
if (headerValue === null || headerValue === undefined) return null;
return String(headerValue).split(',').map((s) => s.trim()).filter(Boolean);
}
/** Is the client's scope set exactly the wrong shape for this. Pure. */
export function scopeGap(scopes, visibility) {
if (visibility === 'public') {
return ['not-applicable', 'the repository is public, so no scope is '
+ 'required to read it.'];
}
if (scopes === null || scopes === undefined) {
return ['no-scopes-reported', 'this credential reports no OAuth scopes, so '
+ `it is a fine-grained or App token. It needs ${FINE_GRAINED_PERMISSIONS.join(', ')} `
+ 'on this repository, granted by the owner.'];
}
if (scopes.includes(PRIVATE_SCOPE)) {
return ['scope-sufficient', `the token carries '${PRIVATE_SCOPE}', which `
+ 'covers a private repository. If it still cannot read this one, the '
+ 'account behind it has no grant on the repository.'];
}
if (scopes.includes(BLIND_SCOPE)) {
return ['blind-scope', `the token carries '${BLIND_SCOPE}' and not `
+ `'${PRIVATE_SCOPE}'. That scope grants every public repository and no `
+ 'private one, so it is exactly as blind here as sending no token at all.'];
}
return ['scope-insufficient', `the token carries ${scopes.join(', ') || 'no scopes at all'}, `
+ `none of which reaches a private repository. It needs '${PRIVATE_SCOPE}'.`];
}
/** Sort a pair of readings of one URL. Pure. [state, detail]. */
export function classify(anonStatus, authStatus, repo = null) {
const visibility = visibilityOf(repo);
if (String(anonStatus) === '301' || String(authStatus) === '301') {
return ['moved', 'a 301 means the repository was renamed or transferred and '
+ 'a redirect was left behind. That is a different note; follow it once '
+ 'and rewrite your configuration.'];
}
if (authStatus === 200 && anonStatus === 404) {
if (visibility === 'internal') {
return ['internal-visibility', 'the repository is internal: private=true, '
+ 'but readable by every member of the enterprise rather than by a named '
+ 'list. A client keying on the private boolean cannot see that '
+ 'difference, and the repair for it is membership rather than a '
+ 'repository grant.'];
}
return ['went-private', 'the repository is readable with a token and '
+ 'invisible without one, so it exists and is no longer public. Deletion '
+ 'would answer 404 to both readings.'];
}
if (authStatus === 200 && anonStatus === 200) {
return ['still-public', 'both readings succeeded, so visibility is not what '
+ 'broke. The 404 your client recorded has another cause.'];
}
if (authStatus === 404 && anonStatus === 404) {
return ['invisible-to-both', 'neither reading can see it, so this is '
+ 'deletion or an account that was never granted access. That is the wider '
+ '404 triage and not this note.'];
}
if (authStatus !== 200 && anonStatus === 200) {
return ['token-is-the-problem', 'the anonymous read succeeded and the '
+ 'authenticated one did not, so the repository is public and the '
+ 'credential is failing. Check whether the token is expired or revoked.'];
}
return ['unclassified', `authenticated ${authStatus} and anonymous `
+ `${anonStatus} is not a combination this sorts. Report both codes before `
+ 'drawing a conclusion.'];
}
/** The second, slower failure this change produces. Pure, or null. */
export function forkFallout(repo) {
const r = repo || {};
if (visibilityOf(r) === 'public') return null;
if ((r.forks_count || 0) <= 0) return null;
return 'forks that existed while it was public were split into their own '
+ 'network and are still public, so a link that still resolves may be a copy '
+ 'that stopped receiving commits.';
}
/** What this cannot establish, said out loud. Pure. */
export function blindSpot() {
return 'no visibility-changed timestamp is exposed to a reader, and the audit '
+ 'log that records it needs organization-level access. When it happened is '
+ 'in your own logs, not in this response.';
}
/** The sentence a reader has to act on. Pure. */
export function repair(state, scopeState = null) {
const credential = `give the client the '${PRIVATE_SCOPE}' scope (classic) or `
+ `${FINE_GRAINED_PERMISSIONS.join(' and ')} on this repository `
+ '(fine-grained), granted by the owner.';
if (state === 'went-private') {
let text = credential;
if (scopeState === 'blind-scope') {
text += ` The scope it holds now, '${BLIND_SCOPE}', covers exactly what `
+ 'has stopped being true.';
}
return text;
}
if (state === 'internal-visibility') {
return 'the repository is internal, so access follows enterprise membership. '
+ 'A machine account has to be a member of the enterprise, and after that '
+ credential;
}
if (state === 'invisible-to-both') {
return 'stop here and run the wider 404 triage. Nothing about visibility can '
+ 'be established when no credential can see it.';
}
if (state === 'still-public') {
return 'look elsewhere. The repository is public and readable anonymously, '
+ 'so the 404 came from something other than visibility.';
}
if (state === 'token-is-the-problem') {
return 'check the credential rather than the repository. An expired or '
+ 'revoked token authenticates as nobody.';
}
if (state === 'moved') {
return 'follow the redirect once, take full_name from the response, and '
+ 'store the repository id so the next rename is not a surprise either.';
}
return 'report both status codes before drawing a conclusion.';
}
const COMMON = {
Accept: 'application/vnd.github+json',
'X-GitHub-Api-Version': '2022-11-28',
'User-Agent': UA,
};
async function coreLimit(headers) {
const res = await fetch(`${API}/rate_limit`, { headers });
if (!res.ok) return null;
try {
const body = await res.json();
return ((body.resources || {}).core || {}).limit ?? null;
} catch { return null; }
}
async function main() {
const token = process.env.GITHUB_TOKEN;
const repoName = process.env.GITHUB_REPO;
if (!token || !repoName) {
console.error('set GITHUB_TOKEN (read-only with access) and GITHUB_REPO');
process.exitCode = 2;
return;
}
console.log(`read cost: ${readCost()} request(s) against the core hourly quota, `
+ `plus 2 free /rate_limit calls. One read is anonymous and is billed to the `
+ `unauthenticated bucket for this IP address, which is ${ANONYMOUS_CORE_LIMIT} an hour.`);
const authedHeaders = { ...COMMON, Authorization: `Bearer ${token}` };
// Deliberately credential-free, and it must stay that way.
const anonHeaders = { ...COMMON };
const authLimit = await coreLimit(authedHeaders);
const anonLimit = await coreLimit(anonHeaders);
const authRes = await fetch(`${API}/repos/${repoName}`,
{ headers: authedHeaders, redirect: 'manual' });
const anonRes = await fetch(`${API}/repos/${repoName}`,
{ headers: anonHeaders, redirect: 'manual' });
console.log(`authenticated: HTTP ${authRes.status} core.limit=${authLimit}`);
console.log(`anonymous: HTTP ${anonRes.status} core.limit=${anonLimit}`);
let repo = null;
const scopes = scopeList(authRes.headers.get('x-oauth-scopes'));
if (authRes.status === 200) {
repo = await authRes.json();
console.log(`${repoName}: private=${repo.private} visibility=${visibilityOf(repo)}`);
}
const [state, detail] = classify(anonRes.status, authRes.status, repo);
console.log(`${state}: ${detail}`);
const [scopeState, scopeDetail] = scopeGap(scopes, visibilityOf(repo));
console.log(`${scopeState}: ${scopeDetail}`);
const fallout = forkFallout(repo);
if (fallout) console.log(`forks-note: ${fallout}`);
console.log(`blind-spot: ${blindSpot()}`);
console.log(`repair: ${repair(state, scopeState)}`);
console.log(JSON.stringify({
repository: repoName,
authenticated_status: authRes.status,
anonymous_status: anonRes.status,
authenticated_core_limit: authLimit,
anonymous_core_limit: anonLimit,
client_was_anonymous: clientIsAnonymous(authLimit),
private: (repo || {}).private ?? null,
visibility: visibilityOf(repo),
scopes,
state,
detail,
scope_state: scopeState,
forks_note: fallout,
blind_spot: blindSpot(),
repair: repair(state, scopeState),
}, null, 2));
process.exitCode = ['went-private', 'internal-visibility'].includes(state) ? 1 : 0;
}
// Guarded so importing this file from the test runner does not start a run and
// fail the suite even as every test passes.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The classifier is tested as a matrix of pairs rather than as a set of single readings, because that is the claim the note makes: 200-with-a-token beside 404-without-one is a visibility change, and 404 to both is not this note at all and has to say so. The internal case gets its own assertions since it is private: true and still not private, and the boolean cannot separate them. Then the scope arithmetic, where public_repo has to come back as blind rather than as merely insufficient, and finally the two honest refusals: the missing timestamp and the ambiguous pair, both of which are asserted to be reported rather than resolved.
from github_visibility_change import (
ANONYMOUS_CORE_LIMIT, BLIND_SCOPE, PRIVATE_SCOPE, blind_spot, classify,
client_is_anonymous, fork_fallout, read_cost, repair, scope_gap, scope_list,
visibility_of,
)
PRIVATE_NOW = {"private": True, "visibility": "private", "forks_count": 37}
INTERNAL = {"private": True, "visibility": "internal", "forks_count": 0}
PUBLIC = {"private": False, "visibility": "public", "forks_count": 12}
def test_the_pair_of_readings_is_the_finding():
state, detail = classify(404, 200, PRIVATE_NOW)
assert state == "went-private"
assert "invisible without one" in detail
assert "Deletion would answer 404 to both" in detail
def test_404_to_both_readings_is_handed_to_the_wider_triage():
state, detail = classify(404, 404, None)
assert state == "invisible-to-both"
assert "wider 404 triage" in detail
assert "wider 404 triage" in repair(state)
def test_a_public_repository_is_not_reported_as_a_transition():
assert classify(200, 200, PUBLIC)[0] == "still-public"
def test_an_anonymous_success_beside_an_authenticated_failure_blames_the_token():
state, detail = classify(200, 401, None)
assert state == "token-is-the-problem"
assert "expired or revoked" in detail
def test_a_redirect_anywhere_is_a_rename_and_a_different_note():
assert classify(301, 200, PRIVATE_NOW)[0] == "moved"
assert classify(404, 301, None)[0] == "moved"
def test_internal_is_private_true_and_still_not_private():
assert visibility_of(INTERNAL) == "internal"
assert INTERNAL["private"] is True
state, detail = classify(404, 200, INTERNAL)
assert state == "internal-visibility"
assert "every member of the enterprise" in detail
assert "membership" in repair(state)
def test_visibility_falls_back_to_the_boolean_but_prefers_the_field():
assert visibility_of({"private": True}) == "private"
assert visibility_of({"private": False}) == "public"
assert visibility_of({}) == "unreported"
assert visibility_of(None) == "unreported"
def test_the_anonymous_bucket_proves_whether_a_client_authenticated():
assert ANONYMOUS_CORE_LIMIT == 60
assert client_is_anonymous(60) is True
assert client_is_anonymous(5000) is False
assert client_is_anonymous(None) is None
def test_public_repo_is_blind_rather_than_merely_narrow():
state, detail = scope_gap([BLIND_SCOPE], "private")
assert state == "blind-scope"
assert "as blind here as sending no token at all" in detail
assert BLIND_SCOPE in repair("went-private", state)
def test_the_repo_scope_covers_it_and_points_at_the_account_instead():
state, detail = scope_gap([PRIVATE_SCOPE, "workflow"], "private")
assert state == "scope-sufficient"
assert "no grant on the repository" in detail
def test_a_fine_grained_token_reports_no_scopes_and_needs_permissions():
state, detail = scope_gap(None, "private")
assert state == "no-scopes-reported"
assert "Metadata: Read" in detail and "Contents: Read" in detail
def test_scopes_are_not_asked_about_a_public_repository():
assert scope_gap(["public_repo"], "public")[0] == "not-applicable"
assert scope_gap([], "private")[0] == "scope-insufficient"
def test_absent_and_empty_scope_headers_are_different_readings():
assert scope_list(None) is None
assert scope_list("") == []
assert scope_list("repo, workflow") == ["repo", "workflow"]
def test_the_detached_forks_are_reported_as_a_second_failure():
note = fork_fallout(PRIVATE_NOW)
assert note and "still public" in note
assert fork_fallout(PUBLIC) is None
assert fork_fallout(INTERNAL) is None
def test_the_missing_timestamp_is_stated_rather_than_guessed():
assert "no visibility-changed timestamp" in blind_spot()
assert "your own logs" in blind_spot()
def test_an_unsorted_pair_is_left_unsorted():
state, detail = classify(403, 500, None)
assert state == "unclassified"
assert "500" in detail and "403" in detail
def test_the_run_costs_two_billable_reads():
assert read_cost() == 2
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
ANONYMOUS_CORE_LIMIT, BLIND_SCOPE, PRIVATE_SCOPE, blindSpot, classify,
clientIsAnonymous, forkFallout, readCost, repair, scopeGap, scopeList,
visibilityOf,
} from './github-visibility-change.mjs';
const PRIVATE_NOW = { private: true, visibility: 'private', forks_count: 37 };
const INTERNAL = { private: true, visibility: 'internal', forks_count: 0 };
const PUBLIC = { private: false, visibility: 'public', forks_count: 12 };
test('the pair of readings is the finding', () => {
const [state, detail] = classify(404, 200, PRIVATE_NOW);
assert.equal(state, 'went-private');
assert.match(detail, /invisible without one/);
assert.match(detail, /Deletion would answer 404 to both/);
});
test('404 to both readings is handed to the wider triage', () => {
const [state, detail] = classify(404, 404, null);
assert.equal(state, 'invisible-to-both');
assert.match(detail, /wider 404 triage/);
assert.match(repair(state), /wider 404 triage/);
});
test('a public repository is not reported as a transition', () => {
assert.equal(classify(200, 200, PUBLIC)[0], 'still-public');
});
test('an anonymous success beside an authenticated failure blames the token', () => {
const [state, detail] = classify(200, 401, null);
assert.equal(state, 'token-is-the-problem');
assert.match(detail, /expired or revoked/);
});
test('a redirect anywhere is a rename and a different note', () => {
assert.equal(classify(301, 200, PRIVATE_NOW)[0], 'moved');
assert.equal(classify(404, 301, null)[0], 'moved');
});
test('internal is private true and still not private', () => {
assert.equal(visibilityOf(INTERNAL), 'internal');
assert.equal(INTERNAL.private, true);
const [state, detail] = classify(404, 200, INTERNAL);
assert.equal(state, 'internal-visibility');
assert.match(detail, /every member of the enterprise/);
assert.match(repair(state), /membership/);
});
test('visibility falls back to the boolean but prefers the field', () => {
assert.equal(visibilityOf({ private: true }), 'private');
assert.equal(visibilityOf({ private: false }), 'public');
assert.equal(visibilityOf({}), 'unreported');
assert.equal(visibilityOf(null), 'unreported');
});
test('the anonymous bucket proves whether a client authenticated', () => {
assert.equal(ANONYMOUS_CORE_LIMIT, 60);
assert.equal(clientIsAnonymous(60), true);
assert.equal(clientIsAnonymous(5000), false);
assert.equal(clientIsAnonymous(null), null);
});
test('public_repo is blind rather than merely narrow', () => {
const [state, detail] = scopeGap([BLIND_SCOPE], 'private');
assert.equal(state, 'blind-scope');
assert.match(detail, /as blind here as sending no token at all/);
assert.match(repair('went-private', state), new RegExp(BLIND_SCOPE));
});
test('the repo scope covers it and points at the account instead', () => {
const [state, detail] = scopeGap([PRIVATE_SCOPE, 'workflow'], 'private');
assert.equal(state, 'scope-sufficient');
assert.match(detail, /no grant on the repository/);
});
test('a fine grained token reports no scopes and needs permissions', () => {
const [state, detail] = scopeGap(null, 'private');
assert.equal(state, 'no-scopes-reported');
assert.match(detail, /Metadata: Read/);
assert.match(detail, /Contents: Read/);
});
test('scopes are not asked about a public repository', () => {
assert.equal(scopeGap(['public_repo'], 'public')[0], 'not-applicable');
assert.equal(scopeGap([], 'private')[0], 'scope-insufficient');
});
test('absent and empty scope headers are different readings', () => {
assert.equal(scopeList(null), null);
assert.deepEqual(scopeList(''), []);
assert.deepEqual(scopeList('repo, workflow'), ['repo', 'workflow']);
});
test('the detached forks are reported as a second failure', () => {
const note = forkFallout(PRIVATE_NOW);
assert.match(note, /still public/);
assert.equal(forkFallout(PUBLIC), null);
assert.equal(forkFallout(INTERNAL), null);
});
test('the missing timestamp is stated rather than guessed', () => {
assert.match(blindSpot(), /no visibility-changed timestamp/);
assert.match(blindSpot(), /your own logs/);
});
test('an unsorted pair is left unsorted', () => {
const [state, detail] = classify(403, 500, null);
assert.equal(state, 'unclassified');
assert.match(detail, /500/);
assert.match(detail, /403/);
});
test('the run costs two billable reads', () => {
assert.equal(readCost(), 2);
});
FAQ
How is this different from the general 404 triage already in this section?
That note answers a wider question for a caller who may never have had access: it sorts a 404 into a dead token, a missing scope, a repository outside an App's installation, or no grant at all. This one is the narrower case where a caller demonstrably <em>did</em> have access, anonymously, for a long time, and the resource changed underneath it. The evidence is different too. The triage takes three reads with one credential; this takes the same read twice with two different callers, and it is the asymmetry between them that names the cause. When both readings 404, this note stops and hands you to that one.
Can the script tell me when the repository went private?
No, and it says so rather than estimating. There is no visibility-changed timestamp on the repository object. <code>updated_at</code> moves whenever a description, a topic or a default branch changes, so reading a date out of it would be a guess dressed as a finding. The organization audit log does record the event, and reading it needs <code>admin:org</code>-class access that a repository reader does not have. What you can date precisely is the first 404 in your own logs, which is usually close enough and is evidence you already own.
The repository shows private: true but colleagues outside the team can read it. Why?
Because it is internal rather than private, and the boolean cannot express the difference. <code>visibility</code> has three values — <code>public</code>, <code>private</code> and <code>internal</code> — and internal repositories report <code>private: true</code> while being readable by every member of the enterprise. It matters for the repair: private access is granted per repository, internal access follows enterprise membership, so a machine account that is not a member will keep getting 404 no matter how many repository-level grants it is given.
Our token has public_repo. Is that not enough for a repository we already read?
It is the one scope guaranteed not to be enough. <code>public_repo</code> grants access to public repositories and to nothing else, so at the exact moment this repository stopped being public, that token became precisely as blind as sending no credential at all. A classic token needs <code>repo</code>, which is read and write together and is more authority than a reader should hold; a fine-grained token with <code>Metadata: Read</code> and <code>Contents: Read</code> on that one repository is the narrower and better answer, and it has to be granted by the repository's owner.
A URL that looks like the repository still works. Which one am I reading?
Probably a fork. When a public repository becomes private, the forks that already existed are split off into their own network and remain public, so links, search results and vendored references keep resolving to something with the same name and the same history up to the moment of the change. A dependency tool that quietly follows one starts tracking a copy that no longer receives commits, which surfaces weeks later as a version that never updates. Check <code>fork</code> and <code>source</code> on whatever is still answering before you rely on it.
Related field notes
- The wider triage for a 404 with several possible causes
- What an unauthenticated client is actually working with
- The copy that still resolves after the change
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Repositories — GitHub REST API
- Setting repository visibility — GitHub Docs
- About repositories — GitHub Docs
- Rate limits for the REST API — GitHub Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.