Diagnostic GitHub API
An outside collaborator has repos in an org, not the org
The integration reads three repositories in the customer's organization perfectly well, and has done for months. Then somebody adds a feature that needs the team list, and GET /orgs/{org}/teams returns 404. Which is odd, because the repositories are right there and they belong to that organization. So the token gets read:org added, and it is still 404. Then admin:org, briefly, against everybody's better judgement, and it is still 404. The account was never an organization member. It is an outside collaborator, which means it has repositories inside the organization and no standing in the organization, and no scope on earth grants standing.
An outside collaborator is granted specific repositories in an organization without being a member of it. Organization-level endpoints require membership, so teams, members and the rest answer 404 for an account whose repository reads work fine. Scopes bound what a token may do on the account's behalf; they cannot give the account a relationship it does not have.
The partition that proves it costs two reads. GET /user/repos?affiliation=organization_member returns the repositories reachable because the account is in an organization; GET /user/repos?affiliation=collaborator returns the ones reachable because somebody added the account to them one at a time. An account with repositories in the organization under collaborator and none under organization_member, and the organization missing from GET /user/orgs, is an outside collaborator, definitively, from the token's own side.
The trap on the way is GET /orgs/{org}/repos, which does not refuse a non-member. It answers 200 with the public repositories and nothing else, and no header says a word about it.
The problem in plain words
Every symptom points the wrong way. The 404s are on organization endpoints, the token has organization scopes, and 404 is the status GitHub uses when it will not tell you a private thing exists. That combination reads unmistakably as "the scope is not taking effect", which is a real problem with real causes — SSO authorization, App installation, fine-grained permissions — and none of them is this. Widening the token is the first thing anybody tries and it produces no change at all, which is genuinely disorienting because widening a token normally changes something.
The second trap is the endpoint that does not refuse. GET /orgs/{org}/repos answers 200 for a non-member and returns the organization's public repositories. If your integration lists repositories that way, it does not error; it under-reports, silently, and looks like it is working. A customer with forty private repositories and two public ones shows up in your product as a customer with two repositories, and the first person to notice is the customer.
The third is that the relationship is invisible from the account's side unless you go looking for it. Nothing on GET /user says "outside collaborator". There is an endpoint that lists them, GET /orgs/{org}/outside_collaborators, and it needs organization read access — which the outside collaborator, by definition, does not have. So the one call that names the condition outright is the one call this account cannot make.
Why it happens
Repository access and organization membership are different grants. Being added to a repository owned by an organization does not put you in the organization; it puts you on the repository. Organization endpoints are gated on membership, and no permission model layered on top — classic scopes, fine-grained permissions, an App installation — changes the underlying fact. This is why the scope ladder produces no movement: every rung is bounded by an access the account never had.
The affiliation parameter names the difference out loud. GET /user/repos takes an affiliation of owner, collaborator or organization_member, and those three are how the account reached each repository. Requesting them separately turns a vague "I can see some of this organization" into a partition, and the partition is the diagnosis: repositories present under collaborator and absent under organization_member is what an outside collaborator looks like and nothing else looks like it.
This is a shorter list with no announcement, which is what separates it from the SAML case. The section already publishes the note about organization data withheld from a successful response — and the point of that one is that GitHub tells you, in an X-GitHub-SSO: partial-results header naming the organizations it left out. Nothing announces this. A non-member listing an organization's repositories gets a clean 200, no header, and a shorter array. The script reads that header anyway, precisely so it can say "this is not that note" with evidence rather than by assertion.
The membership redirect cannot separate the two ways of not being a member. GET /orgs/{org}/members/{username} answers 302 when the requester is not an organization member, which is a genuinely useful signal and belongs to the note about accounts removed by a two-factor requirement. It is not useful here, because an outside collaborator and a former member are both non-members and both get the redirect. What tells them apart is what happens to the repositories: an outside collaborator still reaches theirs, and a removed member reaches nothing.
The credential class can make this worse in a way that is diagnostic. GitHub documents, among the things fine-grained personal access tokens cannot yet do, contributing to repositories where the user is an outside or repository collaborator. So an outside collaborator can find a classic token working where a fine-grained one does not, on the same repository, with the same person behind it. If swapping the token class changes the answer, that inversion is itself evidence of the role.
The fix, as a flow
The partition is the proof and it costs two reads. Repositories reached because the account is in an organization arrive under one affiliation; repositories reached because somebody added the account to them one at a time arrive under another. Present in the second and absent from the first, with the organization missing from the membership list, is an outside collaborator and nothing else is. The SSO header is read on the same response for a reason: this note is next door to the one about data GitHub tells you it withheld, and the difference is that nothing here is announced. Where the header does say partial, the script stops and hands the case over rather than drawing a membership conclusion from a list that admits it is short.
How to fix it
Ask the token who it is and which organizations it is in
GET /user for the login, then GET /user/orgs. If the organization is absent from that list, the account is not a member of it. Read x-github-sso on the same response before believing the absence: a partial result there means the list is incomplete for a completely different reason and the section has a note about that. The script reports which of the two it is looking at.
Partition the repositories by how they were reached
One page of GET /user/repos?affiliation=collaborator and one of affiliation=organization_member, counting the ones owned by the organization in question. Repositories present in the first and absent from the second is the finding. The script reports a count as a floor rather than a total when the Link header shows another page, because an under-count that says it is an under-count is useful and one that does not is a lie.
Take one reading against an organization endpoint that refuses
--org-probe reads GET /orgs/{org}/members?per_page=1 and records the status. A 404 there beside a 200 on a repository in the same organization is the pair to put in the ticket. It is a reading rather than a conclusion: the script pairs it with the affiliation partition instead of concluding from it alone.
Notice the endpoint that does not refuse
The script explicitly reports what GET /orgs/{org}/repos will do for this role, which is answer 200 with the public repositories only. Any inventory built on that call is under-reporting rather than failing, and it will keep looking healthy. This is the part worth taking back to the code, because it is the bug that has no error message attached to it.
Decide which of the two repairs you actually want
Either the account becomes an organization member with a role, which somebody with organization admin has to do and which is a change in the customer's security posture, or the integration stops making organization-level calls and works at repository scope where its access genuinely is. The script prints both and adds nothing to anything; an invitation is a decision, not a diagnostic step.
How to check it worked
Once the account is a member, the same run finds the organization in GET /user/orgs and the repositories move from collaborator to organization_member.
GITHUB_TOKEN=$READ_ONLY_TOKEN python3 github_outside_collaborator.py acme --org-probe
# read cost: 4 REST request(s) against the core hourly quota
# identity: dana-integration
# membership: acme is not in GET /user/orgs, and no partial-results header
# accompanied that list, so the absence is a real absence and not the SAML
# note.
# affiliation partition: 3 repo(s) in acme reached as collaborator,
# 0 reached as organization_member
# org probe: GET /orgs/acme/members?per_page=1 -> HTTP 404
# outside-collaborator: repositories inside the organization, no standing in
# the organization. No scope grants standing, which is why widening the
# token changed nothing.
# quiet-failure-ahead: GET /orgs/acme/repos will answer 200 for this account
# and return public repositories only. An inventory built on it under-reports
# instead of failing.
# repair: either ask an owner of acme to add dana-integration as a member with
# a role, or drop the organization-level calls and work at repository scope.
# Nothing here invites anybody.The full code
Four cheap GETs and one comparison. The comparison is the whole note, so it is pure and it is tested against the case that matters most: an account with repositories in the organization and no membership, which is the only shape that produces this. The Link parsing is pure too, because a count that silently truncates is worse than no count and the boundary between "three" and "at least three" is one header away.
"""Tell an outside collaborator from a member with a narrow token.
Read only. Four cheap GETs. Nothing is invited, added or promoted: making an
account an organization member is a decision about who is inside a company's
organization, and this script prints the request rather than making it.
The point of the note: an outside collaborator holds specific repositories
inside an organization without being in the organization. Repository reads
work, organization reads do not, and no scope closes the gap because a scope
bounds what a token may do on the account's behalf rather than granting the
account a relationship.
What this can and cannot see: GET /orgs/{org}/outside_collaborators names the
condition outright and needs organization read access, which is exactly what
this account lacks. So the diagnosis is made from the token's own side, by
partitioning its repositories by the affiliation that reached them.
Environment:
GITHUB_TOKEN the token the failing integration holds
"""
import argparse
import json
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("github_outside_collaborator")
API = "https://api.github.com"
UA = "github-outside-collaborator/1.0"
# The three ways GET /user/repos says an account reached a repository.
AFFILIATIONS = ("owner", "collaborator", "organization_member")
def read_cost(with_org_probe):
"""REST requests this run will spend. Pure. Printed before any are spent."""
return 3 + (1 if with_org_probe else 0)
def header_value(headers, name):
"""Case-insensitive header read against a plain dict. Pure."""
if not isinstance(headers, dict):
return None
wanted = str(name).lower()
for key, value in headers.items():
if str(key).lower() == wanted:
return value
return None
def has_next_page(link_header):
"""Is there another page after this one. Pure. No regular expression.
Used to turn a count into a floor. A number that admits it is a lower bound
is useful; one that quietly truncates is the same bug this note is about,
committed by the diagnostic instead of the integration.
"""
for part in str(link_header or "").split(","):
if 'rel="next"' in part.replace("'", '"').replace(" ", ""):
return True
if 'rel="next"' in part:
return True
return False
def sso_reading(headers):
"""Is this response's shortness announced. Pure. (state, detail).
Read so the script can say "this is not the SAML note" with evidence. That
note owns organization data withheld from a 200 and announced in a header;
the case here announces nothing at all.
"""
value = header_value(headers, "x-github-sso")
if not value:
return ("no-sso-header",
"no partial-results header accompanied this list, so nothing "
"was announced as withheld. The SAML note is about the case "
"where GitHub does tell you.")
if "partial-results" in str(value):
return ("sso-partial-results",
"this list is explicitly incomplete: GitHub withheld "
"organizations this token is not SSO-authorized for and said "
"so in the header. That is a different note, and any "
"membership conclusion from this list is unsafe.")
return ("sso-header-present",
"an SSO header came back without a partial-results marker. Nothing "
"is stated as withheld, but treat the list with care.")
def is_member(orgs, org):
"""Does GET /user/orgs list this organization. Pure."""
wanted = str(org or "").lower()
for entry in orgs or []:
if isinstance(entry, dict) and str(entry.get("login") or "").lower() == wanted:
return True
return False
def repos_in_org(repos, org):
"""Full names of the repositories in this organization. Pure."""
wanted = str(org or "").lower()
out = []
for repo in repos or []:
if not isinstance(repo, dict):
continue
owner = str(((repo.get("owner") or {}).get("login")) or "").lower()
if owner == wanted:
out.append(repo.get("full_name"))
return out
def counted(names, more_pages):
"""A count, honest about being a floor. Pure. (count, exact, phrase)."""
total = len(names or [])
if more_pages:
return (total, False, "at least %d" % total)
return (total, True, str(total))
def role_verdict(member, collaborator_count, member_affiliated_count):
"""Which relationship the account has. Pure. (state, detail).
Four states, and three of them send you somewhere else. The one this note
owns is the account with repositories in the organization and no membership.
"""
if member and member_affiliated_count > 0:
return ("organization-member",
"the organization is in this account's membership list and its "
"repositories arrive under organization_member. Whatever is "
"failing, it is not this.")
if member and member_affiliated_count == 0:
return ("member-with-no-implicit-repos",
"the account is a member and reaches no repository through "
"that membership. That is what a base permission of none looks "
"like organization-wide, and it has its own note.")
if not member and collaborator_count > 0:
return ("outside-collaborator",
"repositories inside the organization, no standing in the "
"organization. No scope grants standing, which is why widening "
"the token changes nothing.")
return ("no-relationship",
"not a member and no repositories in this organization reachable "
"as a collaborator. An account that used to have access and now "
"has none is a removal rather than a role, and that has its own "
"note.")
def org_endpoint_expectation(role):
"""What organization endpoints will do for this role. Pure. dict.
The second entry is the one worth carrying back to the code: it does not
fail, it under-reports.
"""
if role == "organization-member":
return {"members-and-teams": "answer for a member",
"org-repos-listing": "returns the repositories a member may see",
"outside-collaborators-listing": "needs organization read access"}
return {
"members-and-teams": "refuse a non-member, and 404 rather than 403 so "
"nothing is confirmed to exist",
"org-repos-listing": "answers 200 and returns the public repositories "
"only. This does not fail; it under-reports, with "
"no header and no error.",
"outside-collaborators-listing": "names this condition outright and "
"needs organization read access, "
"which this account does not have",
}
def token_class_caveat(token):
"""A documented gap that can invert the diagnosis. Pure. (state, detail)."""
value = (token or "").strip()
if value.startswith("github_pat_"):
return ("fine-grained-gap",
"GitHub documents, among the things fine-grained tokens cannot "
"yet do, contributing to repositories where the user is an "
"outside or repository collaborator. If a classic token works "
"where this one does not, that inversion is evidence of the "
"role rather than a bug in your code.")
if value.startswith("ghp_"):
return ("classic-token",
"a classic token is not subject to the documented fine-grained "
"gap for outside collaborators, so a difference between the "
"two classes is worth testing before blaming anything else.")
return ("class-not-recognised",
"the credential class could not be named from its prefix, so the "
"fine-grained caveat cannot be applied either way.")
def org_probe_reading(repo_status, org_status):
"""One repository read against one organization read. Pure. (state, detail)."""
repo = None if repo_status is None else int(repo_status)
org = None if org_status is None else int(org_status)
if org is None:
return ("org-not-probed",
"no organization endpoint was probed, so the partition is the "
"only evidence here.")
if repo is not None and repo == 200 and org == 404:
return ("repo-yes-org-no",
"a repository in the organization answers and an organization "
"endpoint does not. That pair is the sentence to put in the "
"ticket.")
if org in (200, 204):
return ("org-reachable",
"the organization endpoint answered, so membership is not what "
"is missing.")
if org in (401, 403):
return ("org-refused-not-hidden",
"a refusal rather than a 404 points at a credential or a "
"policy rather than at membership. Sort that first.")
return ("org-probe-inconclusive",
"the pair of statuses does not describe a membership problem.")
def verdict(role, sso_state):
"""The finding, in one state. Pure. (state, detail)."""
if sso_state == "sso-partial-results":
return ("membership-list-incomplete",
"the organization list this conclusion would rest on is "
"explicitly partial, so no membership answer from it can be "
"trusted. Authorize the token for SSO and re-run.")
return (role,
"this is the relationship the readings describe." if role
else "no relationship could be determined.")
def repair(state, org, login):
"""The sentence a reader has to act on. Pure. Nothing here invites anybody."""
if state == "outside-collaborator":
return ("either ask an owner of %s to add %s as a member with an "
"appropriate role, which is a change to who is inside that "
"organization, or drop the organization-level calls and work "
"at repository scope where this account's access actually is. "
"Nothing here invites anybody." % (org, login))
if state == "member-with-no-implicit-repos":
return ("read the organization's default repository permission before "
"anything else; an organization-wide default of none produces "
"exactly this and has its own note.")
if state == "no-relationship":
return ("find out whether this account was removed from the "
"organization rather than never added. A removal leaves a "
"healthy token with no access at all.")
if state == "membership-list-incomplete":
return ("authorize this token for the organization's SSO and re-run. "
"Until then the membership list is not evidence.")
return "nothing to repair from this reading."
def get(session, path):
"""One GET. Returns the response object."""
return session.get(API + path, timeout=30)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("org", help="the organization whose data is invisible")
parser.add_argument("--org-probe", action="store_true",
help="also read one organization-level endpoint")
parser.add_argument("--repo",
help="a repository in that org to pair the probe with, "
"as owner/name")
args = parser.parse_args()
token = os.environ.get("GITHUB_TOKEN")
if not token:
log.error("set GITHUB_TOKEN (the token the failing integration holds)")
return 2
log.info("read cost: %d REST request(s) against the core hourly quota",
read_cost(args.org_probe))
session = requests.Session()
session.headers.update({
"Authorization": "Bearer " + token,
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
# GitHub refuses requests with no User-Agent before it looks at auth.
"User-Agent": UA,
})
class_state, class_detail = token_class_caveat(token)
log.info("%s: %s", class_state, class_detail)
me = get(session, "/user")
login = (me.json() or {}).get("login") if me.status_code == 200 else None
log.info("identity: %s", login or "unreadable")
orgs_response = get(session, "/user/orgs?per_page=100")
orgs = orgs_response.json() if orgs_response.status_code == 200 else []
sso_state, sso_detail = sso_reading(dict(orgs_response.headers))
member = is_member(orgs, args.org)
log.info("membership: %s is %sin GET /user/orgs. %s", args.org,
"" if member else "not ", sso_detail)
partition = {}
for affiliation in ("collaborator", "organization_member"):
response = get(session, "/user/repos?affiliation=%s&per_page=100"
% affiliation)
names = repos_in_org(response.json() if response.status_code == 200 else [],
args.org)
more = has_next_page(header_value(dict(response.headers), "link"))
total, exact, phrase = counted(names, more)
partition[affiliation] = {"count": total, "exact": exact,
"phrase": phrase, "names": names[:20]}
log.info("affiliation partition: %s repo(s) in %s reached as collaborator, "
"%s reached as organization_member",
partition["collaborator"]["phrase"], args.org,
partition["organization_member"]["phrase"])
org_status, repo_status = None, None
if args.org_probe:
org_status = get(session, "/orgs/%s/members?per_page=1" % args.org).status_code
log.info("org probe: GET /orgs/%s/members?per_page=1 -> HTTP %s",
args.org, org_status)
if args.repo:
repo_status = get(session, "/repos/%s" % args.repo).status_code
log.info("repo probe: GET /repos/%s -> HTTP %s", args.repo, repo_status)
probe_state, probe_detail = org_probe_reading(repo_status, org_status)
log.info("%s: %s", probe_state, probe_detail)
role, role_detail = role_verdict(member,
partition["collaborator"]["count"],
partition["organization_member"]["count"])
log.info("%s: %s", role, role_detail)
expectation = org_endpoint_expectation(role)
log.info("quiet-failure-ahead: %s", expectation["org-repos-listing"])
state, detail = verdict(role, sso_state)
fix = repair(state, args.org, login or "this account")
log.info("repair: %s", fix)
print(json.dumps({
"organization": args.org,
"login": login,
"is_member": member,
"sso_state": sso_state,
"affiliation_partition": partition,
"org_probe_status": org_status,
"repo_probe_status": repo_status,
"probe_state": probe_state,
"token_class_state": class_state,
"org_endpoint_expectation": expectation,
"state": state,
"detail": detail,
"repair": fix,
}, indent=2, default=str))
return 1 if state in ("outside-collaborator", "member-with-no-implicit-repos",
"no-relationship", "membership-list-incomplete") else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Tell an outside collaborator from a member with a narrow token.
*
* Read only. Four cheap GETs. Nothing is invited, added or promoted: making an
* account an organization member is a decision about who is inside a company's
* organization, and this script prints the request rather than making it.
*
* An outside collaborator holds specific repositories inside an organization
* without being in the organization. No scope closes that gap, because a scope
* bounds what a token may do on the account's behalf rather than granting the
* account a relationship.
*
* Environment:
* GITHUB_TOKEN the token the failing integration holds
* GITHUB_ORG the organization whose data is invisible
* GITHUB_ORG_PROBE set to 1 to read one organization-level endpoint
* GITHUB_REPO optional owner/name in that org to pair the probe with
*/
const API = 'https://api.github.com';
const UA = 'github-outside-collaborator/1.0';
/** The three ways GET /user/repos says an account reached a repository. */
export const AFFILIATIONS = ['owner', 'collaborator', 'organization_member'];
/** REST requests this run will spend. Pure. */
export function readCost(withOrgProbe) {
return 3 + (withOrgProbe ? 1 : 0);
}
/** Case-insensitive header read against a plain object. Pure. */
export function headerValue(headers, name) {
if (!headers || typeof headers !== 'object') return null;
const wanted = String(name).toLowerCase();
for (const key of Object.keys(headers)) {
if (key.toLowerCase() === wanted) return headers[key];
}
return null;
}
/** Is there another page after this one. Pure. Turns a count into a floor. */
export function hasNextPage(linkHeader) {
for (const part of String(linkHeader ?? '').split(',')) {
if (part.includes('rel="next"')) return true;
if (part.split(' ').join('').includes('rel="next"')) return true;
}
return false;
}
/** Is this response's shortness announced. Pure. [state, detail]. */
export function ssoReading(headers) {
const value = headerValue(headers, 'x-github-sso');
if (!value) {
return ['no-sso-header', 'no partial-results header accompanied this list, '
+ 'so nothing was announced as withheld. The SAML note is about the case '
+ 'where GitHub does tell you.'];
}
if (String(value).includes('partial-results')) {
return ['sso-partial-results', 'this list is explicitly incomplete: GitHub '
+ 'withheld organizations this token is not SSO-authorized for and said '
+ 'so in the header. That is a different note, and any membership '
+ 'conclusion from this list is unsafe.'];
}
return ['sso-header-present', 'an SSO header came back without a '
+ 'partial-results marker. Nothing is stated as withheld, but treat the '
+ 'list with care.'];
}
/** Does GET /user/orgs list this organization. Pure. */
export function isMember(orgs, org) {
const wanted = String(org ?? '').toLowerCase();
for (const entry of orgs || []) {
if (entry && String(entry.login ?? '').toLowerCase() === wanted) return true;
}
return false;
}
/** Full names of the repositories in this organization. Pure. */
export function reposInOrg(repos, org) {
const wanted = String(org ?? '').toLowerCase();
const out = [];
for (const repo of repos || []) {
if (!repo || typeof repo !== 'object') continue;
const owner = String((repo.owner && repo.owner.login) || '').toLowerCase();
if (owner === wanted) out.push(repo.full_name);
}
return out;
}
/** A count, honest about being a floor. Pure. [count, exact, phrase]. */
export function counted(names, morePages) {
const total = (names || []).length;
if (morePages) return [total, false, `at least ${total}`];
return [total, true, String(total)];
}
/** Which relationship the account has. Pure. [state, detail]. */
export function roleVerdict(member, collaboratorCount, memberAffiliatedCount) {
if (member && memberAffiliatedCount > 0) {
return ['organization-member', "the organization is in this account's "
+ 'membership list and its repositories arrive under organization_member. '
+ 'Whatever is failing, it is not this.'];
}
if (member && memberAffiliatedCount === 0) {
return ['member-with-no-implicit-repos', 'the account is a member and '
+ 'reaches no repository through that membership. That is what a base '
+ 'permission of none looks like organization-wide, and it has its own note.'];
}
if (!member && collaboratorCount > 0) {
return ['outside-collaborator', 'repositories inside the organization, no '
+ 'standing in the organization. No scope grants standing, which is why '
+ 'widening the token changes nothing.'];
}
return ['no-relationship', 'not a member and no repositories in this '
+ 'organization reachable as a collaborator. An account that used to have '
+ 'access and now has none is a removal rather than a role, and that has '
+ 'its own note.'];
}
/** What organization endpoints will do for this role. Pure. */
export function orgEndpointExpectation(role) {
if (role === 'organization-member') {
return {
'members-and-teams': 'answer for a member',
'org-repos-listing': 'returns the repositories a member may see',
'outside-collaborators-listing': 'needs organization read access',
};
}
return {
'members-and-teams': 'refuse a non-member, and 404 rather than 403 so '
+ 'nothing is confirmed to exist',
'org-repos-listing': 'answers 200 and returns the public repositories only. '
+ 'This does not fail; it under-reports, with no header and no error.',
'outside-collaborators-listing': 'names this condition outright and needs '
+ 'organization read access, which this account does not have',
};
}
/** A documented gap that can invert the diagnosis. Pure. [state, detail]. */
export function tokenClassCaveat(token) {
const value = String(token ?? '').trim();
if (value.startsWith('github_pat_')) {
return ['fine-grained-gap', 'GitHub documents, among the things '
+ 'fine-grained tokens cannot yet do, contributing to repositories where '
+ 'the user is an outside or repository collaborator. If a classic token '
+ 'works where this one does not, that inversion is evidence of the role '
+ 'rather than a bug in your code.'];
}
if (value.startsWith('ghp_')) {
return ['classic-token', 'a classic token is not subject to the documented '
+ 'fine-grained gap for outside collaborators, so a difference between '
+ 'the two classes is worth testing before blaming anything else.'];
}
return ['class-not-recognised', 'the credential class could not be named from '
+ 'its prefix, so the fine-grained caveat cannot be applied either way.'];
}
/** One repository read against one organization read. Pure. [state, detail]. */
export function orgProbeReading(repoStatus, orgStatus) {
const repo = (repoStatus === null || repoStatus === undefined) ? null : Number(repoStatus);
const org = (orgStatus === null || orgStatus === undefined) ? null : Number(orgStatus);
if (org === null) {
return ['org-not-probed', 'no organization endpoint was probed, so the '
+ 'partition is the only evidence here.'];
}
if (repo === 200 && org === 404) {
return ['repo-yes-org-no', 'a repository in the organization answers and an '
+ 'organization endpoint does not. That pair is the sentence to put in '
+ 'the ticket.'];
}
if ([200, 204].includes(org)) {
return ['org-reachable', 'the organization endpoint answered, so membership '
+ 'is not what is missing.'];
}
if ([401, 403].includes(org)) {
return ['org-refused-not-hidden', 'a refusal rather than a 404 points at a '
+ 'credential or a policy rather than at membership. Sort that first.'];
}
return ['org-probe-inconclusive', 'the pair of statuses does not describe a '
+ 'membership problem.'];
}
/** The finding, in one state. Pure. [state, detail]. */
export function verdict(role, ssoState) {
if (ssoState === 'sso-partial-results') {
return ['membership-list-incomplete', 'the organization list this conclusion '
+ 'would rest on is explicitly partial, so no membership answer from it '
+ 'can be trusted. Authorize the token for SSO and re-run.'];
}
return [role, role ? 'this is the relationship the readings describe.'
: 'no relationship could be determined.'];
}
/** The sentence a reader has to act on. Pure. Nothing here invites anybody. */
export function repair(state, org, login) {
if (state === 'outside-collaborator') {
return `either ask an owner of ${org} to add ${login} as a member with an `
+ 'appropriate role, which is a change to who is inside that '
+ 'organization, or drop the organization-level calls and work at '
+ "repository scope where this account's access actually is. Nothing here "
+ 'invites anybody.';
}
if (state === 'member-with-no-implicit-repos') {
return "read the organization's default repository permission before "
+ 'anything else; an organization-wide default of none produces exactly '
+ 'this and has its own note.';
}
if (state === 'no-relationship') {
return 'find out whether this account was removed from the organization '
+ 'rather than never added. A removal leaves a healthy token with no '
+ 'access at all.';
}
if (state === 'membership-list-incomplete') {
return "authorize this token for the organization's SSO and re-run. Until "
+ 'then the membership list is not evidence.';
}
return 'nothing to repair from this reading.';
}
function headers(token) {
return {
Authorization: `Bearer ${token}`,
Accept: 'application/vnd.github+json',
'X-GitHub-Api-Version': '2022-11-28',
'User-Agent': UA,
};
}
function bagOf(response) {
const bag = {};
response.headers.forEach((value, key) => { bag[key] = value; });
return bag;
}
async function main() {
const token = process.env.GITHUB_TOKEN;
const org = process.env.GITHUB_ORG;
if (!token || !org) {
console.error('set GITHUB_TOKEN (read-only is enough) and GITHUB_ORG');
process.exitCode = 2;
return;
}
const withOrgProbe = process.env.GITHUB_ORG_PROBE === '1';
const repoName = process.env.GITHUB_REPO || '';
console.log(`read cost: ${readCost(withOrgProbe)} REST request(s) against the `
+ 'core hourly quota');
const [classState, classDetail] = tokenClassCaveat(token);
console.log(`${classState}: ${classDetail}`);
const me = await fetch(`${API}/user`, { headers: headers(token) });
const login = me.status === 200 ? (await me.json()).login : null;
console.log(`identity: ${login || 'unreadable'}`);
const orgsResponse = await fetch(`${API}/user/orgs?per_page=100`,
{ headers: headers(token) });
const orgs = orgsResponse.status === 200 ? await orgsResponse.json() : [];
const [ssoState, ssoDetail] = ssoReading(bagOf(orgsResponse));
const member = isMember(orgs, org);
console.log(`membership: ${org} is ${member ? '' : 'not '}in GET /user/orgs. ${ssoDetail}`);
const partition = {};
for (const affiliation of ['collaborator', 'organization_member']) {
const response = await fetch(
`${API}/user/repos?affiliation=${affiliation}&per_page=100`,
{ headers: headers(token) },
);
const body = response.status === 200 ? await response.json() : [];
const names = reposInOrg(body, org);
const more = hasNextPage(headerValue(bagOf(response), 'link'));
const [count, exact, phrase] = counted(names, more);
partition[affiliation] = { count, exact, phrase, names: names.slice(0, 20) };
}
console.log(`affiliation partition: ${partition.collaborator.phrase} repo(s) `
+ `in ${org} reached as collaborator, `
+ `${partition.organization_member.phrase} reached as organization_member`);
let orgStatus = null;
let repoStatus = null;
if (withOrgProbe) {
const probe = await fetch(`${API}/orgs/${org}/members?per_page=1`,
{ headers: headers(token) });
orgStatus = probe.status;
console.log(`org probe: GET /orgs/${org}/members?per_page=1 -> HTTP ${orgStatus}`);
}
if (repoName) {
const probe = await fetch(`${API}/repos/${repoName}`, { headers: headers(token) });
repoStatus = probe.status;
console.log(`repo probe: GET /repos/${repoName} -> HTTP ${repoStatus}`);
}
const [probeState, probeDetail] = orgProbeReading(repoStatus, orgStatus);
console.log(`${probeState}: ${probeDetail}`);
const [role, roleDetail] = roleVerdict(member, partition.collaborator.count,
partition.organization_member.count);
console.log(`${role}: ${roleDetail}`);
const expectation = orgEndpointExpectation(role);
console.log(`quiet-failure-ahead: ${expectation['org-repos-listing']}`);
const [state, detail] = verdict(role, ssoState);
const fix = repair(state, org, login || 'this account');
console.log(`repair: ${fix}`);
console.log(JSON.stringify({
organization: org,
login,
is_member: member,
sso_state: ssoState,
affiliation_partition: partition,
org_probe_status: orgStatus,
repo_probe_status: repoStatus,
probe_state: probeState,
token_class_state: classState,
org_endpoint_expectation: expectation,
state,
detail,
repair: fix,
}, null, 2));
process.exitCode = ['outside-collaborator', 'member-with-no-implicit-repos',
'no-relationship', 'membership-list-incomplete'].includes(state) ? 1 : 0;
}
// Guarded so importing this file from the test runner does not start a run.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The core is a four-way sort and each of its arms sends you somewhere different, so each arm gets a test: an outside collaborator, a member whose implicit access was withdrawn organization-wide, a member who is fine, and an account with no relationship at all. Two more groups guard the honest parts. One is the SAML interaction: if the organization list arrived with a partial-results header, no membership conclusion may be drawn from it, and that has to override the sort rather than colour it. The other is the count, which must announce itself as a floor the moment the Link header says there is another page.
from github_outside_collaborator import (
AFFILIATIONS, counted, has_next_page, header_value, is_member,
org_endpoint_expectation, org_probe_reading, read_cost, repair,
repos_in_org, role_verdict, sso_reading, token_class_caveat, verdict,
)
# Obviously fake and far shorter than any real credential.
FINE = "github_pat_FAKE"
CLASSIC = "ghp_FAKE"
ORGS = [{"login": "acme"}, {"login": "Other"}]
REPOS = [
{"full_name": "acme/payments", "owner": {"login": "acme"}},
{"full_name": "acme/billing", "owner": {"login": "ACME"}},
{"full_name": "elsewhere/thing", "owner": {"login": "elsewhere"}},
]
NEXT_LINK = ('<https://api.github.com/user/repos?page=2>; rel="next", '
'<https://api.github.com/user/repos?page=9>; rel="last"')
LAST_ONLY = '<https://api.github.com/user/repos?page=1>; rel="prev"'
def test_the_partition_is_the_diagnosis():
# Repositories in the org reached as a collaborator, none as a member.
state, detail = role_verdict(False, 3, 0)
assert state == "outside-collaborator"
assert "No scope grants standing" in detail
def test_each_other_arm_of_the_sort_sends_you_somewhere_else():
assert role_verdict(True, 0, 12)[0] == "organization-member"
state, detail = role_verdict(True, 0, 0)
assert state == "member-with-no-implicit-repos"
assert "base permission of none" in detail
state, detail = role_verdict(False, 0, 0)
assert state == "no-relationship"
assert "removal rather than a role" in detail
def test_an_announced_partial_list_overrides_the_whole_sort():
# If GET /user/orgs was explicitly incomplete, the membership answer it
# rests on is not evidence, whatever the affiliation counts say.
state, detail = verdict("outside-collaborator", "sso-partial-results")
assert state == "membership-list-incomplete"
assert "no membership answer from it can be trusted" in detail
assert verdict("outside-collaborator", "no-sso-header")[0] == "outside-collaborator"
def test_the_absence_of_the_sso_header_is_read_and_reported():
state, detail = sso_reading({})
assert state == "no-sso-header"
assert "The SAML note is about the case where GitHub does tell you." in detail
partial = {"X-GitHub-SSO": "partial-results; organizations=1,2"}
assert sso_reading(partial)[0] == "sso-partial-results"
assert sso_reading({"x-github-sso": "required; url=https://example"})[0] == (
"sso-header-present")
def test_membership_and_ownership_comparisons_are_case_insensitive():
assert is_member(ORGS, "ACME") is True
assert is_member(ORGS, "nope") is False
assert is_member([], "acme") is False
assert repos_in_org(REPOS, "acme") == ["acme/payments", "acme/billing"]
assert repos_in_org(REPOS, "elsewhere") == ["elsewhere/thing"]
def test_a_count_says_when_it_is_only_a_floor():
assert has_next_page(NEXT_LINK) is True
assert has_next_page(LAST_ONLY) is False
assert has_next_page(None) is False
total, exact, phrase = counted(["a", "b"], True)
assert (total, exact, phrase) == (2, False, "at least 2")
assert counted(["a", "b"], False) == (2, True, "2")
assert counted([], False) == (0, True, "0")
def test_the_endpoint_that_does_not_fail_is_named_as_the_dangerous_one():
expectation = org_endpoint_expectation("outside-collaborator")
assert "under-reports" in expectation["org-repos-listing"]
assert "404 rather than 403" in expectation["members-and-teams"]
# And the one call that would name the condition needs access this
# account does not have, which is the joke at the centre of the note.
assert "organization read access" in expectation["outside-collaborators-listing"]
assert "answer for a member" in org_endpoint_expectation(
"organization-member")["members-and-teams"]
def test_the_pair_of_readings_is_the_sentence_for_the_ticket():
state, detail = org_probe_reading(200, 404)
assert state == "repo-yes-org-no"
assert "put in the ticket" in detail
assert org_probe_reading(200, 200)[0] == "org-reachable"
assert org_probe_reading(200, 403)[0] == "org-refused-not-hidden"
assert org_probe_reading(200, None)[0] == "org-not-probed"
def test_the_documented_fine_grained_gap_can_invert_the_answer():
state, detail = token_class_caveat(FINE)
assert state == "fine-grained-gap"
assert "outside or repository collaborator" in detail
assert token_class_caveat(CLASSIC)[0] == "classic-token"
assert token_class_caveat("")[0] == "class-not-recognised"
def test_the_repair_offers_two_choices_and_takes_neither():
fix = repair("outside-collaborator", "acme", "dana-integration")
assert "ask an owner of acme" in fix
assert "work at repository scope" in fix
assert "Nothing here invites anybody" in fix
assert "default repository permission" in repair(
"member-with-no-implicit-repos", "acme", "dana")
def test_the_read_cost_and_the_affiliation_names():
assert read_cost(False) == 3
assert read_cost(True) == 4
assert AFFILIATIONS == ("owner", "collaborator", "organization_member")
def test_header_reads_survive_whatever_case_the_client_gives_them():
assert header_value({"Link": "x"}, "link") == "x"
assert header_value({"link": "x"}, "LINK") == "x"
assert header_value(None, "link") is None
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
AFFILIATIONS, counted, hasNextPage, headerValue, isMember,
orgEndpointExpectation, orgProbeReading, readCost, repair, reposInOrg,
roleVerdict, ssoReading, tokenClassCaveat, verdict,
} from './github-outside-collaborator.mjs';
// Obviously fake and far shorter than any real credential.
const FINE = 'github_pat_FAKE';
const CLASSIC = 'ghp_FAKE';
const ORGS = [{ login: 'acme' }, { login: 'Other' }];
const REPOS = [
{ full_name: 'acme/payments', owner: { login: 'acme' } },
{ full_name: 'acme/billing', owner: { login: 'ACME' } },
{ full_name: 'elsewhere/thing', owner: { login: 'elsewhere' } },
];
const NEXT_LINK = '<https://api.github.com/user/repos?page=2>; rel="next", '
+ '<https://api.github.com/user/repos?page=9>; rel="last"';
const LAST_ONLY = '<https://api.github.com/user/repos?page=1>; rel="prev"';
test('the partition is the diagnosis', () => {
const [state, detail] = roleVerdict(false, 3, 0);
assert.equal(state, 'outside-collaborator');
assert.match(detail, /No scope grants standing/);
});
test('each other arm of the sort sends you somewhere else', () => {
assert.equal(roleVerdict(true, 0, 12)[0], 'organization-member');
let [state, detail] = roleVerdict(true, 0, 0);
assert.equal(state, 'member-with-no-implicit-repos');
assert.match(detail, /base permission of none/);
[state, detail] = roleVerdict(false, 0, 0);
assert.equal(state, 'no-relationship');
assert.match(detail, /removal rather than a role/);
});
test('an announced partial list overrides the whole sort', () => {
const [state, detail] = verdict('outside-collaborator', 'sso-partial-results');
assert.equal(state, 'membership-list-incomplete');
assert.match(detail, /no membership answer from it can be trusted/);
assert.equal(verdict('outside-collaborator', 'no-sso-header')[0], 'outside-collaborator');
});
test('the absence of the sso header is read and reported', () => {
const [state, detail] = ssoReading({});
assert.equal(state, 'no-sso-header');
assert.match(detail, /The SAML note is about the case where GitHub does tell you\./);
assert.equal(ssoReading({ 'X-GitHub-SSO': 'partial-results; organizations=1,2' })[0],
'sso-partial-results');
assert.equal(ssoReading({ 'x-github-sso': 'required; url=https://example' })[0],
'sso-header-present');
});
test('membership and ownership comparisons are case insensitive', () => {
assert.equal(isMember(ORGS, 'ACME'), true);
assert.equal(isMember(ORGS, 'nope'), false);
assert.equal(isMember([], 'acme'), false);
assert.deepEqual(reposInOrg(REPOS, 'acme'), ['acme/payments', 'acme/billing']);
assert.deepEqual(reposInOrg(REPOS, 'elsewhere'), ['elsewhere/thing']);
});
test('a count says when it is only a floor', () => {
assert.equal(hasNextPage(NEXT_LINK), true);
assert.equal(hasNextPage(LAST_ONLY), false);
assert.equal(hasNextPage(null), false);
assert.deepEqual(counted(['a', 'b'], true), [2, false, 'at least 2']);
assert.deepEqual(counted(['a', 'b'], false), [2, true, '2']);
assert.deepEqual(counted([], false), [0, true, '0']);
});
test('the endpoint that does not fail is named as the dangerous one', () => {
const expectation = orgEndpointExpectation('outside-collaborator');
assert.match(expectation['org-repos-listing'], /under-reports/);
assert.match(expectation['members-and-teams'], /404 rather than 403/);
assert.match(expectation['outside-collaborators-listing'], /organization read access/);
assert.match(orgEndpointExpectation('organization-member')['members-and-teams'],
/answer for a member/);
});
test('the pair of readings is the sentence for the ticket', () => {
const [state, detail] = orgProbeReading(200, 404);
assert.equal(state, 'repo-yes-org-no');
assert.match(detail, /put in the ticket/);
assert.equal(orgProbeReading(200, 200)[0], 'org-reachable');
assert.equal(orgProbeReading(200, 403)[0], 'org-refused-not-hidden');
assert.equal(orgProbeReading(200, null)[0], 'org-not-probed');
});
test('the documented fine grained gap can invert the answer', () => {
const [state, detail] = tokenClassCaveat(FINE);
assert.equal(state, 'fine-grained-gap');
assert.match(detail, /outside or repository collaborator/);
assert.equal(tokenClassCaveat(CLASSIC)[0], 'classic-token');
assert.equal(tokenClassCaveat('')[0], 'class-not-recognised');
});
test('the repair offers two choices and takes neither', () => {
const fix = repair('outside-collaborator', 'acme', 'dana-integration');
assert.match(fix, /ask an owner of acme/);
assert.match(fix, /work at repository scope/);
assert.match(fix, /Nothing here invites anybody/);
assert.match(repair('member-with-no-implicit-repos', 'acme', 'dana'),
/default repository permission/);
});
test('the read cost and the affiliation names', () => {
assert.equal(readCost(false), 3);
assert.equal(readCost(true), 4);
assert.deepEqual(AFFILIATIONS, ['owner', 'collaborator', 'organization_member']);
});
test('header reads survive whatever case the client gives them', () => {
assert.equal(headerValue({ Link: 'x' }, 'link'), 'x');
assert.equal(headerValue({ link: 'x' }, 'LINK'), 'x');
assert.equal(headerValue(null, 'link'), null);
});
FAQ
How is this different from the note about SSO withholding organizations?
That note is about GitHub telling you it left something out. A cross-organization list returns 200 and an <code>X-GitHub-SSO: partial-results</code> header naming the organizations withheld, and the finding is that header on a successful response. Here nothing is announced: an outside collaborator listing an organization's repositories gets a clean 200 with a shorter array and no header at all. The script reads that header specifically so it can rule the other note out with evidence — and if the header <em>is</em> there, it stops and says so, because a membership conclusion drawn from an admittedly partial list is worthless.
Would <code>GET /orgs/{org}/members/{me}</code> not settle this faster?
It answers, and it does not settle it. That endpoint returns 302 when the requester is not an organization member, which is a genuinely useful signal and belongs to the note about accounts removed by a two-factor requirement. The trouble here is that an outside collaborator and a former member are both non-members and both get the redirect. The thing that separates them is the repositories: an outside collaborator still reaches theirs and a removed member reaches nothing, which is why the diagnosis is built on the affiliation partition instead.
Why does adding <code>read:org</code> not help?
Because a scope is an upper bound on what a token may do <em>on the account's behalf</em>, not a grant of access to the account. The account is not in the organization, so there is no organization access for the scope to permit. This is the same reason the section's note about a repository role exists: the token and the relationship are two different things, and only one of them is something you can change by editing a token.
Can the script confirm it by listing the outside collaborators?
That is the endpoint that would say it outright — <code>GET /orgs/{org}/outside_collaborators</code> — and it needs organization read access, which is precisely what an outside collaborator does not have. So the one call that names the condition is the one call this account cannot make. Where you happen to hold an organization credential as well, run it and the answer is immediate; from the affected token, the partition is the evidence available and it is sufficient.
Our repository listing looks fine. Are we affected?
Check which call builds it. <code>GET /orgs/{org}/repos</code> does not refuse a non-member; it answers 200 with the public repositories and stops. An integration built on that does not raise, does not log an error and does not alert — it just shows a customer with forty repositories as a customer with two. That silent version is more expensive than the 404s, because the 404s at least send somebody to look.
Related field notes
- The other short list, and the header that announces it
- The other way to stop being a member
- When the account is a member and the repos still vanish
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Outside collaborators — GitHub REST API
- Adding outside collaborators to repositories in your organization — GitHub Docs
- Repositories — GitHub REST API
- Managing your personal access tokens — GitHub Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.