Diagnostic GitHub API
Enforcing 2FA removed the machine account from the org
On Tuesday the sync job read forty repositories. On Wednesday it reads none of them, and every call comes back 404 rather than 403. The token is checked first, because it always is: GET /user answers 200, the login is right, the expiry is months away, the scopes are unchanged. Nothing was rotated and nothing was deployed. What happened is that an owner turned on required two-factor authentication, and the accounts that did not have 2FA were removed from the organization — which for a machine account created in a hurry three years ago means all of them.
Enabling a 2FA requirement does not refuse non-compliant members, it removes them. Members and outside collaborators without two-factor authentication are taken out of the organization, and nothing tells the integration holding their token.
Two reads settle it. GET /user proves the token is alive and names the login. GET /orgs/{org}/members/{login}, without following redirects, answers 204 if the account is a member and 302 if the requester is not one — and after a removal the requester is you. A client that follows that redirect lands on the public-members endpoint and answers a different question entirely. GET /orgs/{org} supplies the motive in two_factor_requirement_enabled, when you can still read it.
The problem in plain words
The first hour goes on the token, because the token is the thing that usually breaks and the thing that can be checked in one call. That call passes. So does the next one, and the next: the credential authenticates, it has not expired, its scopes are the same as they were on Tuesday, and it can still read public repositories and the account's own gists. Everything about the credential is healthy, which is precisely the shape that makes this hard: the credential really is healthy. It just belongs to an account that is no longer in the organization.
The second hour goes on the 404s. A 404 on a private repository has a long list of causes, and the list is somebody else's note. What sends people down it is the belief that the repository is the subject. It is not; the subject is the account, and one read of the membership answers for all forty repositories at once instead of triaging them one at a time.
Then there is the part that makes it look like an outage rather than a policy change: it is not uniform. The requirement removes only the accounts that do not comply, so the two humans on the team, who have had 2FA on for years, notice nothing at all. Their tokens keep working. Only the bot went, and bots do not read email, so the notification that an owner enabled the requirement went to people for whom nothing changed.
Why it happens
The policy removes, it does not refuse. This is the sentence the whole note turns on. Most policy in this API produces a refusal you can read: a 403 with a rule named in the body, a header naming what would have been accepted. This one changes the membership graph. Once the account is out, there is no policy left to report, because policy is not what is being applied any more — the account simply has no relationship to that organization, and its private repositories are invisible in the ordinary way.
Which is why the symptom is 404 and not 403. GitHub answers 404 rather than 403 for private resources a caller cannot see, so a removed member gets exactly the response a caller who never had access gets, and exactly the response somebody would get for a repository that was deleted. The 404 triage note is the one that sorts those causes generally. This note is the specific cause you can confirm with a single membership read, and it is worth checking early because it explains an entire organization's worth of 404s in one call.
The membership endpoint answers with a redirect, and the redirect is the finding. GET /orgs/{org}/members/{username} returns 204 when the account is a member, 404 when the requester is a member and the named account is not, and 302 when the requester is not a member of the organization at all. When you are asking about yourself after being removed, the answer is that 302. Almost every HTTP client follows redirects by default, and the redirect goes to the public members endpoint, which asks whether the account's membership is publicly listed. A publicly-listed former member is not a thing, but the two endpoints can and do disagree in the other direction, so a client that follows the redirect quietly swaps one question for another. Turn redirect-following off for this call.
The motive becomes unreadable at the moment you need it. two_factor_requirement_enabled on the organization object is only returned to a caller with enough organization access. Being removed is what takes that access away, so the field that would explain the removal is frequently absent because of the removal. That is not a bug to work around, it is a fact to report: the script says the field was unreadable and names the removal as the finding rather than pretending the requirement is off.
Removal and never having joined are indistinguishable from here. A read-only observer sees the current graph, not its history. There is an audit log that records the removal, and reading it needs organization admin. So the honest finding is "this account is not a member, and the organization requires 2FA", which is a cause and a motive rather than a proof, and it is enough to act on. The script says so in those terms rather than claiming to have watched it happen.
The fix, as a flow
Two reads settle this and the second one has to be sent carefully. GET /user proves the credential is healthy, which is the line that ends the search everybody runs first. The membership call is then sent with redirects disabled, because its 302 means the account asking is not a member, and asking about yourself that redirect is the finding. A client that follows it lands on the public members endpoint and answers a question about public listing instead, just as confidently. The motive is read last and is often unreadable, because losing organization access is exactly what the removal did.
How to fix it
Prove the token is healthy, so that stops being the search
The script reads GET /user first and prints that it succeeded, with the login it resolved. That line exists to close the investigation everyone runs first: the credential is fine, and every further minute spent on it is wasted. The same response carries two_factor_authentication when the token has the user scope, which becomes useful two steps later.
Ask the membership question without letting the client rewrite it
GET /orgs/{org}/members/{login} is sent with redirects disabled. 204 means the account is still a member. 302 means the requester is not one, which — when the requester is the account you are asking about — is the removal. A client that follows the redirect gets an answer from the public-members endpoint instead, and that answer is about a different property. The script reports which question was actually answered.
Read the motive while you still can
GET /orgs/{org} carries two_factor_requirement_enabled for callers with organization access. If it is true, the removal has a name. If the field is absent, the script says it was unreadable rather than reporting the requirement as off, because the most common reason for it to be missing is the removal itself.
Check whether you are the next one out
When the account is still a member and the organization requires 2FA, the script compares that against two_factor_authentication on the account. A member with 2FA off inside an organization that requires it is a removal that has not happened yet — usually because the requirement was enabled after a grace conversation, or because the account joined a different organization with the same policy. That is the one reading here that is worth having before anything breaks.
Take the repair to the account, or past it
The printed repair is either "enable 2FA on this account and have an owner re-invite it" or, better, "replace the machine account with a GitHub App installation". An installation is not a member and is not subject to member 2FA policy, so the same class of change cannot remove it. The script re-invites nobody and enables nothing; it prints and exits.
How to check it worked
After the account is re-invited with 2FA enabled, the membership read stops redirecting and answers 204.
GITHUB_TOKEN=$READ_ONLY_TOKEN python3 github_org_membership_lost.py acme
# read cost: 4 request(s) against the core hourly quota
# token: healthy — GET /user answered 200 as octobot. Nothing about the
# credential explains what follows.
# membership: GET /orgs/acme/members/octobot -> HTTP 302 (redirects disabled)
# requester-not-a-member: the 302 says the account asking is not a member of
# acme. Asking about yourself, that is the removal.
# question answered: membership. Following the redirect would have answered a
# question about public membership instead.
# motive: two_factor_requirement_enabled was not returned. Reading it needs org
# access, and losing that access is what this finding is.
# state: not-a-member-motive-unreadable
# symptom: every private repository in acme answers 404, not 403, because a
# non-member cannot see them at all.
# repair: enable 2FA on octobot and ask an owner of acme to re-invite it, or
# replace the machine account with a GitHub App installation, which is not a
# member and is unaffected by member 2FA policy.The full code
Three of the four reads are one line each; the work is in refusing to let an HTTP client answer a question you did not ask. The membership call goes out with redirect-following disabled and the script reports which question the status code it got is actually about, because a followed 302 produces a confident answer to a different one. Everything after that is pure: the status-to-state mapping, the three-valued readings where an absent field means unreadable rather than false, and the combination that turns a membership, a requirement and an account's own 2FA into one finding.
"""Say whether an account was removed from an organization by a 2FA rule.
Read only. Four GETs and nothing else. Re-inviting a removed member is a write
and a decision for an organization owner, so this script does not make it: it
establishes the removal from readable state and prints the request.
The point of the note: enabling required two-factor authentication does not
refuse non-compliant members, it removes them. The token keeps working, the
account stops being a member, and every private repository in the organization
answers 404 rather than 403.
What this can and cannot see: the current graph, not its history. Removal and
never having joined look identical from here; the audit log that records the
removal needs organization admin. So the finding is a cause and a motive rather
than a proof, and it is reported in those terms.
Environment:
GITHUB_TOKEN the token the failing integration holds
"""
import argparse
import json
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("github_org_membership_lost")
API = "https://api.github.com"
UA = "github-org-membership-lost/1.0"
# GET /orgs/{org}/members/{username} answers with a status and no body. The 302
# is the one that matters and the one a default HTTP client hides.
MEMBERSHIP_STATUS = {
204: "member",
302: "requester-not-a-member",
404: "not-a-member",
403: "membership-unreadable",
}
def read_cost():
"""Requests this run will spend against the core quota. Pure."""
return 4
def membership_state(status):
"""What one membership status means. Pure. (state, detail).
The 302 is documented as "requester is not an organization member". When
the requester and the subject are the same account, that redirect is the
whole finding, and it is the reading a redirect-following client destroys.
"""
code = int(status or 0)
state = MEMBERSHIP_STATUS.get(code, "unclear")
if state == "member":
return (state, "204 means the account is a member of this organization.")
if state == "requester-not-a-member":
return (state, "the 302 says the account asking is not a member of the "
"organization. Asking about yourself, that is the removal.")
if state == "not-a-member":
return (state, "404 here means the requester is a member and the named "
"account is not.")
if state == "membership-unreadable":
return (state, "403 on the membership read itself. The credential "
"reached GitHub and was refused; sort that refusal first.")
return ("unclear", "HTTP %s is not one of the documented answers for this "
"endpoint." % status)
def question_answered(followed_redirect):
"""Which question the status code is about. Pure. (state, detail).
A followed 302 lands on the public-members endpoint, which asks whether a
membership is publicly listed. That is a different property, and the answer
looks exactly as authoritative as the one you wanted.
"""
if followed_redirect:
return ("public-membership-instead",
"the client followed the redirect, so this answer came from "
"the public members endpoint and is about whether membership "
"is publicly listed. Send the call again with redirects off.")
return ("membership",
"redirects were disabled, so the status describes membership "
"rather than public membership.")
def own_two_factor(user_payload):
"""Whether this account has 2FA on. Pure. True, False or None.
None means unreadable: the field is only present on the authenticated
user's own record and only when the token carries the user scope. Reporting
an absent field as False would invent a compliance failure.
"""
if not isinstance(user_payload, dict):
return None
if "two_factor_authentication" not in user_payload:
return None
value = user_payload.get("two_factor_authentication")
return None if value is None else bool(value)
def requirement_state(org_payload):
"""Whether the org requires 2FA. Pure. True, False or None.
None means unreadable, and the most common reason for that is the removal
itself: the field is returned to callers with organization access, and
being removed is what takes it away.
"""
if not isinstance(org_payload, dict):
return None
if "two_factor_requirement_enabled" not in org_payload:
return None
value = org_payload.get("two_factor_requirement_enabled")
return None if value is None else bool(value)
def listed_in_orgs(orgs, org):
"""Is the organization in GET /user/orgs. Pure.
Weaker evidence than it looks. Without read:org a classic token sees only
the organizations whose membership the account has made public, so absence
here is corroboration and never the finding.
"""
wanted = str(org or "").strip().lower()
for entry in orgs or []:
login = entry.get("login") if isinstance(entry, dict) else entry
if str(login or "").strip().lower() == wanted:
return True
return False
def combine(membership, requirement, own_2fa):
"""Turn three readings into one finding. Pure. (state, detail)."""
gone = membership in ("requester-not-a-member", "not-a-member")
if gone and requirement is True:
return ("not-a-member-2fa-required",
"the account is not a member and the organization requires "
"two-factor authentication. That is the cause and its motive. "
"Removal and never having joined are indistinguishable through "
"the API, so this is a finding to act on rather than a proof.")
if gone and requirement is None:
return ("not-a-member-motive-unreadable",
"the account is not a member and the 2FA requirement could not "
"be read. Reading it needs organization access, and losing that "
"access is what this finding is.")
if gone:
return ("not-a-member-no-requirement",
"the account is not a member and the organization does not "
"require 2FA, so something else removed it. An owner can read "
"the audit log; a read-only token cannot.")
if membership == "member" and requirement is True and own_2fa is False:
return ("member-at-risk",
"still a member, the organization requires 2FA, and this "
"account reports two-factor authentication off. That is a "
"removal that has not happened yet.")
if membership == "member" and requirement is True and own_2fa is None:
return ("member-compliance-unreadable",
"still a member of an organization that requires 2FA, and this "
"token cannot read whether the account complies. The user scope "
"is what exposes that field.")
if membership == "member" and requirement is True:
return ("member-compliant",
"a member, the requirement is on, and this account has 2FA. "
"Nothing here explains a 404.")
if membership == "member":
return ("member-no-requirement",
"a member of an organization with no 2FA requirement. This "
"note is not your problem; sort the 404 another way.")
return ("membership-unreadable",
"the membership question was not answered, so nothing can be "
"concluded about a removal.")
def symptom(state):
"""What the integration is seeing, given the finding. Pure."""
if state.startswith("not-a-member"):
return ("every private repository in the organization answers 404, not "
"403, because a non-member cannot see them at all. Public "
"repositories keep answering, which is what makes the token "
"look healthy.")
if state == "member-at-risk":
return ("nothing yet. The reads still work and will keep working until "
"the requirement is enforced against this account.")
return ("nothing that this note explains.")
def token_health(status):
"""State the credential's health explicitly. Pure. (state, detail)."""
if int(status or 0) == 200:
return ("healthy",
"GET /user answered 200, so the credential authenticates. "
"Nothing about the token explains what follows, and this line "
"exists to end that search early.")
if int(status or 0) == 401:
return ("rejected",
"401 means the credential itself was not accepted, which is a "
"different note. This one starts from a token that works.")
return ("unclear", "GET /user answered %s, which is neither of the two "
"cases this note starts from." % status)
def repair(state, org, login):
"""The request a human has to make. Pure. Nothing here is executed."""
if state.startswith("not-a-member"):
return ("enable 2FA on %s and ask an owner of %s to re-invite it, or "
"replace the machine account with a GitHub App installation, "
"which is not a member and is unaffected by member 2FA policy. "
"Nothing here re-invites anybody." % (login, org))
if state == "member-at-risk":
return ("enable 2FA on %s now, before the requirement is enforced "
"against it. Removal is silent when it comes." % login)
if state == "member-compliance-unreadable":
return ("read this with a token carrying the user scope, or check the "
"account's security settings directly, to confirm it complies.")
if state == "member-compliant":
return ("nothing on membership. Take the 404 to the repository-level "
"causes instead.")
return ("answer the membership question first: send the members call with "
"redirects disabled and read the status rather than the body.")
def get(session, path, allow_redirects=True):
"""One GET. Redirect following is a parameter because one call needs it off."""
return session.get(API + path, timeout=30, allow_redirects=allow_redirects)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("org", help="the organization whose repositories 404")
parser.add_argument("--login",
help="ask about this account instead of the token's own")
args = parser.parse_args()
token = os.environ.get("GITHUB_TOKEN")
if not token:
log.error("set GITHUB_TOKEN (the token the failing integration holds)")
return 2
log.info("read cost: %d request(s) against the core hourly quota", read_cost())
session = requests.Session()
session.headers.update({
"Authorization": "Bearer " + token,
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": UA,
})
me = get(session, "/user")
health, health_detail = token_health(me.status_code)
payload = me.json() if me.status_code == 200 else {}
login = args.login or payload.get("login") or "unknown"
log.info("token: %s — %s", health, health_detail)
if health != "healthy":
return 2
members = get(session, "/orgs/%s/members/%s" % (args.org, login),
allow_redirects=False)
state, detail = membership_state(members.status_code)
log.info("membership: GET /orgs/%s/members/%s -> HTTP %s (redirects disabled)",
args.org, login, members.status_code)
log.info("%s: %s", state, detail)
asked, asked_detail = question_answered(False)
log.info("question answered: %s. %s", asked, asked_detail)
org_response = get(session, "/orgs/" + args.org)
org_payload = org_response.json() if org_response.status_code == 200 else {}
requirement = requirement_state(org_payload)
log.info("motive: two_factor_requirement_enabled=%s",
"unreadable" if requirement is None else requirement)
orgs_response = get(session, "/user/orgs?per_page=100")
orgs = orgs_response.json() if orgs_response.status_code == 200 else []
listed = listed_in_orgs(orgs, args.org)
log.info("corroboration: the organization is %s in GET /user/orgs, which "
"without read:org only lists publicly-visible membership",
"listed" if listed else "absent")
own = own_two_factor(payload) if not args.login else None
finding, finding_detail = combine(state, requirement, own)
log.info("state: %s — %s", finding, finding_detail)
log.info("symptom: %s", symptom(finding))
log.info("repair: %s", repair(finding, args.org, login))
print(json.dumps({
"organization": args.org,
"login": login,
"token_health": health,
"membership_status": members.status_code,
"membership_state": state,
"question_answered": asked,
"two_factor_requirement_enabled": requirement,
"account_two_factor": own,
"listed_in_user_orgs": listed,
"state": finding,
"detail": finding_detail,
"symptom": symptom(finding),
"repair": repair(finding, args.org, login),
}, indent=2, default=str))
return 1 if finding.startswith("not-a-member") or finding == "member-at-risk" else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Say whether an account was removed from an organization by a 2FA rule.
*
* Read only. Four GETs. Re-inviting a removed member is a write and an
* organization owner's decision, so this script establishes the removal from
* readable state and prints the request instead of making it.
*
* Enabling required two-factor authentication removes non-compliant members
* rather than refusing them. The token keeps working, the account stops being
* a member, and every private repository in the organization answers 404.
*
* Environment:
* GITHUB_TOKEN the token the failing integration holds
* GITHUB_ORG the organization whose repositories 404
* GITHUB_LOGIN optional, ask about this account instead of the token's own
*/
const API = 'https://api.github.com';
const UA = 'github-org-membership-lost/1.0';
/** The documented answers of GET /orgs/{org}/members/{username}. */
export const MEMBERSHIP_STATUS = {
204: 'member',
302: 'requester-not-a-member',
404: 'not-a-member',
403: 'membership-unreadable',
};
/** Requests this run will spend against the core quota. Pure. */
export function readCost() {
return 4;
}
/** What one membership status means. Pure. [state, detail]. */
export function membershipState(status) {
const code = Number(status) || 0;
const state = MEMBERSHIP_STATUS[code] || 'unclear';
if (state === 'member') {
return [state, '204 means the account is a member of this organization.'];
}
if (state === 'requester-not-a-member') {
return [state, 'the 302 says the account asking is not a member of the '
+ 'organization. Asking about yourself, that is the removal.'];
}
if (state === 'not-a-member') {
return [state, '404 here means the requester is a member and the named '
+ 'account is not.'];
}
if (state === 'membership-unreadable') {
return [state, '403 on the membership read itself. The credential reached '
+ 'GitHub and was refused; sort that refusal first.'];
}
return ['unclear', `HTTP ${status} is not one of the documented answers for `
+ 'this endpoint.'];
}
/** Which question the status code is about. Pure. [state, detail]. */
export function questionAnswered(followedRedirect) {
if (followedRedirect) {
return ['public-membership-instead', 'the client followed the redirect, so '
+ 'this answer came from the public members endpoint and is about whether '
+ 'membership is publicly listed. Send the call again with redirects off.'];
}
return ['membership', 'redirects were disabled, so the status describes '
+ 'membership rather than public membership.'];
}
/** Whether this account has 2FA on. Pure. true, false or null for unreadable. */
export function ownTwoFactor(userPayload) {
if (!userPayload || typeof userPayload !== 'object') return null;
if (!Object.prototype.hasOwnProperty.call(userPayload, 'two_factor_authentication')) {
return null;
}
const value = userPayload.two_factor_authentication;
return value === null || value === undefined ? null : Boolean(value);
}
/** Whether the org requires 2FA. Pure. true, false or null for unreadable. */
export function requirementState(orgPayload) {
if (!orgPayload || typeof orgPayload !== 'object') return null;
if (!Object.prototype.hasOwnProperty.call(orgPayload, 'two_factor_requirement_enabled')) {
return null;
}
const value = orgPayload.two_factor_requirement_enabled;
return value === null || value === undefined ? null : Boolean(value);
}
/** Is the organization in GET /user/orgs. Pure. Corroboration, never a finding. */
export function listedInOrgs(orgs, org) {
const wanted = String(org ?? '').trim().toLowerCase();
for (const entry of orgs || []) {
const login = entry && typeof entry === 'object' ? entry.login : entry;
if (String(login ?? '').trim().toLowerCase() === wanted) return true;
}
return false;
}
/** Turn three readings into one finding. Pure. [state, detail]. */
export function combine(membership, requirement, ownTwoFa) {
const gone = ['requester-not-a-member', 'not-a-member'].includes(membership);
if (gone && requirement === true) {
return ['not-a-member-2fa-required', 'the account is not a member and the '
+ 'organization requires two-factor authentication. That is the cause and '
+ 'its motive. Removal and never having joined are indistinguishable '
+ 'through the API, so this is a finding to act on rather than a proof.'];
}
if (gone && (requirement === null || requirement === undefined)) {
return ['not-a-member-motive-unreadable', 'the account is not a member and '
+ 'the 2FA requirement could not be read. Reading it needs organization '
+ 'access, and losing that access is what this finding is.'];
}
if (gone) {
return ['not-a-member-no-requirement', 'the account is not a member and the '
+ 'organization does not require 2FA, so something else removed it. An '
+ 'owner can read the audit log; a read-only token cannot.'];
}
if (membership === 'member' && requirement === true && ownTwoFa === false) {
return ['member-at-risk', 'still a member, the organization requires 2FA, '
+ 'and this account reports two-factor authentication off. That is a '
+ 'removal that has not happened yet.'];
}
if (membership === 'member' && requirement === true
&& (ownTwoFa === null || ownTwoFa === undefined)) {
return ['member-compliance-unreadable', 'still a member of an organization '
+ 'that requires 2FA, and this token cannot read whether the account '
+ 'complies. The user scope is what exposes that field.'];
}
if (membership === 'member' && requirement === true) {
return ['member-compliant', 'a member, the requirement is on, and this '
+ 'account has 2FA. Nothing here explains a 404.'];
}
if (membership === 'member') {
return ['member-no-requirement', 'a member of an organization with no 2FA '
+ 'requirement. This note is not your problem; sort the 404 another way.'];
}
return ['membership-unreadable', 'the membership question was not answered, '
+ 'so nothing can be concluded about a removal.'];
}
/** What the integration is seeing, given the finding. Pure. */
export function symptom(state) {
if (String(state).startsWith('not-a-member')) {
return 'every private repository in the organization answers 404, not 403, '
+ 'because a non-member cannot see them at all. Public repositories keep '
+ 'answering, which is what makes the token look healthy.';
}
if (state === 'member-at-risk') {
return 'nothing yet. The reads still work and will keep working until the '
+ 'requirement is enforced against this account.';
}
return 'nothing that this note explains.';
}
/** State the credential's health explicitly. Pure. [state, detail]. */
export function tokenHealth(status) {
const code = Number(status) || 0;
if (code === 200) {
return ['healthy', 'GET /user answered 200, so the credential '
+ 'authenticates. Nothing about the token explains what follows, and this '
+ 'line exists to end that search early.'];
}
if (code === 401) {
return ['rejected', '401 means the credential itself was not accepted, '
+ 'which is a different note. This one starts from a token that works.'];
}
return ['unclear', `GET /user answered ${status}, which is neither of the two `
+ 'cases this note starts from.'];
}
/** The request a human has to make. Pure. Nothing here is executed. */
export function repair(state, org, login) {
if (String(state).startsWith('not-a-member')) {
return `enable 2FA on ${login} and ask an owner of ${org} to re-invite it, `
+ 'or replace the machine account with a GitHub App installation, which is '
+ 'not a member and is unaffected by member 2FA policy. Nothing here '
+ 're-invites anybody.';
}
if (state === 'member-at-risk') {
return `enable 2FA on ${login} now, before the requirement is enforced `
+ 'against it. Removal is silent when it comes.';
}
if (state === 'member-compliance-unreadable') {
return 'read this with a token carrying the user scope, or check the '
+ "account's security settings directly, to confirm it complies.";
}
if (state === 'member-compliant') {
return 'nothing on membership. Take the 404 to the repository-level causes '
+ 'instead.';
}
return 'answer the membership question first: send the members call with '
+ 'redirects disabled and read the status rather than the body.';
}
function headers(token) {
return {
Authorization: `Bearer ${token}`,
Accept: 'application/vnd.github+json',
'X-GitHub-Api-Version': '2022-11-28',
'User-Agent': UA,
};
}
async function main() {
const token = process.env.GITHUB_TOKEN;
const org = process.env.GITHUB_ORG;
if (!token || !org) {
console.error('set GITHUB_TOKEN (the failing token) and GITHUB_ORG');
process.exitCode = 2;
return;
}
console.log(`read cost: ${readCost()} request(s) against the core hourly quota`);
const me = await fetch(`${API}/user`, { headers: headers(token) });
const [health, healthDetail] = tokenHealth(me.status);
console.log(`token: ${health} — ${healthDetail}`);
if (health !== 'healthy') { process.exitCode = 2; return; }
const payload = await me.json();
const login = process.env.GITHUB_LOGIN || payload.login || 'unknown';
// redirect: manual, because the 302 is the finding and following it answers
// a question about public membership instead.
const members = await fetch(`${API}/orgs/${org}/members/${login}`, {
headers: headers(token), redirect: 'manual',
});
const [state, detail] = membershipState(members.status);
console.log(`membership: GET /orgs/${org}/members/${login} -> HTTP ${members.status} `
+ '(redirects disabled)');
console.log(`${state}: ${detail}`);
const [asked, askedDetail] = questionAnswered(false);
console.log(`question answered: ${asked}. ${askedDetail}`);
const orgResponse = await fetch(`${API}/orgs/${org}`, { headers: headers(token) });
const orgPayload = orgResponse.status === 200 ? await orgResponse.json() : {};
const requirement = requirementState(orgPayload);
console.log(`motive: two_factor_requirement_enabled=`
+ `${requirement === null ? 'unreadable' : requirement}`);
const orgsResponse = await fetch(`${API}/user/orgs?per_page=100`, {
headers: headers(token),
});
const orgs = orgsResponse.status === 200 ? await orgsResponse.json() : [];
const listed = listedInOrgs(orgs, org);
console.log(`corroboration: the organization is ${listed ? 'listed' : 'absent'} `
+ 'in GET /user/orgs, which without read:org only lists publicly-visible '
+ 'membership');
const own = process.env.GITHUB_LOGIN ? null : ownTwoFactor(payload);
const [finding, findingDetail] = combine(state, requirement, own);
console.log(`state: ${finding} — ${findingDetail}`);
console.log(`symptom: ${symptom(finding)}`);
console.log(`repair: ${repair(finding, org, login)}`);
console.log(JSON.stringify({
organization: org,
login,
token_health: health,
membership_status: members.status,
membership_state: state,
question_answered: asked,
two_factor_requirement_enabled: requirement,
account_two_factor: own,
listed_in_user_orgs: listed,
state: finding,
detail: findingDetail,
symptom: symptom(finding),
repair: repair(finding, org, login),
}, null, 2));
process.exitCode = (String(finding).startsWith('not-a-member')
|| finding === 'member-at-risk') ? 1 : 0;
}
// Guarded so importing this file from the test runner does not start a run.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The status table comes first, because the 302 is the reading this note exists to rescue and it is the one every default HTTP client throws away. Then the three-valued fields, twice, in both directions: an absent two_factor_requirement_enabled has to be unreadable rather than false, or the script would report a removal as unexplained at exactly the moment the removal is what made the field unreadable; and an absent two_factor_authentication has to be unreadable rather than false, or it would invent a compliance failure. The combination table is asserted state by state, including the pre-emptive one — a member with 2FA off inside an organization that requires it — and the repair is asserted to name a GitHub App, because that is the repair that makes the same change impossible next time.
from github_org_membership_lost import (
combine, listed_in_orgs, membership_state, own_two_factor,
question_answered, read_cost, repair, requirement_state, symptom,
token_health,
)
def test_the_redirect_is_the_finding_and_not_an_error():
state, detail = membership_state(302)
assert state == "requester-not-a-member"
assert "that is the removal" in detail
assert membership_state(204)[0] == "member"
assert membership_state(404)[0] == "not-a-member"
assert membership_state(418)[0] == "unclear"
def test_following_the_redirect_answers_a_different_question():
state, detail = question_answered(True)
assert state == "public-membership-instead"
assert "publicly listed" in detail
assert question_answered(False)[0] == "membership"
def test_an_absent_requirement_field_is_unreadable_not_false():
# The field is returned to callers with org access, and being removed is
# what takes that away. Reading absence as False would report the removal
# as unexplained precisely when the removal is the explanation.
assert requirement_state({}) is None
assert requirement_state({"two_factor_requirement_enabled": None}) is None
assert requirement_state({"two_factor_requirement_enabled": True}) is True
assert requirement_state({"two_factor_requirement_enabled": False}) is False
def test_an_absent_two_factor_field_does_not_invent_a_violation():
assert own_two_factor({"login": "octobot"}) is None
assert own_two_factor({"two_factor_authentication": False}) is False
assert own_two_factor({"two_factor_authentication": True}) is True
assert own_two_factor(None) is None
def test_the_removal_and_its_motive_are_reported_together():
state, detail = combine("requester-not-a-member", True, None)
assert state == "not-a-member-2fa-required"
assert "cause and its motive" in detail
assert "indistinguishable" in detail
def test_an_unreadable_motive_is_still_a_finding():
state, detail = combine("requester-not-a-member", None, None)
assert state == "not-a-member-motive-unreadable"
assert "losing that access is what this finding is" in detail
def test_a_removal_with_no_requirement_is_sent_to_the_audit_log():
state, detail = combine("not-a-member", False, None)
assert state == "not-a-member-no-requirement"
assert "audit log" in detail
def test_a_member_with_2fa_off_is_flagged_before_anything_breaks():
state, detail = combine("member", True, False)
assert state == "member-at-risk"
assert "has not happened yet" in detail
def test_a_compliant_member_is_sent_somewhere_else():
assert combine("member", True, True)[0] == "member-compliant"
assert combine("member", True, None)[0] == "member-compliance-unreadable"
assert combine("member", False, True)[0] == "member-no-requirement"
assert combine("membership-unreadable", True, True)[0] == "membership-unreadable"
def test_the_symptom_is_404_and_not_403():
text = symptom("not-a-member-2fa-required")
assert "404, not 403" in text
assert "Public repositories keep answering" in text
assert "nothing yet" in symptom("member-at-risk")
def test_a_healthy_token_is_stated_so_the_search_can_move_on():
state, detail = token_health(200)
assert state == "healthy"
assert "end that search early" in detail
assert token_health(401)[0] == "rejected"
def test_user_orgs_is_corroboration_and_matches_case_insensitively():
orgs = [{"login": "ACME"}, {"login": "other"}]
assert listed_in_orgs(orgs, "acme") is True
assert listed_in_orgs(orgs, "missing") is False
assert listed_in_orgs([], "acme") is False
def test_the_repair_offers_the_change_that_cannot_happen_again():
fix = repair("not-a-member-2fa-required", "acme", "octobot")
assert "octobot" in fix and "acme" in fix
assert "GitHub App installation" in fix
assert "re-invites anybody" in fix
assert "before the requirement is enforced" in repair(
"member-at-risk", "acme", "octobot")
def test_the_run_costs_four_reads():
assert read_cost() == 4
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
combine, listedInOrgs, membershipState, ownTwoFactor, questionAnswered,
readCost, repair, requirementState, symptom, tokenHealth,
} from './github-org-membership-lost.mjs';
test('the redirect is the finding and not an error', () => {
const [state, detail] = membershipState(302);
assert.equal(state, 'requester-not-a-member');
assert.match(detail, /that is the removal/);
assert.equal(membershipState(204)[0], 'member');
assert.equal(membershipState(404)[0], 'not-a-member');
assert.equal(membershipState(418)[0], 'unclear');
});
test('following the redirect answers a different question', () => {
const [state, detail] = questionAnswered(true);
assert.equal(state, 'public-membership-instead');
assert.match(detail, /publicly listed/);
assert.equal(questionAnswered(false)[0], 'membership');
});
test('an absent requirement field is unreadable not false', () => {
assert.equal(requirementState({}), null);
assert.equal(requirementState({ two_factor_requirement_enabled: null }), null);
assert.equal(requirementState({ two_factor_requirement_enabled: true }), true);
assert.equal(requirementState({ two_factor_requirement_enabled: false }), false);
});
test('an absent two factor field does not invent a violation', () => {
assert.equal(ownTwoFactor({ login: 'octobot' }), null);
assert.equal(ownTwoFactor({ two_factor_authentication: false }), false);
assert.equal(ownTwoFactor({ two_factor_authentication: true }), true);
assert.equal(ownTwoFactor(null), null);
});
test('the removal and its motive are reported together', () => {
const [state, detail] = combine('requester-not-a-member', true, null);
assert.equal(state, 'not-a-member-2fa-required');
assert.match(detail, /cause and its motive/);
assert.match(detail, /indistinguishable/);
});
test('an unreadable motive is still a finding', () => {
const [state, detail] = combine('requester-not-a-member', null, null);
assert.equal(state, 'not-a-member-motive-unreadable');
assert.match(detail, /losing that access is what this finding is/);
});
test('a removal with no requirement is sent to the audit log', () => {
const [state, detail] = combine('not-a-member', false, null);
assert.equal(state, 'not-a-member-no-requirement');
assert.match(detail, /audit log/);
});
test('a member with 2fa off is flagged before anything breaks', () => {
const [state, detail] = combine('member', true, false);
assert.equal(state, 'member-at-risk');
assert.match(detail, /has not happened yet/);
});
test('a compliant member is sent somewhere else', () => {
assert.equal(combine('member', true, true)[0], 'member-compliant');
assert.equal(combine('member', true, null)[0], 'member-compliance-unreadable');
assert.equal(combine('member', false, true)[0], 'member-no-requirement');
assert.equal(combine('membership-unreadable', true, true)[0], 'membership-unreadable');
});
test('the symptom is 404 and not 403', () => {
const text = symptom('not-a-member-2fa-required');
assert.match(text, /404, not 403/);
assert.match(text, /Public repositories keep answering/);
assert.match(symptom('member-at-risk'), /nothing yet/);
});
test('a healthy token is stated so the search can move on', () => {
const [state, detail] = tokenHealth(200);
assert.equal(state, 'healthy');
assert.match(detail, /end that search early/);
assert.equal(tokenHealth(401)[0], 'rejected');
});
test('user orgs is corroboration and matches case insensitively', () => {
const orgs = [{ login: 'ACME' }, { login: 'other' }];
assert.equal(listedInOrgs(orgs, 'acme'), true);
assert.equal(listedInOrgs(orgs, 'missing'), false);
assert.equal(listedInOrgs([], 'acme'), false);
});
test('the repair offers the change that cannot happen again', () => {
const fix = repair('not-a-member-2fa-required', 'acme', 'octobot');
assert.match(fix, /octobot/);
assert.match(fix, /GitHub App installation/);
assert.match(fix, /re-invites anybody/);
assert.match(repair('member-at-risk', 'acme', 'octobot'),
/before the requirement is enforced/);
});
test('the run costs four reads', () => {
assert.equal(readCost(), 4);
});
FAQ
Why would a policy remove accounts rather than just refuse them?
Because a two-factor requirement is a statement about who is in the organization, not about what a request may do. GitHub applies it to the membership graph: when it is enabled, members and outside collaborators without 2FA are removed. That is documented behaviour and it is the reason the symptom is so strange from the inside — there is no refusal to read, no rule named in a body, no header. The account's relationship to the organization simply ended, and everything downstream of that is the ordinary behaviour of a caller with no access.
Everyone else's tokens still work. Does that not rule out an org-wide change?
It rules out an org-wide <em>outage</em>, which is not the same thing. The requirement removes only the accounts that do not comply, so colleagues who have had 2FA on for years see nothing at all. That asymmetry is the signature: humans unaffected, machine account gone. It is also why the notification did not reach anybody who would act on it, since the announcement goes to people, and the account that broke does not read email.
Why does the members endpoint answer with a redirect at all?
Because the question changes depending on who is asking. To a member, <code>GET /orgs/{org}/members/{username}</code> answers about membership: 204 yes, 404 no. To a non-member it redirects to the public-members endpoint, which is the version of the question a stranger is allowed to ask. Once you have been removed you are the stranger, so the redirect is the answer. Send the call with redirect-following disabled, or your HTTP library will quietly go and answer the stranger's question and hand you back a status about public listing.
The organization object does not include two_factor_requirement_enabled. Is the requirement off?
No, and the script deliberately does not say it is. That field is returned to callers with organization access, and losing organization access is exactly the event being investigated, so the field is often missing <em>because</em> of the thing you are trying to confirm. Absence is reported as unreadable. If you need the motive on the record, ask an owner: they can also read the audit log entry for the removal, which a read-only token cannot see at all.
What stops this happening again after we re-invite the account?
Enabling 2FA on the machine account stops this particular rule from removing it again, but the underlying fragility remains: a personal account acting as a robot is subject to every policy that governs people, and there are more of those every year. The durable repair is a GitHub App installation. An installation is not a member of the organization, so member policy does not apply to it, its permissions are declared rather than inherited, and its tokens are short-lived by design.
Related field notes
- Why the symptom arrives as 404 rather than 403
- When the account is present but the role is too low
- The App equivalent: the record survives and the capability does not
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Organization members — GitHub REST API
- Requiring two-factor authentication in your organization — GitHub Docs
- Organizations — GitHub REST API
- Users — GitHub REST API
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.