Skip to content

Diagnostic GitHub API

a 404 that means the App is not installed on that repo

The repository is public. You can open it in a browser without logging in. The App reads eleven other repositories in the same organization with the same token, and this one comes back 404 Not Found. So the search starts with permissions, and permissions are not the problem, because the App was never installed here at all.

Read-only token Python and Node.js Tests included
Photo of library interior
Photo by Cameron Cress on Unsplash
The short answer

An installation access token can only see the repositories inside its installation. Anything else is not forbidden, it is invisible: GitHub answers 404 rather than 403 so that a caller cannot use error codes to enumerate resources it has no business knowing about. A public repository looks exactly like a private one it cannot reach, and both look exactly like one that does not exist.

Ask the question that has a yes-or-no answer. GET /repos/{owner}/{repo}/installation, sent with the App's JWT rather than an installation token, returns the installation object when your App is installed on that repository and 404 when it is not. Then ask the same about the account with GET /orgs/{org}/installation. Installed on the account but not the repository is one repair; not installed on the account at all is a different one.

The problem in plain words

404 is the least informative answer GitHub has, and it is the correct one. Returning 403 would confirm the repository exists, which is a small leak that matters a great deal for private repositories, so the API declines to distinguish you may not from there is nothing here. The cost is paid by everyone debugging the honest case, where the repository plainly exists and the code still cannot see it.

The presence of other working repositories makes it worse. Eleven successes and one failure looks like a property of the twelfth repository — archived, renamed, moved, private in some new way — rather than a property of the installation. So the investigation goes into the repository's settings, which are fine, instead of into the App's installation, which is where the missing entry is.

And the commonest cause is invisible by design. An installation set to selected repositories does not gain new ones. A repository created this morning is not in an installation configured last year, so an integration that has worked for eleven months starts 404ing on exactly the repositories that are newest and most active, and nobody changed anything.

New repocreatedthe installationis unchanged404 from theAppon a publicrepositoryRepo settingsauditedarchived, renamed,privatePermissionswidenedinstallers askedto acceptSame 404returnsand the next newrepo tooso a broader grant is tried next
Eleven repositories work, so the twelfth looks like the odd one. The thing that differs is not in the repository's settings.

Why it happens

The installation is the boundary, not the permission. Permissions say what an App may do with a repository it can reach. The installation says which repositories those are. A missing permission produces a 403 that names what it wanted; a missing installation produces a 404 that names nothing, because as far as that credential is concerned there is no resource to talk about.

There is an endpoint that answers this directly. GET /repos/{owner}/{repo}/installation is asked with the App's JWT and returns the installation covering that repository, or 404. It is the only call in this note that gives a yes or a no rather than an inference, which is why it is the one the script leans on.

Account and repository are two separate questions. An App can be installed on an organization and still not cover a given repository, so the account-level route has to be asked too. Installed on the account is a good state to be in: the repair is adding one repository. Not installed at all means somebody has to accept an installation, which is a different conversation with a different person.

Creation dates turn a guess into a sentence. When the selection is selected and the repository was created after the installation, the cause is not a mistake anybody made, it is the default behaviour of a selection that does not grow. Comparing the two timestamps lets the report say that instead of listing possibilities, and it points at the durable repair rather than the immediate one.

What this cannot tell you. If the unauthenticated read also 404s, the repository is either private or absent and no read-only check can separate those two: that is the same masking, one level out. And the size of the gap between what the installation covers and what the account holds is a different measurement, kept in the note on partial repository selection. This note is about one repository and whether the App is there at all.

The fix, as a flow

Three GETs and the first one carries no credential, because whether the repository is publicly readable is a fact about the world rather than about your App. The other two are the same presence question asked at two scopes, and the pair of answers is what splits one unhelpful 404 into two repairs that involve different people.

Presence at both scopesasked with the App JWTOn the account, not hereadd it, or select them allNot on the accountsomebody has to install itNewer than the installit will happen again next timeInstalled on this repothe 404 is something else
Two different repairs and two different people, from a pair of status codes that the failing call could never have given you.

How to fix it

Confirm the repository exists without any credential

An unauthenticated GET /repos/{owner}/{repo} settles whether there is a public repository at that path. A 200 makes the rest of the investigation about your App, which is a much smaller search. A 404 leaves two possibilities that no read-only check can separate, and the report should say so rather than picking one.

Ask the per-repository question with the App JWT

GET /repos/{owner}/{repo}/installation is the direct question and it is answered with the App's JWT, not with an installation token. A 200 returns the installation, its id, its repository_selection and the account it sits on. A 404 is the finding: your App is not installed on that repository.

Ask the same about the account

GET /orgs/{org}/installation, or /users/{username}/installation for a personal account, says whether the App is installed anywhere on that owner. This is the step that turns one 404 into two different repairs, and it takes one request.

Compare the creation dates when the selection is partial

If the installation covers selected repositories and the repository was created after the installation, you have found a recurring problem rather than a one-off. It will happen again with the next repository, and again after that, and adding this one by hand does nothing about the next.

Add the repository, or switch the installation to all repositories

Adding one repository is done from the installation's configuration page and fixes today. Switching the installation to all repositories fixes every future one, at the cost of a broader grant that somebody has to be comfortable with. Which of those is right is a policy question; the script prints both and names the one that matches what it found.

How to check it worked

Re-run after the repository is added to the installation. The state moves from installed-on-account-not-repo to installed-on-this-repo, with no permission changed and no token re-minted.

python3 github_app_installation_presence.py --repo acme/reporting
# public-repo: acme/reporting exists and is publicly readable
# GET /repos/acme/reporting/installation returned 200
# installed-on-this-repo: installation 42891842 covers this repository.
# selection-covers-everything: repository_selection is all, so new
# repositories are covered automatically.

The full code

Three GETs, and the first one deliberately carries no credential at all: whether the repository is publicly readable is a fact about the world rather than about your App, and separating it out keeps the rest of the report honest. The other two are the same question asked at two scopes, repository and account, under the App's JWT. Everything that turns those three status codes into a verdict is pure, including the repository-reference parser, which has to reduce a URL, an owner/name pair and a clone path to the same two strings, and the creation-date comparison that names a recurring cause rather than a one-off.

Get this script on GitHub Follow @allanninal Python and Node.js, with tests. Read only, it never writes. One of 101 GitHub API fixes, free and open source.
github_app_installation_presence.py
"""Say whether a GitHub App is installed on one specific repository.

Read only. Three GETs: one unauthenticated existence check, and two presence
questions asked with the App's JWT, at repository scope and at account scope.
Nothing is installed, added, changed or minted.

An installation access token cannot see outside its installation, and GitHub
answers 404 rather than 403 for anything outside it, so a public repository the
App was never installed on is indistinguishable from one that does not exist.
GET /repos/{owner}/{repo}/installation is the call that answers directly.

The JWT is read from the environment and never printed. The output is three
status codes, two timestamps and a verdict.
"""
import argparse
import json
import logging
import os
import re
import sys
from datetime import datetime, timezone

import requests

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("github_app_installation_presence")

API = "https://api.github.com"
UA = "github-app-installation-presence/1.0"

# Owner and repository names GitHub will actually issue.
NAME = re.compile(r"[A-Za-z0-9._-]+")

# Every way a repository reference tends to arrive in a bug report.
PREFIXES = ("https://github.com/", "http://github.com/",
            "https://api.github.com/repos/", "git@github.com:")


def split_repo(value):
    """Reduce any repository reference to (owner, name). Pure.

    A browser URL, an API URL, a clone path and a plain owner/name pair all
    describe the same repository, and a bug report will contain whichever one
    the reporter had on their clipboard. Returns None when it is not a
    repository reference at all.
    """
    text = str(value or "").strip().rstrip("/")
    for prefix in PREFIXES:
        if text.startswith(prefix):
            text = text[len(prefix):]
            break
    if text.endswith(".git"):
        text = text[:-4]
    parts = [p for p in text.split("/") if p]
    if len(parts) < 2:
        return None
    owner, name = parts[0], parts[1]
    if not NAME.fullmatch(owner) or not NAME.fullmatch(name):
        return None
    return (owner, name)


def account_route(owner, owner_type):
    """The account-scope installation route for this kind of owner. Pure.

    Organizations and user accounts have separate routes and the wrong one
    404s for reasons that have nothing to do with the App, which would be a
    very annoying way to get a false finding.
    """
    if str(owner_type or "").lower() == "user":
        return "/users/%s/installation" % owner
    return "/orgs/%s/installation" % owner


def parse_iso(value):
    """Parse an ISO-8601 timestamp into epoch seconds. Pure. None if unusable."""
    text = str(value or "").strip()
    if not text:
        return None
    try:
        moment = datetime.fromisoformat(text.replace("Z", "+00:00"))
    except ValueError:
        return None
    if moment.tzinfo is None:
        moment = moment.replace(tzinfo=timezone.utc)
    return moment.timestamp()


def visibility(status):
    """What the unauthenticated read proved about the repository. Pure."""
    if status == 200:
        return ("public-repo",
                "the repository exists and is publicly readable, so whatever "
                "is wrong is on your side of the request.")
    if status == 404:
        return ("not-public-or-absent",
                "an unauthenticated read also returns 404, which means the "
                "repository is private or does not exist. A read-only check "
                "cannot separate those two, and neither can anyone else "
                "without access.")
    return ("visibility-unknown",
            "the unauthenticated read returned something other than 200 or "
            "404, so nothing was established about the repository itself.")


def classify(repo_status, account_status):
    """Turn two presence questions into one verdict. Pure.

    Repository scope is asked first because a 200 there ends the matter. The
    account-scope answer only matters when the repository-scope answer is no,
    and it is what splits one 404 into two different repairs.
    """
    if repo_status in (401, 403) or account_status in (401, 403):
        return ("jwt-not-accepted",
                "the App JWT was refused, so nothing was learned about any "
                "installation. Fix the JWT first; a signing or clock fault "
                "looks like an absent installation from here.")
    if repo_status == 200:
        return ("installed-on-this-repo",
                "the App is installed on this repository, so a 404 from your "
                "integration is about something else: a permission, a wrong "
                "path, or a credential that is not this App's.")
    if repo_status == 404 and account_status == 200:
        return ("installed-on-account-not-repo",
                "the App is installed on the account and this repository is "
                "not in the installation. The installation is set to selected "
                "repositories and this one was never selected.")
    if repo_status == 404 and account_status == 404:
        return ("not-installed-on-account",
                "the App is not installed anywhere on this account. Somebody "
                "with admin rights on the account has to install it; no "
                "permission or token change will do anything until they do.")
    return ("inconclusive",
            "the two presence checks did not return a pair this check "
            "recognises, so no verdict is safe.")


def creation_order(repo_created, installation_created, selection):
    """Say whether the repository is simply newer than the installation. Pure.

    This is the difference between a mistake and a recurring condition. A
    selection that does not grow will keep producing this finding for every
    repository created after it, and adding this one by hand fixes only today.
    """
    if str(selection or "").lower() == "all":
        return ("selection-covers-everything",
                "repository_selection is all, so new repositories are covered "
                "automatically and creation order is irrelevant.")
    if not selection:
        return ("selection-unknown",
                "no repository_selection was returned, so nothing can be said "
                "about how the installation grows.")
    if repo_created is None or installation_created is None:
        return ("creation-order-unknown",
                "one of the two creation dates is missing, so the order "
                "cannot be established.")
    if repo_created > installation_created:
        days = int((repo_created - installation_created) // 86400)
        return ("repo-created-after-installation",
                "this repository was created %d day(s) after the installation, "
                "and a selected-repositories installation does not gain new "
                "ones. Every repository created from now on will land in the "
                "same state." % days)
    return ("repo-predates-installation",
            "the repository already existed when the installation was "
            "configured, so it was left out deliberately or by oversight "
            "rather than by the passage of time.")


def repair_for(state, selection):
    """The sentence worth printing under a verdict. Pure."""
    if state == "installed-on-account-not-repo":
        if str(selection or "").lower() == "selected":
            return ("add this repository to the installation, or switch the "
                    "installation to all repositories so future ones are "
                    "covered without anybody remembering to.")
        return ("open the installation's configuration and add this "
                "repository to it.")
    if state == "not-installed-on-account":
        return ("install the App on this account. This needs somebody with "
                "admin rights on the account, and it is not something a token "
                "change can substitute for.")
    if state == "installed-on-this-repo":
        return ("nothing to repair here. If calls still fail, read the status "
                "code and the message rather than assuming coverage.")
    if state == "jwt-not-accepted":
        return ("fix the App JWT before reading anything above; an unusable "
                "JWT and an absent installation look the same from here.")
    return "no repair applies to this state."


def get(path, token=None):
    """One GET. Returns (status, body). The only network in this script."""
    headers = {"Accept": "application/vnd.github+json",
               "X-GitHub-Api-Version": "2022-11-28", "User-Agent": UA}
    if token:
        headers["Authorization"] = "Bearer " + token
    r = requests.get(API + path, timeout=30, headers=headers)
    try:
        body = r.json()
    except ValueError:
        body = None
    return r.status_code, body


def main():
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("--repo", required=True,
                    help="the repository, as owner/name or as any GitHub URL "
                         "for it")
    args = ap.parse_args()

    target = split_repo(args.repo)
    if target is None:
        log.error("could not read %r as a repository reference", args.repo)
        return 2
    owner, name = target

    jwt = os.environ.get("GITHUB_APP_JWT")
    if not jwt:
        log.error("set GITHUB_APP_JWT to the App JWT. The per-repository "
                  "installation route is answered with the JWT, not with an "
                  "installation access token")
        return 2

    # No credential on this one on purpose: whether the repository is publicly
    # readable is a fact about the world, not about the App.
    public_status, public_body = get("/repos/%s/%s" % (owner, name))
    vis_state, vis_detail = visibility(public_status)
    log.info("%s: %s", vis_state, vis_detail)
    owner_type = None
    repo_created = None
    if isinstance(public_body, dict):
        owner_type = (public_body.get("owner") or {}).get("type")
        repo_created = parse_iso(public_body.get("created_at"))

    repo_status, repo_body = get("/repos/%s/%s/installation" % (owner, name), jwt)
    log.info("GET /repos/%s/%s/installation returned %d", owner, name, repo_status)

    route = account_route(owner, owner_type)
    account_status, account_body = get(route, jwt)
    log.info("GET %s returned %d", route, account_status)

    state, detail = classify(repo_status, account_status)
    log.info("%s: %s", state, detail)

    installation = repo_body if repo_status == 200 else account_body
    selection = None
    installation_created = None
    installation_id = None
    if isinstance(installation, dict):
        selection = installation.get("repository_selection")
        installation_created = parse_iso(installation.get("created_at"))
        installation_id = installation.get("id")
    if installation_id is not None:
        log.info("installation %s, repository_selection=%s",
                 installation_id, selection or "unknown")

    order_state, order_detail = creation_order(repo_created,
                                               installation_created, selection)
    log.info("%s: %s", order_state, order_detail)
    log.info("repair: %s", repair_for(state, selection))

    print(json.dumps({"owner": owner, "repo": name,
                      "public_status": public_status,
                      "repo_installation_status": repo_status,
                      "account_installation_status": account_status,
                      "account_route": route,
                      "repository_selection": selection,
                      "installation_id": installation_id,
                      "visibility": vis_state, "order": order_state,
                      "state": state}, indent=2))
    return 0 if state == "installed-on-this-repo" else 1


if __name__ == "__main__":
    sys.exit(main())
github-app-installation-presence.mjs
/**
 * Say whether a GitHub App is installed on one specific repository.
 *
 * Read only. Three GETs: one unauthenticated existence check, and two
 * presence questions asked with the App's JWT, at repository scope and at
 * account scope. Nothing is installed, added, changed or minted.
 *
 * An installation access token cannot see outside its installation, and
 * GitHub answers 404 rather than 403 for anything outside it, so a public
 * repository the App was never installed on is indistinguishable from one
 * that does not exist. GET /repos/{owner}/{repo}/installation answers
 * directly.
 *
 * The JWT is read from the environment and never printed.
 */
const API = 'https://api.github.com';
const UA = 'github-app-installation-presence/1.0';

/** Owner and repository names GitHub will actually issue. */
const NAME = /^[A-Za-z0-9._-]+$/;

/** Every way a repository reference tends to arrive in a bug report. */
const PREFIXES = ['https://github.com/', 'http://github.com/',
  'https://api.github.com/repos/', 'git@github.com:'];

/**
 * Reduce any repository reference to [owner, name]. Pure.
 * null when it is not a repository reference at all.
 */
export function splitRepo(value) {
  let text = String(value ?? '').trim().replace(/\/+$/, '');
  for (const prefix of PREFIXES) {
    if (text.startsWith(prefix)) {
      text = text.slice(prefix.length);
      break;
    }
  }
  if (text.endsWith('.git')) text = text.slice(0, -4);
  const parts = text.split('/').filter(Boolean);
  if (parts.length < 2) return null;
  const [owner, name] = parts;
  if (!NAME.test(owner) || !NAME.test(name)) return null;
  return [owner, name];
}

/**
 * The account-scope installation route for this kind of owner. Pure.
 * The wrong one 404s for reasons that have nothing to do with the App.
 */
export function accountRoute(owner, ownerType) {
  if (String(ownerType ?? '').toLowerCase() === 'user') {
    return `/users/${owner}/installation`;
  }
  return `/orgs/${owner}/installation`;
}

/** Parse an ISO-8601 timestamp into epoch seconds. Pure. null if unusable. */
export function parseIso(value) {
  const text = String(value ?? '').trim();
  if (!text) return null;
  const ms = Date.parse(text);
  return Number.isNaN(ms) ? null : ms / 1000;
}

/** What the unauthenticated read proved about the repository. Pure. */
export function visibility(status) {
  if (status === 200) {
    return ['public-repo',
      'the repository exists and is publicly readable, so whatever is wrong ' +
      'is on your side of the request.'];
  }
  if (status === 404) {
    return ['not-public-or-absent',
      'an unauthenticated read also returns 404, which means the repository ' +
      'is private or does not exist. A read-only check cannot separate those ' +
      'two, and neither can anyone else without access.'];
  }
  return ['visibility-unknown',
    'the unauthenticated read returned something other than 200 or 404, so ' +
    'nothing was established about the repository itself.'];
}

/**
 * Turn two presence questions into one verdict. Pure.
 * Repository scope first: a 200 there ends the matter. The account answer is
 * what splits one 404 into two different repairs.
 */
export function classify(repoStatus, accountStatus) {
  if ([401, 403].includes(repoStatus) || [401, 403].includes(accountStatus)) {
    return ['jwt-not-accepted',
      'the App JWT was refused, so nothing was learned about any ' +
      'installation. Fix the JWT first; a signing or clock fault looks like ' +
      'an absent installation from here.'];
  }
  if (repoStatus === 200) {
    return ['installed-on-this-repo',
      'the App is installed on this repository, so a 404 from your ' +
      'integration is about something else: a permission, a wrong path, or a ' +
      'credential that is not this App\'s.'];
  }
  if (repoStatus === 404 && accountStatus === 200) {
    return ['installed-on-account-not-repo',
      'the App is installed on the account and this repository is not in the ' +
      'installation. The installation is set to selected repositories and ' +
      'this one was never selected.'];
  }
  if (repoStatus === 404 && accountStatus === 404) {
    return ['not-installed-on-account',
      'the App is not installed anywhere on this account. Somebody with ' +
      'admin rights on the account has to install it; no permission or token ' +
      'change will do anything until they do.'];
  }
  return ['inconclusive',
    'the two presence checks did not return a pair this check recognises, so ' +
    'no verdict is safe.'];
}

/**
 * Say whether the repository is simply newer than the installation. Pure.
 * This is the difference between a mistake and a recurring condition.
 */
export function creationOrder(repoCreated, installationCreated, selection) {
  if (String(selection ?? '').toLowerCase() === 'all') {
    return ['selection-covers-everything',
      'repository_selection is all, so new repositories are covered ' +
      'automatically and creation order is irrelevant.'];
  }
  if (!selection) {
    return ['selection-unknown',
      'no repository_selection was returned, so nothing can be said about ' +
      'how the installation grows.'];
  }
  if (repoCreated === null || repoCreated === undefined
      || installationCreated === null || installationCreated === undefined) {
    return ['creation-order-unknown',
      'one of the two creation dates is missing, so the order cannot be ' +
      'established.'];
  }
  if (repoCreated > installationCreated) {
    const days = Math.trunc((repoCreated - installationCreated) / 86400);
    return ['repo-created-after-installation',
      `this repository was created ${days} day(s) after the installation, and ` +
      'a selected-repositories installation does not gain new ones. Every ' +
      'repository created from now on will land in the same state.'];
  }
  return ['repo-predates-installation',
    'the repository already existed when the installation was configured, so ' +
    'it was left out deliberately or by oversight rather than by the passage ' +
    'of time.'];
}

/** The sentence worth printing under a verdict. Pure. */
export function repairFor(state, selection) {
  if (state === 'installed-on-account-not-repo') {
    if (String(selection ?? '').toLowerCase() === 'selected') {
      return 'add this repository to the installation, or switch the ' +
        'installation to all repositories so future ones are covered without ' +
        'anybody remembering to.';
    }
    return 'open the installation\'s configuration and add this repository to it.';
  }
  if (state === 'not-installed-on-account') {
    return 'install the App on this account. This needs somebody with admin ' +
      'rights on the account, and it is not something a token change can ' +
      'substitute for.';
  }
  if (state === 'installed-on-this-repo') {
    return 'nothing to repair here. If calls still fail, read the status code ' +
      'and the message rather than assuming coverage.';
  }
  if (state === 'jwt-not-accepted') {
    return 'fix the App JWT before reading anything above; an unusable JWT ' +
      'and an absent installation look the same from here.';
  }
  return 'no repair applies to this state.';
}

async function get(path, token) {
  const headers = {
    Accept: 'application/vnd.github+json',
    'X-GitHub-Api-Version': '2022-11-28',
    'User-Agent': UA,
  };
  if (token) headers.Authorization = `Bearer ${token}`;
  const res = await fetch(API + path, { headers });
  let body = null;
  try { body = await res.json(); } catch { body = null; }
  return [res.status, body];
}

function flag(name) {
  const at = process.argv.indexOf(name);
  return (at === -1 || at === process.argv.length - 1) ? null : process.argv[at + 1];
}

async function main() {
  const target = splitRepo(flag('--repo'));
  if (!target) {
    console.error('pass --repo owner/name, or any GitHub URL for the repository');
    process.exitCode = 2;
    return;
  }
  const [owner, name] = target;

  const jwt = process.env.GITHUB_APP_JWT;
  if (!jwt) {
    console.error('set GITHUB_APP_JWT to the App JWT. The per-repository ' +
      'installation route is answered with the JWT, not with an installation ' +
      'access token');
    process.exitCode = 2;
    return;
  }

  // No credential on this one on purpose: whether the repository is publicly
  // readable is a fact about the world, not about the App.
  const [publicStatus, publicBody] = await get(`/repos/${owner}/${name}`);
  const [visState, visDetail] = visibility(publicStatus);
  console.log(`${visState}: ${visDetail}`);
  const ownerType = publicBody && publicBody.owner ? publicBody.owner.type : null;
  const repoCreated = publicBody ? parseIso(publicBody.created_at) : null;

  const [repoStatus, repoBody] = await get(`/repos/${owner}/${name}/installation`, jwt);
  console.log(`GET /repos/${owner}/${name}/installation returned ${repoStatus}`);

  const route = accountRoute(owner, ownerType);
  const [accountStatus, accountBody] = await get(route, jwt);
  console.log(`GET ${route} returned ${accountStatus}`);

  const [state, detail] = classify(repoStatus, accountStatus);
  console.log(`${state}: ${detail}`);

  const installation = repoStatus === 200 ? repoBody : accountBody;
  const selection = installation ? installation.repository_selection : null;
  const installationCreated = installation ? parseIso(installation.created_at) : null;
  const installationId = installation ? installation.id : null;
  if (installationId !== null && installationId !== undefined) {
    console.log(`installation ${installationId}, ` +
      `repository_selection=${selection ?? 'unknown'}`);
  }

  const [orderState, orderDetail] = creationOrder(repoCreated, installationCreated, selection);
  console.log(`${orderState}: ${orderDetail}`);
  console.log(`repair: ${repairFor(state, selection)}`);

  console.log(JSON.stringify({
    owner,
    repo: name,
    public_status: publicStatus,
    repo_installation_status: repoStatus,
    account_installation_status: accountStatus,
    account_route: route,
    repository_selection: selection ?? null,
    installation_id: installationId ?? null,
    visibility: visState,
    order: orderState,
    state,
  }, null, 2));
  process.exitCode = state === 'installed-on-this-repo' ? 0 : 1;
}

// Only run when invoked directly, so importing this module from the test file
// does not execute main() and set an exit code that fails a passing suite.
if (import.meta.url === `file://${process.argv[1]}`) {
  main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}

Add a test

There is nothing to fake here but two status codes, which is the point: the whole verdict is a lookup over the pair of answers the two presence routes gave, so every case is one line. The reference parser gets its own group of tests because a bug report will contain a browser URL as often as an owner/name pair, and a parser that quietly disagreed with itself would produce a confident 404 about a repository nobody asked about. The creation-date comparison is tested for the case that matters most: a repository younger than the installation that will never be covered by it.

test_github_app_installation_presence.py
from github_app_installation_presence import (
    account_route, classify, creation_order, parse_iso, repair_for, split_repo,
    visibility,
)

JAN = parse_iso("2026-01-01T00:00:00Z")
MAR = parse_iso("2026-03-02T00:00:00Z")


def test_every_shape_of_repository_reference_lands_on_the_same_pair():
    assert split_repo("acme/reporting") == ("acme", "reporting")
    assert split_repo("https://github.com/acme/reporting") == ("acme", "reporting")
    assert split_repo("https://github.com/acme/reporting/") == ("acme", "reporting")
    assert split_repo("https://github.com/acme/reporting/pulls/12") == ("acme", "reporting")
    assert split_repo("https://api.github.com/repos/acme/reporting") == ("acme", "reporting")
    assert split_repo("git@github.com:acme/reporting.git") == ("acme", "reporting")


def test_something_that_is_not_a_repository_reference_is_refused():
    assert split_repo("acme") is None
    assert split_repo("") is None
    assert split_repo(None) is None
    assert split_repo("acme/repo with spaces") is None


def test_organizations_and_user_accounts_have_different_routes():
    assert account_route("acme", "Organization") == "/orgs/acme/installation"
    assert account_route("octocat", "User") == "/users/octocat/installation"
    assert account_route("acme", None) == "/orgs/acme/installation"


def test_a_public_repository_narrows_the_search_to_your_own_app():
    state, detail = visibility(200)
    assert state == "public-repo"
    assert "your side of the request" in detail


def test_a_404_without_a_credential_is_two_answers_and_says_so():
    state, detail = visibility(404)
    assert state == "not-public-or-absent"
    assert "cannot separate those two" in detail
    assert visibility(500)[0] == "visibility-unknown"


def test_installed_on_the_account_but_not_the_repository_is_the_headline():
    state, detail = classify(404, 200)
    assert state == "installed-on-account-not-repo"
    assert "never selected" in detail
    assert "add this repository" in repair_for(state, "selected")


def test_not_installed_at_all_is_a_different_repair_and_a_different_person():
    state, detail = classify(404, 404)
    assert state == "not-installed-on-account"
    assert "admin rights" in repair_for(state, None)


def test_installed_here_means_the_404_came_from_somewhere_else():
    state, detail = classify(200, 200)
    assert state == "installed-on-this-repo"
    assert "about something else" in detail


def test_a_refused_jwt_is_not_reported_as_an_absent_installation():
    assert classify(401, 404)[0] == "jwt-not-accepted"
    assert classify(404, 403)[0] == "jwt-not-accepted"
    assert "Fix the JWT first" in classify(401, 404)[1]


def test_an_unrecognised_pair_gets_no_verdict():
    assert classify(500, 200)[0] == "inconclusive"


def test_a_repository_newer_than_the_installation_is_a_recurring_cause():
    state, detail = creation_order(MAR, JAN, "selected")
    assert state == "repo-created-after-installation"
    assert "60 day(s) after" in detail
    assert "Every repository created from now on" in detail


def test_a_repository_older_than_the_installation_was_left_out_by_hand():
    assert creation_order(JAN, MAR, "selected")[0] == "repo-predates-installation"


def test_an_installation_covering_everything_makes_the_dates_irrelevant():
    state, detail = creation_order(MAR, JAN, "all")
    assert state == "selection-covers-everything"
    assert "automatically" in detail


def test_missing_inputs_produce_a_named_state_rather_than_a_guess():
    assert creation_order(None, JAN, "selected")[0] == "creation-order-unknown"
    assert creation_order(MAR, None, "selected")[0] == "creation-order-unknown"
    assert creation_order(MAR, JAN, None)[0] == "selection-unknown"


def test_timestamps_that_cannot_be_read_are_none_rather_than_an_exception():
    assert parse_iso("2026-01-01T00:00:00Z") is not None
    assert parse_iso("last thursday") is None
    assert parse_iso("") is None
    assert parse_iso(None) is None
github-app-installation-presence.test.mjs
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
  accountRoute, classify, creationOrder, parseIso, repairFor, splitRepo,
  visibility,
} from './github-app-installation-presence.mjs';

const JAN = parseIso('2026-01-01T00:00:00Z');
const MAR = parseIso('2026-03-02T00:00:00Z');

test('every shape of repository reference lands on the same pair', () => {
  assert.deepEqual(splitRepo('acme/reporting'), ['acme', 'reporting']);
  assert.deepEqual(splitRepo('https://github.com/acme/reporting'), ['acme', 'reporting']);
  assert.deepEqual(splitRepo('https://github.com/acme/reporting/'), ['acme', 'reporting']);
  assert.deepEqual(splitRepo('https://github.com/acme/reporting/pulls/12'), ['acme', 'reporting']);
  assert.deepEqual(splitRepo('https://api.github.com/repos/acme/reporting'), ['acme', 'reporting']);
  assert.deepEqual(splitRepo('git@github.com:acme/reporting.git'), ['acme', 'reporting']);
});

test('something that is not a repository reference is refused', () => {
  assert.equal(splitRepo('acme'), null);
  assert.equal(splitRepo(''), null);
  assert.equal(splitRepo(null), null);
  assert.equal(splitRepo('acme/repo with spaces'), null);
});

test('organizations and user accounts have different routes', () => {
  assert.equal(accountRoute('acme', 'Organization'), '/orgs/acme/installation');
  assert.equal(accountRoute('octocat', 'User'), '/users/octocat/installation');
  assert.equal(accountRoute('acme', null), '/orgs/acme/installation');
});

test('a public repository narrows the search to your own app', () => {
  const [state, detail] = visibility(200);
  assert.equal(state, 'public-repo');
  assert.match(detail, /your side of the request/);
});

test('a 404 without a credential is two answers and says so', () => {
  const [state, detail] = visibility(404);
  assert.equal(state, 'not-public-or-absent');
  assert.match(detail, /cannot separate those two/);
  assert.equal(visibility(500)[0], 'visibility-unknown');
});

test('installed on the account but not the repository is the headline', () => {
  const [state, detail] = classify(404, 200);
  assert.equal(state, 'installed-on-account-not-repo');
  assert.match(detail, /never selected/);
  assert.match(repairFor(state, 'selected'), /add this repository/);
});

test('not installed at all is a different repair and a different person', () => {
  const [state] = classify(404, 404);
  assert.equal(state, 'not-installed-on-account');
  assert.match(repairFor(state, null), /admin rights/);
});

test('installed here means the 404 came from somewhere else', () => {
  const [state, detail] = classify(200, 200);
  assert.equal(state, 'installed-on-this-repo');
  assert.match(detail, /about something else/);
});

test('a refused jwt is not reported as an absent installation', () => {
  assert.equal(classify(401, 404)[0], 'jwt-not-accepted');
  assert.equal(classify(404, 403)[0], 'jwt-not-accepted');
  assert.match(classify(401, 404)[1], /Fix the JWT first/);
});

test('an unrecognised pair gets no verdict', () => {
  assert.equal(classify(500, 200)[0], 'inconclusive');
});

test('a repository newer than the installation is a recurring cause', () => {
  const [state, detail] = creationOrder(MAR, JAN, 'selected');
  assert.equal(state, 'repo-created-after-installation');
  assert.match(detail, /60 day\(s\) after/);
  assert.match(detail, /Every repository created from now on/);
});

test('a repository older than the installation was left out by hand', () => {
  assert.equal(creationOrder(JAN, MAR, 'selected')[0], 'repo-predates-installation');
});

test('an installation covering everything makes the dates irrelevant', () => {
  const [state, detail] = creationOrder(MAR, JAN, 'all');
  assert.equal(state, 'selection-covers-everything');
  assert.match(detail, /automatically/);
});

test('missing inputs produce a named state rather than a guess', () => {
  assert.equal(creationOrder(null, JAN, 'selected')[0], 'creation-order-unknown');
  assert.equal(creationOrder(MAR, null, 'selected')[0], 'creation-order-unknown');
  assert.equal(creationOrder(MAR, JAN, null)[0], 'selection-unknown');
});

test('timestamps that cannot be read are null rather than an exception', () => {
  assert.notEqual(parseIso('2026-01-01T00:00:00Z'), null);
  assert.equal(parseIso('last thursday'), null);
  assert.equal(parseIso(''), null);
  assert.equal(parseIso(null), null);
});

FAQ

Why 404 and not 403?

Because 403 would confirm the repository exists, and for a private repository that is information the caller has not earned. GitHub applies the same rule everywhere rather than case by case, so a public repository outside your installation gets the same answer as a private one you have never heard of. It is the right trade and it is genuinely expensive to debug, which is why the useful move is to stop reading the status code and ask a route whose whole job is to answer the presence question.

The App works on eleven repositories and fails on the twelfth. Surely that is a repository problem?

It almost never is. Eleven successes prove the credential, the App, the permissions and the network, all of which are shared. What is not shared is membership of the installation, and that is per repository. The quickest disproof is one request: GET /repos/{owner}/{repo}/installation under the App JWT returns the installation for any repository the App is installed on, so a 404 there settles it before anybody opens the repository's settings page.

Why does a brand new repository fail when the old ones work?

Because an installation set to selected repositories does not grow. The list was fixed when somebody configured it, and a repository created afterwards is simply not on it. This is the most common cause of the whole failure and the one most likely to recur, since every future repository lands in the same state. Comparing the repository's creation date against the installation's is what turns a shrug into a sentence, and it is why the durable repair is usually switching the installation to all repositories rather than adding one.

Can I check this with the installation access token instead of the JWT?

The per-repository presence route wants the App's JWT, so no. What an installation token can do is list what it reaches, which answers a related but different question: how much of the account the installation covers, in total. That measurement, and the gap between it and the account's real repository count, is its own note. For a single named repository the JWT route is one request and gives a yes or a no rather than a list you then have to search.

The App is installed on the repository and I still get 404. What now?

Then this note is finished and the 404 is masking something else. The usual candidates are a permission the App does not hold on an endpoint that answers 404 rather than 403 for missing permissions, a path built from the wrong owner or a renamed repository, or a credential that is not this App's at all. Triaging a 404 across those causes is a separate check, and it starts by establishing which credential is actually in the header rather than which one you meant to put there.

Related field notes

Sources

Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.

Stuck on a tricky one?

If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.