Diagnostic GitHub API
the integration polls for events a webhook would push
The integration works. It notices new pull requests, it picks up comments, it has never lost anything. It also makes 4,320 requests an hour to do it, notices each of those events an average of thirty seconds after it happened, and would notice nothing at all if the poll ran once a day instead. There is no bug here. There is a design that was never revisited.
GitHub's own guidance is to subscribe rather than poll, and the check is two GETs: GET /repos/{owner}/{repo}/hooks and GET /orgs/{org}/hooks. An empty array, or an array whose events do not include what the loop is looking for, next to a core counter that climbs at a constant rate, is a poller.
Polling costs quota linearly in time; a webhook costs it linearly in activity, which for most repositories is a much smaller number. It also costs latency: a loop with a 60-second interval notices things 30 seconds late on average and 60 seconds late at worst, and it cannot see a change that happened and was undone between two polls. The script below lists what is subscribed, names which polled concerns nothing would push, and prints the gh command that creates the missing hook.
The problem in plain words
This never gets escalated, because nothing fails. The poll is correct, the data is right, and the only symptom is a graph nobody looks at. It becomes visible on the day the same integration is pointed at forty repositories instead of one and the hourly quota stops being theoretical.
The interval is where the argument usually gets stuck. Someone wants faster reactions so the interval drops to ten seconds, which quadruples the cost to halve a latency that is still measured in seconds. Someone else wants to save quota so the interval goes to five minutes, and now the bot answers pull requests two and a half minutes after they open. Both sides are optimising a trade-off that only exists because the events are being pulled instead of pushed.
And there is a correctness edge nobody plans for. A poll sees state, not history. A label added and removed between two polls never happened as far as the loop is concerned; a branch pushed and force-pushed over looks like one event. A webhook delivers both, because it fires on the transition rather than sampling the result.
Why it happens
Cost scales with the clock, not with the work. Six endpoints polled every thirty seconds is 720 requests an hour whether the repository saw four hundred events or none. A webhook on a quiet weekend costs nothing at all. The comparison people skip is that most repositories are quiet most of the time.
Latency is half the interval, on average, and you are paying for it either way. A poll's mean detection delay is the interval divided by two, and its worst case is the whole interval. There is no configuration that makes polling both cheap and prompt, which is the entire reason the push path exists.
A hook that exists is not necessarily a hook that delivers. Every hook object has an active flag, and an inactive one is configuration that does nothing. So is a hook subscribed to the wrong event names, which is a common outcome of copying a hook between repositories. Read events and active together, and treat anything else as absent.
Subscribing to everything is not the fix either. A wildcard subscription delivers every event GitHub has, including ones you do not handle, which turns a quota problem into a receiver problem. Name the events you actually consume.
A slow poll is still worth keeping. Deliveries can fail, and a receiver can be down for an hour. The correct end state is a webhook for promptness and an infrequent poll for reconciliation, which is a different thing from the poll you have now — hourly rather than every thirty seconds, and reconciling rather than detecting.
The fix, as a flow
How often a client polls is invisible from the API, so the script checks the half that is readable: whether any active hook would push what the loop is reading. Then it costs the loop in latency as well as in requests, because the latency is the number that ends the argument.
How to fix it
List the hooks that exist at both levels
GET /repos/{owner}/{repo}/hooks needs admin on the repository, and GET /orgs/{org}/hooks needs org admin. A read-only token that lacks either gets a 403, and that is a blind spot rather than a finding: report it as unknown instead of reporting zero hooks.
Read events and active together
Collect the events array from every hook where active is true. A hook with active: false delivers nothing, and a hook subscribed to push does not help a loop that is polling for issue comments. Keep the inactive ones aside so you can say "there is a hook for this, it is switched off", which is a much faster fix than creating a new one.
Map each polled endpoint to the event that would replace it
Issues to issues, issue comments to issue_comment, pull requests to pull_request, commits to push, releases to release, workflow runs to workflow_run. Anything in your loop with no event on this list is a genuine reason to keep polling; everything else is not.
Cost the loop in requests an hour and in seconds of latency
Requests an hour is endpoints times repositories times 3,600 over the interval. Mean latency is the interval over two. Put both numbers next to the 5,000-an-hour quota, because the second number is usually the one that changes the conversation: the poll is slower and more expensive.
Create the hook, then slow the poll down rather than deleting it
Subscribe to the named events, point the hook at a receiver that verifies X-Hub-Signature-256, and keep a reconciliation poll at a much longer interval — hourly, say — to catch anything a failed delivery lost. That is a safety net at 24 requests a day rather than a detection mechanism at 4,320.
How to check it worked
Re-run the audit once the hook exists. Every polled concern should come back covered, and the report should show the reconciliation interval rather than the detection one.
python3 github_webhook_vs_poll.py --repo acme/api --interval 3600 \
--concerns issues,pulls,commits
# push: every polled concern already has an active hookThe full code
Two list endpoints and a mapping table. The interesting part is what counts as covered: a hook that exists but is switched off, and a hook subscribed to a wildcard, are opposite mistakes and both are easy to get wrong in a one-line check. So coverage() takes hook objects and returns a row per concern with a reason attached, and poll_cost() reports latency alongside requests, because the latency is the argument that actually lands.
"""Decide whether a polling loop should be a webhook, and cost it if it should.
Read only. Two GETs to list hooks, one to read the quota, and the repair is
printed as a command rather than run.
Detecting the client's polling behaviour from the API is a blind spot: nothing
GitHub returns says how often you call it. What is readable is the other half of
the question, which is whether a push path exists at all.
"""
import argparse
import json
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("github_webhook_vs_poll")
API = "https://api.github.com"
UA = "github-webhook-vs-poll/1.0"
HOURLY_LIMIT = 5000
# The polled endpoint on the left, the event that would push the same thing on
# the right. Anything a loop reads that is not on this list is a real reason to
# keep polling, and the report says so rather than pretending otherwise.
CONCERNS = {
"issues": ("GET /repos/{owner}/{repo}/issues", ("issues",)),
"issue_comments": ("GET /repos/{owner}/{repo}/issues/comments", ("issue_comment",)),
"pulls": ("GET /repos/{owner}/{repo}/pulls", ("pull_request",)),
"commits": ("GET /repos/{owner}/{repo}/commits", ("push",)),
"releases": ("GET /repos/{owner}/{repo}/releases", ("release",)),
"workflow_runs": ("GET /repos/{owner}/{repo}/actions/runs", ("workflow_run",)),
}
def subscribed_events(hooks):
"""Split hook subscriptions into what delivers and what does not. Pure.
Inactive hooks are kept separately rather than dropped. "There is a hook for
this and it is switched off" is a thirty-second fix; "there is no hook" is a
different job, and reporting the first as the second wastes the difference.
"""
active, inactive = set(), set()
wildcard = inactive_wildcard = False
for hook in hooks or []:
if not isinstance(hook, dict):
continue
live = hook.get("active") is not False
for event in hook.get("events") or []:
name = str(event)
if live:
active.add(name)
wildcard = wildcard or name == "*"
else:
inactive.add(name)
inactive_wildcard = inactive_wildcard or name == "*"
return {"events": active, "wildcard": wildcard,
"inactive": inactive, "inactive_wildcard": inactive_wildcard}
def coverage(concerns, hooks):
"""One row per polled concern saying whether anything would push it. Pure."""
subs = subscribed_events(hooks)
rows = []
for concern in concerns or []:
wanted = CONCERNS.get(concern, (None, (concern,)))[1]
names = "/".join(wanted)
if subs["wildcard"]:
rows.append({"concern": concern, "state": "covered",
"detail": "a wildcard subscription delivers %s, though "
"it delivers everything else too" % names})
elif any(w in subs["events"] for w in wanted):
rows.append({"concern": concern, "state": "covered",
"detail": "an active hook subscribes to %s" % names})
elif any(w in subs["inactive"] for w in wanted) or subs["inactive_wildcard"]:
rows.append({"concern": concern, "state": "uncovered",
"detail": "a hook subscribes to %s but it is not "
"active, and an inactive hook delivers "
"nothing" % names})
else:
rows.append({"concern": concern, "state": "uncovered",
"detail": "no hook subscribes to %s" % names})
return rows
def poll_cost(concerns, interval_s, repos=1):
"""Requests and detection latency for the loop as configured. Pure.
Latency is reported alongside cost because it is usually the number that
settles the argument: the poll is both slower and more expensive than the
push it replaces.
"""
try:
repos = max(0, int(repos))
except (TypeError, ValueError):
repos = 0
interval = max(1, int(interval_s or 1))
calls = len(concerns or []) * repos
per_hour = round(calls * 3600 / interval)
return {"requests_per_hour": per_hour,
"requests_per_day": per_hour * 24,
"mean_latency_s": interval / 2,
"worst_latency_s": interval}
def verdict(rows, cost, hourly_limit=HOURLY_LIMIT):
"""Turn coverage and cost into one finding. Pure."""
if not rows:
return ("nothing-polled", "no concerns were named, so there is nothing "
"to compare against the hooks")
uncovered = [r for r in rows if r["state"] == "uncovered"]
share = cost.get("requests_per_hour", 0) / max(1, hourly_limit)
if not uncovered:
return ("push",
"every polled concern already has an active hook, so this loop "
"is a reconciliation pass rather than a detection mechanism. "
"Run it on a slow schedule.")
summary = ("%d of %d polled concern(s) have no active hook. The loop costs "
"%d request(s) an hour to notice them %.0fs late on average."
% (len(uncovered), len(rows), cost.get("requests_per_hour", 0),
cost.get("mean_latency_s", 0)))
if share >= 0.5:
return ("polling-dominates",
summary + " That is %.0f%% of the hourly quota spent on the "
"clock rather than on activity." % (share * 100))
return ("polling", summary)
def get(session, path):
"""One GET. Returns (status, parsed-json-or-None)."""
url = API + path if path.startswith("/") else path
r = session.get(url, timeout=30)
try:
return r.status_code, r.json()
except ValueError:
return r.status_code, None
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--repo", required=True, help="owner/name")
ap.add_argument("--org", help="also read the org-level hooks (needs org admin)")
ap.add_argument("--concerns", default="issues,issue_comments,pulls",
help="comma-separated list of what the loop polls for; "
"known names: " + ", ".join(sorted(CONCERNS)))
ap.add_argument("--interval", type=int, default=30,
help="seconds between polls")
ap.add_argument("--repos", type=int, default=1,
help="how many repositories the loop covers")
args = ap.parse_args()
token = os.environ.get("GITHUB_TOKEN")
if not token:
log.error("set GITHUB_TOKEN. Listing hooks needs admin on the "
"repository, but only read access to it")
return 2
session = requests.Session()
session.headers.update({
"Authorization": "Bearer " + token,
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": UA,
})
hooks, blind = [], []
status, body = get(session, "/repos/%s/hooks" % args.repo)
if status == 200 and isinstance(body, list):
hooks.extend(body)
log.info("%s: %d repository hook(s)", args.repo, len(body))
else:
blind.append("repository hooks (%d)" % status)
log.warning("could not read repository hooks: %d. This token cannot see "
"them, which is not the same as there being none.", status)
if args.org:
status, body = get(session, "/orgs/%s/hooks" % args.org)
if status == 200 and isinstance(body, list):
hooks.extend(body)
log.info("%s: %d organisation hook(s)", args.org, len(body))
else:
blind.append("organisation hooks (%d)" % status)
log.warning("could not read organisation hooks: %d", status)
for hook in hooks:
log.info(" hook %s active=%s events=%s", hook.get("id"),
hook.get("active"), ",".join(hook.get("events") or []) or "none")
concerns = [c.strip() for c in args.concerns.split(",") if c.strip()]
unknown = [c for c in concerns if c not in CONCERNS]
for name in unknown:
log.warning("%r is not a concern with a known event; it will be "
"matched against an event of the same name", name)
rows = coverage(concerns, hooks)
cost = poll_cost(concerns, args.interval, args.repos)
for row in rows:
log.info("%-16s %-10s %s", row["concern"], row["state"], row["detail"])
status, payload = get(session, "/rate_limit")
if status == 200:
core = ((payload or {}).get("resources") or {}).get("core") or {}
log.info("core quota: %s used of %s", core.get("used"), core.get("limit"))
state, detail = verdict(rows, cost)
log.info("%s: %s", state, detail)
if blind:
log.warning("unread: %s. Anything reported as uncovered may already be "
"covered by a hook this token cannot see.", "; ".join(blind))
if state in ("polling", "polling-dominates"):
needed = sorted({e for r in rows if r["state"] == "uncovered"
for e in CONCERNS.get(r["concern"], (None, (r["concern"],)))[1]})
log.info("repair: create one hook for the events you consume. This "
"script does not create it:")
log.info(" gh api --method POST /repos/%s/hooks -f name=web "
"-f config[url]=https://example.test/hooks "
"-f config[content_type]=json -f config[secret]=YOURSECRET %s",
args.repo, " ".join("-f events[]=%s" % e for e in needed))
log.info("repair: keep the poll as reconciliation at a much longer "
"interval, an hour rather than %ds.", args.interval)
print(json.dumps({"rows": rows, "cost": cost, "state": state,
"hooks": len(hooks), "unread": blind}, indent=2))
return 1 if state in ("polling", "polling-dominates") else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Decide whether a polling loop should be a webhook, and cost it if it should.
*
* Read only. Two GETs to list hooks, one to read the quota, and the repair is
* printed as a command rather than run.
*
* How often a client polls is not visible from the API. Whether a push path
* exists at all is, and that is the half worth checking.
*/
const API = 'https://api.github.com';
const UA = 'github-webhook-vs-poll/1.0';
export const HOURLY_LIMIT = 5000;
// The polled endpoint on the left, the event that would push the same thing on
// the right. Anything not on this list is a real reason to keep polling.
export const CONCERNS = {
issues: ['GET /repos/{owner}/{repo}/issues', ['issues']],
issue_comments: ['GET /repos/{owner}/{repo}/issues/comments', ['issue_comment']],
pulls: ['GET /repos/{owner}/{repo}/pulls', ['pull_request']],
commits: ['GET /repos/{owner}/{repo}/commits', ['push']],
releases: ['GET /repos/{owner}/{repo}/releases', ['release']],
workflow_runs: ['GET /repos/{owner}/{repo}/actions/runs', ['workflow_run']],
};
/**
* Split hook subscriptions into what delivers and what does not. Pure.
* Inactive hooks are kept separately: "there is a hook and it is switched off"
* is a much faster fix than "there is no hook".
*/
export function subscribedEvents(hooks) {
const active = new Set();
const inactive = new Set();
let wildcard = false;
let inactiveWildcard = false;
for (const hook of hooks ?? []) {
if (!hook || typeof hook !== 'object') continue;
const live = hook.active !== false;
for (const event of hook.events ?? []) {
const name = String(event);
if (live) {
active.add(name);
wildcard = wildcard || name === '*';
} else {
inactive.add(name);
inactiveWildcard = inactiveWildcard || name === '*';
}
}
}
return { events: active, wildcard, inactive, inactive_wildcard: inactiveWildcard };
}
/** One row per polled concern saying whether anything would push it. Pure. */
export function coverage(concerns, hooks) {
const subs = subscribedEvents(hooks);
const rows = [];
for (const concern of concerns ?? []) {
const wanted = (CONCERNS[concern] ?? [null, [concern]])[1];
const names = wanted.join('/');
if (subs.wildcard) {
rows.push({ concern, state: 'covered',
detail: `a wildcard subscription delivers ${names}, though it delivers everything else too` });
} else if (wanted.some((w) => subs.events.has(w))) {
rows.push({ concern, state: 'covered', detail: `an active hook subscribes to ${names}` });
} else if (wanted.some((w) => subs.inactive.has(w)) || subs.inactive_wildcard) {
rows.push({ concern, state: 'uncovered',
detail: `a hook subscribes to ${names} but it is not active, and an inactive hook delivers nothing` });
} else {
rows.push({ concern, state: 'uncovered', detail: `no hook subscribes to ${names}` });
}
}
return rows;
}
/** Requests and detection latency for the loop as configured. Pure. */
export function pollCost(concerns, intervalS, repos = 1) {
const n = Math.max(0, Number.parseInt(repos, 10) || 0);
const interval = Math.max(1, Number.parseInt(intervalS, 10) || 1);
const calls = (concerns ?? []).length * n;
const perHour = Math.round((calls * 3600) / interval);
return {
requests_per_hour: perHour,
requests_per_day: perHour * 24,
mean_latency_s: interval / 2,
worst_latency_s: interval,
};
}
/** Turn coverage and cost into one finding. Pure. */
export function verdict(rows, cost, hourlyLimit = HOURLY_LIMIT) {
if (!rows || !rows.length) {
return ['nothing-polled',
'no concerns were named, so there is nothing to compare against the hooks'];
}
const uncovered = rows.filter((r) => r.state === 'uncovered');
const share = (cost.requests_per_hour ?? 0) / Math.max(1, hourlyLimit);
if (!uncovered.length) {
return ['push',
'every polled concern already has an active hook, so this loop is a ' +
'reconciliation pass rather than a detection mechanism. Run it on a slow schedule.'];
}
const summary = `${uncovered.length} of ${rows.length} polled concern(s) have ` +
`no active hook. The loop costs ${cost.requests_per_hour ?? 0} request(s) an ` +
`hour to notice them ${Math.round(cost.mean_latency_s ?? 0)}s late on average.`;
if (share >= 0.5) {
return ['polling-dominates',
`${summary} That is ${Math.round(share * 100)}% of the hourly quota spent ` +
'on the clock rather than on activity.'];
}
return ['polling', summary];
}
async function get(token, path) {
const url = path.startsWith('/') ? API + path : path;
const res = await fetch(url, {
headers: {
Authorization: `Bearer ${token}`,
Accept: 'application/vnd.github+json',
'X-GitHub-Api-Version': '2022-11-28',
'User-Agent': UA,
},
});
let body = null;
try { body = await res.json(); } catch { body = null; }
return { status: res.status, body };
}
async function main() {
const token = process.env.GITHUB_TOKEN;
if (!token) {
console.error('set GITHUB_TOKEN. Listing hooks needs admin on the ' +
'repository, but only read access to it');
process.exitCode = 2;
return;
}
const repo = process.argv[2];
if (!repo) {
console.error('usage: node github-webhook-vs-poll.mjs owner/name [concerns] [interval] [repos] [org]');
process.exitCode = 2;
return;
}
const concerns = (process.argv[3] ?? 'issues,issue_comments,pulls')
.split(',').map((c) => c.trim()).filter(Boolean);
const interval = Number.parseInt(process.argv[4] ?? '30', 10) || 30;
const repos = Number.parseInt(process.argv[5] ?? '1', 10) || 1;
const org = process.argv[6];
const hooks = [];
const blind = [];
const repoHooks = await get(token, `/repos/${repo}/hooks`);
if (repoHooks.status === 200 && Array.isArray(repoHooks.body)) {
hooks.push(...repoHooks.body);
console.log(`${repo}: ${repoHooks.body.length} repository hook(s)`);
} else {
blind.push(`repository hooks (${repoHooks.status})`);
console.warn(`could not read repository hooks: ${repoHooks.status}. This ` +
'token cannot see them, which is not the same as there being none.');
}
if (org) {
const orgHooks = await get(token, `/orgs/${org}/hooks`);
if (orgHooks.status === 200 && Array.isArray(orgHooks.body)) {
hooks.push(...orgHooks.body);
console.log(`${org}: ${orgHooks.body.length} organisation hook(s)`);
} else {
blind.push(`organisation hooks (${orgHooks.status})`);
console.warn(`could not read organisation hooks: ${orgHooks.status}`);
}
}
for (const hook of hooks) {
console.log(` hook ${hook.id} active=${hook.active} events=` +
`${(hook.events ?? []).join(',') || 'none'}`);
}
const rows = coverage(concerns, hooks);
const cost = pollCost(concerns, interval, repos);
for (const row of rows) console.log(`${row.concern} ${row.state} ${row.detail}`);
const rate = await get(token, '/rate_limit');
if (rate.status === 200) {
const core = ((rate.body ?? {}).resources ?? {}).core ?? {};
console.log(`core quota: ${core.used} used of ${core.limit}`);
}
const [state, detail] = verdict(rows, cost);
console.log(`${state}: ${detail}`);
if (blind.length) {
console.warn(`unread: ${blind.join('; ')}. Anything reported as uncovered ` +
'may already be covered by a hook this token cannot see.');
}
if (state === 'polling' || state === 'polling-dominates') {
const needed = [...new Set(rows.filter((r) => r.state === 'uncovered')
.flatMap((r) => (CONCERNS[r.concern] ?? [null, [r.concern]])[1]))].sort();
console.log('repair: create one hook for the events you consume. This ' +
'script does not create it:');
console.log(` gh api --method POST /repos/${repo}/hooks -f name=web ` +
'-f config[url]=https://example.test/hooks -f config[content_type]=json ' +
`-f config[secret]=YOURSECRET ${needed.map((e) => `-f events[]=${e}`).join(' ')}`);
console.log(`repair: keep the poll as reconciliation at a much longer ` +
`interval, an hour rather than ${interval}s.`);
}
console.log(JSON.stringify({ rows, cost, state, hooks: hooks.length, unread: blind }, null, 2));
process.exitCode = (state === 'polling' || state === 'polling-dominates') ? 1 : 0;
}
// Only run when invoked directly, so importing this module from the test file
// does not execute main() and fail on the missing token.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
Coverage has three answers, not two, and the third is the one worth pinning: a hook that subscribes to exactly the right event but has active set to false is uncovered, and saying so in those words saves someone from creating a duplicate hook next to the disabled one. The wildcard case goes the other way, and the cost function is checked in both units it reports, because a latency of half the interval is the number the argument turns on.
from github_webhook_vs_poll import coverage, poll_cost, subscribed_events, verdict
ACTIVE = [{"id": 1, "active": True, "events": ["issues", "issue_comment"]}]
DISABLED = [{"id": 2, "active": False, "events": ["issues"]}]
WILDCARD = [{"id": 3, "active": True, "events": ["*"]}]
def test_active_and_inactive_subscriptions_are_kept_apart():
subs = subscribed_events(ACTIVE + DISABLED)
assert "issue_comment" in subs["events"]
assert subs["inactive"] == {"issues"}
assert subs["wildcard"] is False
def test_a_wildcard_is_recognised_only_when_the_hook_is_active():
assert subscribed_events(WILDCARD)["wildcard"] is True
off = [{"id": 4, "active": False, "events": ["*"]}]
assert subscribed_events(off)["wildcard"] is False
assert subscribed_events(off)["inactive_wildcard"] is True
def test_junk_in_the_hook_list_does_not_raise():
assert subscribed_events([None, "nope", {}])["events"] == set()
assert subscribed_events(None)["events"] == set()
def test_an_active_hook_covers_its_concern():
rows = coverage(["issues", "pulls"], ACTIVE)
assert rows[0]["state"] == "covered"
assert rows[1]["state"] == "uncovered"
def test_a_disabled_hook_is_uncovered_and_says_why():
rows = coverage(["issues"], DISABLED)
assert rows[0]["state"] == "uncovered"
assert "not active" in rows[0]["detail"]
def test_a_wildcard_covers_everything_and_warns_that_it_does():
rows = coverage(["issues", "commits", "releases"], WILDCARD)
assert [r["state"] for r in rows] == ["covered"] * 3
assert "everything else" in rows[0]["detail"]
def test_an_unknown_concern_is_matched_against_its_own_name():
rows = coverage(["deployment"], [{"active": True, "events": ["deployment"]}])
assert rows[0]["state"] == "covered"
def test_no_hooks_at_all_leaves_every_concern_uncovered():
rows = coverage(["issues", "pulls"], [])
assert all(r["state"] == "uncovered" for r in rows)
assert "no hook subscribes" in rows[0]["detail"]
def test_the_poll_costs_endpoints_times_repos_times_the_clock():
cost = poll_cost(["issues", "pulls"], 60, repos=3)
assert cost["requests_per_hour"] == 360
assert cost["requests_per_day"] == 8640
def test_latency_is_half_the_interval_on_average_and_all_of_it_at_worst():
cost = poll_cost(["issues"], 60)
assert cost["mean_latency_s"] == 30
assert cost["worst_latency_s"] == 60
def test_a_zero_interval_is_clamped_rather_than_dividing_by_zero():
assert poll_cost(["issues"], 0)["requests_per_hour"] == 3600
def test_an_uncovered_concern_is_reported_with_both_numbers():
rows = coverage(["issues", "pulls"], [])
state, detail = verdict(rows, poll_cost(["issues", "pulls"], 60, repos=3))
assert state == "polling"
assert "2 of 2" in detail
assert "360 request(s)" in detail
assert "30s late" in detail
def test_a_loop_spending_half_the_quota_is_called_out_as_such():
rows = coverage(["issues", "pulls"], [])
state, detail = verdict(rows, poll_cost(["issues", "pulls"], 1, repos=1))
assert state == "polling-dominates"
assert "%" in detail
def test_full_coverage_reframes_the_loop_as_reconciliation():
state, detail = verdict(coverage(["issues"], ACTIVE), poll_cost(["issues"], 3600))
assert state == "push"
assert "reconciliation" in detail
def test_polling_nothing_is_its_own_state():
assert verdict([], poll_cost([], 60))[0] == "nothing-polled"
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
coverage, pollCost, subscribedEvents, verdict,
} from './github-webhook-vs-poll.mjs';
const ACTIVE = [{ id: 1, active: true, events: ['issues', 'issue_comment'] }];
const DISABLED = [{ id: 2, active: false, events: ['issues'] }];
const WILDCARD = [{ id: 3, active: true, events: ['*'] }];
test('active and inactive subscriptions are kept apart', () => {
const subs = subscribedEvents([...ACTIVE, ...DISABLED]);
assert.ok(subs.events.has('issue_comment'));
assert.deepEqual([...subs.inactive], ['issues']);
assert.equal(subs.wildcard, false);
});
test('a wildcard is recognised only when the hook is active', () => {
assert.equal(subscribedEvents(WILDCARD).wildcard, true);
const off = [{ id: 4, active: false, events: ['*'] }];
assert.equal(subscribedEvents(off).wildcard, false);
assert.equal(subscribedEvents(off).inactive_wildcard, true);
});
test('junk in the hook list does not throw', () => {
assert.equal(subscribedEvents([null, 'nope', {}]).events.size, 0);
assert.equal(subscribedEvents(null).events.size, 0);
});
test('an active hook covers its concern', () => {
const rows = coverage(['issues', 'pulls'], ACTIVE);
assert.equal(rows[0].state, 'covered');
assert.equal(rows[1].state, 'uncovered');
});
test('a disabled hook is uncovered and says why', () => {
const rows = coverage(['issues'], DISABLED);
assert.equal(rows[0].state, 'uncovered');
assert.match(rows[0].detail, /not active/);
});
test('a wildcard covers everything and warns that it does', () => {
const rows = coverage(['issues', 'commits', 'releases'], WILDCARD);
assert.deepEqual(rows.map((r) => r.state), ['covered', 'covered', 'covered']);
assert.match(rows[0].detail, /everything else/);
});
test('an unknown concern is matched against its own name', () => {
const rows = coverage(['deployment'], [{ active: true, events: ['deployment'] }]);
assert.equal(rows[0].state, 'covered');
});
test('no hooks at all leaves every concern uncovered', () => {
const rows = coverage(['issues', 'pulls'], []);
assert.ok(rows.every((r) => r.state === 'uncovered'));
assert.match(rows[0].detail, /no hook subscribes/);
});
test('the poll costs endpoints times repos times the clock', () => {
const cost = pollCost(['issues', 'pulls'], 60, 3);
assert.equal(cost.requests_per_hour, 360);
assert.equal(cost.requests_per_day, 8640);
});
test('latency is half the interval on average and all of it at worst', () => {
const cost = pollCost(['issues'], 60);
assert.equal(cost.mean_latency_s, 30);
assert.equal(cost.worst_latency_s, 60);
});
test('a zero interval is clamped rather than dividing by zero', () => {
assert.equal(pollCost(['issues'], 0).requests_per_hour, 3600);
});
test('an uncovered concern is reported with both numbers', () => {
const rows = coverage(['issues', 'pulls'], []);
const [state, detail] = verdict(rows, pollCost(['issues', 'pulls'], 60, 3));
assert.equal(state, 'polling');
assert.match(detail, /2 of 2/);
assert.match(detail, /360 request\(s\)/);
assert.match(detail, /30s late/);
});
test('a loop spending half the quota is called out as such', () => {
const rows = coverage(['issues', 'pulls'], []);
const [state, detail] = verdict(rows, pollCost(['issues', 'pulls'], 1, 1));
assert.equal(state, 'polling-dominates');
assert.match(detail, /%/);
});
test('full coverage reframes the loop as reconciliation', () => {
const [state, detail] = verdict(coverage(['issues'], ACTIVE), pollCost(['issues'], 3600));
assert.equal(state, 'push');
assert.match(detail, /reconciliation/);
});
test('polling nothing is its own state', () => {
assert.equal(verdict([], pollCost([], 60))[0], 'nothing-polled');
});
FAQ
Can a script tell that my integration is polling?
Not directly. Nothing GitHub returns says how often you call it, so the client half of this is a blind spot. What the API does show is the other half: whether any active hook exists for the events you are reading, and how fast the core counter is climbing between two samples. An empty hook list next to steady consumption is strong evidence, and it is the evidence this script collects.
Is it wrong to keep polling once the webhook exists?
No, and removing the poll entirely is usually a mistake. Deliveries fail, receivers go down, and a webhook has no replay you can rely on beyond the retained delivery log. Keep a reconciliation poll, but change what it is for: hourly rather than every thirty seconds, sweeping for anything missed rather than being the way things are noticed.
Should I just subscribe to every event with a wildcard?
It is tempting and it moves the cost rather than removing it. A wildcard delivers everything GitHub emits for that repository, including large payloads for events you do not handle, so your receiver spends its time discarding them and your signature verification runs on all of it. Name the events you consume; the list is short and it documents the integration.
What do I need to read the hook list?
Admin permission on the repository for the repository hooks, and org admin for the org ones. A read-only token that lacks those gets a 403, which is genuinely different from an empty list: it means unknown, not none. The script reports it as unread rather than folding it into the finding, because treating a permissions gap as an absence is how you end up creating a second hook beside the one you could not see.
How much latency does polling actually add?
Half the interval on average and the full interval at worst, plus whatever your own processing takes. A thirty-second loop notices a pull request fifteen seconds after it opens, typically. A webhook is delivered in about the time it takes to make the request. The gap matters most for anything a person is waiting on, which is usually the bot comment on a pull request.
Related field notes
- The hook is not subscribed to the event
- Deliveries have been failing unnoticed
- The x-poll-interval header is ignored
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Best practices for using the REST API — GitHub Docs
- Repository webhooks — GitHub REST API
- About webhooks — GitHub Docs
- Webhook events and payloads — GitHub Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.