Diagnostic GitHub API
the hook subscribes to every event with a wildcard
The hook was set up in a hurry, with * in the events list, because nobody was sure yet which events the integration would need. That was two years ago. The receiver now handles four event types and is delivered somewhere north of forty, and every one of the ones it does not want still arrives, still gets its signature verified, and is still thrown away. Nothing has ever failed.
events: ["*"] subscribes the hook to every event type GitHub currently has and every event type it adds in future. There is no error and no failure: the hook does precisely what it was configured to do. The cost is volume — deliveries you pay to receive, authenticate and discard — and it grows on GitHub's release schedule rather than yours.
Read GET /repos/{owner}/{repo}/hooks and look for * in the events array. Then quantify it: count the deliveries by event over the retained window and work out what fraction came from events your handlers do not implement. That fraction is the finding, and it is usually large, because the highest-volume events on a busy repository — push, status, check_run, workflow_job — are rarely the ones a small integration cares about. The repair is a list, and the script can print it in full: the explicit set of events the receiver actually handles.
The problem in plain words
Nothing draws attention to it, because nothing is wrong. There is no error rate to alert on, no failed delivery, no 4xx, no queue backing up in a way anybody has noticed. Every dashboard is green. A configuration that is quietly three times more expensive than it needs to be produces exactly the same monitoring signal as one that is right, which is why this survives audits that were genuinely looking for problems.
The cost shows up somewhere else, wearing a disguise. The receiver is a little slower than it should be, the bill for whatever runs it is a little higher, and when somebody eventually profiles it, the time is going into signature verification and JSON parsing of payloads that are discarded on the next line. That reads as a performance problem in the receiver, and it gets fixed as one: a faster parser, a bigger instance, more workers. The volume itself is treated as a given.
The part that makes it grow is that nobody revisits it. A wildcard subscription is a standing agreement to receive event types that did not exist when the agreement was made. Every time GitHub ships a new event, the firehose widens on its own, and the change arrives in production without a deploy, a review or a note in anybody's changelog. The volume in six months is not a number anybody chose.
Why it happens
The wildcard is open ended by design. It does not expand to the list of events at the moment you save it; it means all events, evaluated at delivery time. That is a useful property for a mirror or an audit sink and a liability for anything else, because it means the subscription changes without you.
The expensive events are rarely the interesting ones. Volume on a repository is dominated by pushes, statuses and CI events, and those payloads are large. An integration that cares about issues and pull requests receives all of it anyway, and the ratio gets worse the busier and healthier the repository is. Growth in the team makes the waste grow with it.
Every delivery has a fixed cost before you know what it is. The connection, the body read, the signature computed over the raw bytes, the parse. Only after all of that can the handler discard the event. So the cost of an unwanted delivery is not zero and it is not small; it is most of the cost of a wanted one.
It is the opposite failure to an unsubscribed event. That note looks for events in your code that are missing from the hook, where the symptom is a handler that never runs. This one looks in the other direction: events on the hook that are missing from your code, where the symptom is nothing at all. Same two sets, subtracted the other way round, and the reason both notes exist is that one of them has a symptom and the other has to be gone looking for.
Data leaves GitHub either way. A wildcard hook sends your repository's activity — commit messages, branch names, issue bodies, review comments — to your endpoint whether or not anything reads it. Narrowing the subscription is a reduction in what crosses the boundary, which is worth saying out loud when the alternative is a conversation about parser performance.
The fix, as a flow
Set arithmetic in the direction nobody runs it. The usual check asks which events the code handles that the hook does not send; this one asks the opposite, and the answer has no symptom attached to it, so it has to be gone looking for. The delivery tally turns the objection into a fraction, and the proposal is built from what the receiver implements rather than from what happened to arrive, because a retention window shorter than a release cycle would quietly prune a handler that is still needed.
How to fix it
Find the wildcards
GET /repos/{owner}/{repo}/hooks, and the organization equivalent if you have org read. A hook whose events array contains * is the finding; it does not matter what else is in the array, because the wildcard subsumes it.
Count what actually arrives
Page the deliveries feed and tally by event. This turns an abstract objection into a number: five hundred deliveries in the window, of which four hundred and ten were push, status and check_run, none of which the receiver implements. That is the sentence that gets the change approved.
State the fraction, not the count
Deliveries the receiver discards, divided by all deliveries. A count depends on how long the window is and how busy the week was; a fraction does not, and it is comparable between repositories. Report both, lead with the fraction.
Write the explicit list
Replace * with the events your handlers implement. The script prints the list, so the repair is a copy rather than an exercise. Include the events you handle but did not see in the window: an event that has not fired recently is not an event you do not need, and pruning on observation alone is how a release handler stops working three months later.
Decide deliberately about new events
The wildcard's one genuine feature is that new event types arrive automatically. If you actually want that — a mirror, a compliance sink, an event lake — keep it, and say so in the code so the next audit does not undo it. If you do not, then adding a new event type should be a change somebody makes on purpose, which is what the explicit list buys.
How to check it worked
After the list replaces the wildcard, the same read shows the subscription bounded and the discarded fraction at zero, with no change to the events the receiver actually uses.
GITHUB_TOKEN=$READ_ONLY_TOKEN python3 github_hook_event_volume.py \
--repo acme-corp/platform --handles issues,pull_request,pull_request_review
# hook 517238001 -> https://hooks.example.com/github
# subscribed: * (wildcard)
# 500 deliveries in the window across 11 event type(s)
# wildcard: 410 of 500 deliveries (82.0%) were events this receiver does not
# implement, and * also subscribes to every event type GitHub ships next
# repair: replace ["*"] with
# ["issues", "pull_request", "pull_request_review"]
# after the hook is narrowed
# tight: every subscribed event is one the receiver implementsThe full code
One GET for the hooks, a cursor-paginated GET for the deliveries, and the rest is set arithmetic in the direction nobody runs it: subscribed minus handled, rather than handled minus subscribed. The tally is the part that turns an opinion into a number, and the proposal function is the deliverable — it normalises, drops the wildcard, deduplicates and sorts, so the output is a list you can paste rather than a principle you have to apply. The one judgement encoded in it is that a handled event with no deliveries in the window stays on the list.
"""Quantify what a wildcard webhook subscription costs a receiver.
Read only. GETs the repository's hooks and their delivery records, tallies the
deliveries by event type, and reports the fraction that came from events the
receiver does not implement. Nothing is created, edited or deleted: the script
prints the explicit events list to install in place of the wildcard.
A hook configured with ["*"] receives every event type GitHub has and every one
it adds afterwards. Nothing fails. The cost is volume, paid on every delivery
before the handler can decide it does not want it, and it grows on GitHub's
release schedule rather than yours.
This is the opposite comparison to an unsubscribed-event check. That one looks
for events in your code that are missing from the hook. This one looks for
events on the hook that are missing from your code.
Environment:
GITHUB_TOKEN a read-only token with access to the repository
GITHUB_HANDLED_EVENTS comma separated events your receiver implements
"""
import argparse
import json
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("github_hook_event_volume")
API = "https://api.github.com"
UA = "github-hook-event-volume/1.0"
WILDCARD = "*"
def normalize(name):
"""One event name, lowercased and trimmed. Pure.
Deliberately narrow: case and surrounding space only. A genuinely misspelled
event name should stay visible as itself rather than be quietly corrected
into something that looks subscribed.
"""
return str(name or "").strip().lower()
def subscribed(hook):
"""The normalised events array on a hook. Pure."""
if not isinstance(hook, dict):
return []
events = hook.get("events")
if not isinstance(events, list):
return []
return [e for e in (normalize(x) for x in events) if e]
def is_wildcard(events):
"""Whether this subscription is open ended. Pure."""
return WILDCARD in {normalize(e) for e in (events or [])}
def handled_set(names):
"""The events a receiver implements, as a normalised set. Pure."""
if isinstance(names, str):
names = names.replace(";", ",").split(",")
return {e for e in (normalize(n) for n in (names or [])) if e and e != WILDCARD}
def tally(rows):
"""Deliveries by event type. Pure."""
counts = {}
for row in rows or []:
if not isinstance(row, dict):
continue
event = normalize(row.get("event")) or "unknown"
counts[event] = counts.get(event, 0) + 1
return counts
def waste(counts, handled):
"""How much of the delivered volume the receiver discards. Pure."""
counts = counts or {}
handled = handled_set(handled)
total = sum(counts.values())
unwanted = {e: n for e, n in counts.items() if e not in handled}
discarded = sum(unwanted.values())
return {"total": total,
"unhandled_deliveries": discarded,
"unhandled_events": sorted(unwanted),
"share": round(100.0 * discarded / total, 1) if total else 0.0}
def proposed_events(handled):
"""The explicit events list to install in place of the wildcard. Pure.
Built from what the receiver implements rather than from what happened to
arrive. An event that has not fired during the retained window is not an
event you do not need, and pruning on observation alone is how a release
handler stops working three months later.
"""
return sorted(handled_set(handled))
def never_seen(counts, handled):
"""Handled events with no deliveries in the window. Pure.
Reported as a caution, never as a reason to drop them.
"""
counts = counts or {}
return sorted(e for e in handled_set(handled) if not counts.get(e))
def verdict(events, counts, handled):
"""Turn the subscription and the volume into a finding. Pure."""
subs = {e for e in (normalize(x) for x in (events or [])) if e}
wanted = handled_set(handled)
if WILDCARD in subs:
w = waste(counts, wanted)
if not w["total"]:
return ("wildcard-unmeasured",
"this hook subscribes to every event with *, and no "
"deliveries in the retained window let the volume be "
"measured. The subscription is open ended either way: "
"every event type GitHub ships next joins it.")
if w["unhandled_deliveries"]:
return ("wildcard",
"%d of %d deliveries (%.1f%%) were events this receiver "
"does not implement, and * also subscribes to every event "
"type GitHub ships next."
% (w["unhandled_deliveries"], w["total"], w["share"]))
return ("wildcard-all-handled",
"every delivery in the window happened to be an event this "
"receiver implements, which is luck rather than design: * "
"subscribes to event types that do not exist yet.")
extra = sorted(subs - wanted)
if extra:
return ("over-subscribed",
"this hook subscribes to %d event(s) the receiver does not "
"implement: %s." % (len(extra), ", ".join(extra)))
if not subs:
return ("no-events",
"this hook has an empty events array, so nothing is delivered "
"to it at all.")
return ("tight", "every subscribed event is one the receiver implements.")
def repair(state, handled=None, counts=None):
"""The sentence a reader has to act on. Pure."""
listing = json.dumps(proposed_events(handled))
if state in ("wildcard", "wildcard-unmeasured", "wildcard-all-handled"):
caution = ""
pending = never_seen(counts, handled)
if pending:
caution = (" Keep %s on the list even though nothing arrived for "
"them in this window." % ", ".join(pending))
return ("replace [\"*\"] with %s, which bounds the subscription and "
"stops new event types joining it without a decision.%s"
% (listing, caution))
if state == "over-subscribed":
return ("narrow the events array to %s. Nothing is failing; this is "
"volume the receiver pays for and discards." % listing)
if state == "no-events":
return ("add the events the receiver implements: %s. An empty array "
"delivers nothing." % listing)
return "nothing. The subscription matches what the receiver handles."
def next_link(headers):
"""The rel=next URL from a Link header, or None. Pure."""
link = (headers or {}).get("Link") or (headers or {}).get("link") or ""
for part in str(link).split(","):
url = part.split(";")[0].strip()
if 'rel="next"' in part and url.startswith("<") and url.endswith(">"):
return url[1:-1]
return None
def get(session, url):
"""One GET. Returns (status, json-or-None, headers)."""
full = API + url if url.startswith("/") else url
r = session.get(full, timeout=30)
try:
body = r.json()
except ValueError:
body = None
return r.status_code, body, r.headers
def deliveries(session, owner, repo, hook_id, pages=8):
"""Delivery records for one hook, following the cursor. Read only."""
url = "/repos/%s/%s/hooks/%s/deliveries?per_page=100" % (owner, repo, hook_id)
out = []
for _ in range(pages):
status, body, headers = get(session, url)
if status != 200 or not isinstance(body, list):
log.error("deliveries for hook %s returned %d", hook_id, status)
break
out.extend(body)
url = next_link(headers)
if not url:
break
return out
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--repo", required=True, help="owner/repo")
ap.add_argument("--handles", default=os.environ.get("GITHUB_HANDLED_EVENTS", ""),
help="comma separated events the receiver implements")
args = ap.parse_args()
token = os.environ.get("GITHUB_TOKEN")
if not token:
log.error("set GITHUB_TOKEN to a read-only token that can read the "
"repository's hooks")
return 2
if "/" not in args.repo:
log.error("--repo takes owner/repo")
return 2
handled = handled_set(args.handles)
if not handled:
log.error("pass --handles with the events your receiver implements; "
"without them there is nothing to compare the hook against")
return 2
owner, repo = args.repo.split("/", 1)
session = requests.Session()
session.headers.update({
"Authorization": "Bearer " + token,
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": UA,
})
status, hooks, _ = get(session, "/repos/%s/%s/hooks?per_page=100" % (owner, repo))
if status != 200 or not isinstance(hooks, list):
log.error("GET hooks returned %d", status)
return 2
findings = []
for hook in hooks:
hook_id = hook.get("id")
url = (hook.get("config") or {}).get("url", "?")
events = subscribed(hook)
log.info("hook %s -> %s", hook_id, url)
log.info("subscribed: %s", "* (wildcard)" if is_wildcard(events)
else ", ".join(events) or "nothing")
rows = deliveries(session, owner, repo, hook_id)
counts = tally(rows)
log.info("%d deliveries in the window across %d event type(s)",
sum(counts.values()), len(counts))
state, detail = verdict(events, counts, handled)
log.info("%s: %s", state, detail)
log.info("repair: %s", repair(state, handled, counts))
findings.append({"hook_id": hook_id, "url": url, "events": events,
"wildcard": is_wildcard(events), "state": state,
"detail": detail, "counts": counts,
"waste": waste(counts, handled),
"proposed_events": proposed_events(handled),
"handled_but_unseen": never_seen(counts, handled)})
print(json.dumps({"repo": args.repo, "handled": sorted(handled),
"hooks": findings}, indent=2, default=str))
return 1 if any(f["state"] != "tight" for f in findings) else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Quantify what a wildcard webhook subscription costs a receiver.
*
* Read only. GETs the repository's hooks and their delivery records, tallies
* the deliveries by event type, and reports the fraction that came from events
* the receiver does not implement. Nothing is created, edited or removed: the
* script prints the explicit events list to install in place of the wildcard.
*
* Environment:
* GITHUB_TOKEN a read-only token with access to the repository
* GITHUB_REPO owner/repo
* GITHUB_HANDLED_EVENTS comma separated events the receiver implements
*/
const API = 'https://api.github.com';
const UA = 'github-hook-event-volume/1.0';
export const WILDCARD = '*';
/** One event name, lowercased and trimmed. Pure. */
export function normalize(name) {
return String(name ?? '').trim().toLowerCase();
}
/** The normalised events array on a hook. Pure. */
export function subscribed(hook) {
if (!hook || typeof hook !== 'object' || !Array.isArray(hook.events)) return [];
return hook.events.map(normalize).filter(Boolean);
}
/** Whether this subscription is open ended. Pure. */
export function isWildcard(events) {
return (events || []).map(normalize).includes(WILDCARD);
}
/** The events a receiver implements, as a normalised set. Pure. */
export function handledSet(names) {
const list = typeof names === 'string' ? names.replace(/;/g, ',').split(',') : (names || []);
return new Set([...list].map(normalize).filter((e) => e && e !== WILDCARD));
}
/** Deliveries by event type. Pure. */
export function tally(rows) {
const counts = {};
for (const row of rows || []) {
if (!row || typeof row !== 'object') continue;
const event = normalize(row.event) || 'unknown';
counts[event] = (counts[event] || 0) + 1;
}
return counts;
}
/** How much of the delivered volume the receiver discards. Pure. */
export function waste(counts, handled) {
const table = counts || {};
const wanted = handledSet(handled instanceof Set ? [...handled] : handled);
const total = Object.values(table).reduce((a, b) => a + b, 0);
const unwanted = Object.entries(table).filter(([e]) => !wanted.has(e));
const discarded = unwanted.reduce((a, [, n]) => a + n, 0);
return {
total,
unhandled_deliveries: discarded,
unhandled_events: unwanted.map(([e]) => e).sort(),
share: total ? Math.round((1000 * discarded) / total) / 10 : 0,
};
}
/** The explicit events list to install in place of the wildcard. Pure. */
export function proposedEvents(handled) {
return [...handledSet(handled instanceof Set ? [...handled] : handled)].sort();
}
/** Handled events with no deliveries in the window. Pure. */
export function neverSeen(counts, handled) {
const table = counts || {};
return [...handledSet(handled instanceof Set ? [...handled] : handled)]
.filter((e) => !table[e]).sort();
}
/** Turn the subscription and the volume into a finding. Pure. */
export function verdict(events, counts, handled) {
const subs = new Set((events || []).map(normalize).filter(Boolean));
const wanted = handledSet(handled instanceof Set ? [...handled] : handled);
if (subs.has(WILDCARD)) {
const w = waste(counts, wanted);
if (!w.total) {
return ['wildcard-unmeasured',
'this hook subscribes to every event with *, and no deliveries in the '
+ 'retained window let the volume be measured. The subscription is open '
+ 'ended either way: every event type GitHub ships next joins it.'];
}
if (w.unhandled_deliveries) {
return ['wildcard',
`${w.unhandled_deliveries} of ${w.total} deliveries (${w.share.toFixed(1)}%) `
+ 'were events this receiver does not implement, and * also subscribes to '
+ 'every event type GitHub ships next.'];
}
return ['wildcard-all-handled',
'every delivery in the window happened to be an event this receiver '
+ 'implements, which is luck rather than design: * subscribes to event '
+ 'types that do not exist yet.'];
}
const extra = [...subs].filter((e) => !wanted.has(e)).sort();
if (extra.length) {
return ['over-subscribed',
`this hook subscribes to ${extra.length} event(s) the receiver does not `
+ `implement: ${extra.join(', ')}.`];
}
if (!subs.size) {
return ['no-events',
'this hook has an empty events array, so nothing is delivered to it at all.'];
}
return ['tight', 'every subscribed event is one the receiver implements.'];
}
/** The sentence a reader has to act on. Pure. */
export function repair(state, handled = null, counts = null) {
const listing = JSON.stringify(proposedEvents(handled));
if (['wildcard', 'wildcard-unmeasured', 'wildcard-all-handled'].includes(state)) {
const pending = neverSeen(counts, handled);
const caution = pending.length
? ` Keep ${pending.join(', ')} on the list even though nothing arrived for them in this window.`
: '';
return `replace ["*"] with ${listing}, which bounds the subscription and stops `
+ `new event types joining it without a decision.${caution}`;
}
if (state === 'over-subscribed') {
return `narrow the events array to ${listing}. Nothing is failing; this is `
+ 'volume the receiver pays for and discards.';
}
if (state === 'no-events') {
return `add the events the receiver implements: ${listing}. An empty array `
+ 'delivers nothing.';
}
return 'nothing. The subscription matches what the receiver handles.';
}
/** The rel=next URL from a Link header, or null. Pure. */
export function nextLink(headers) {
const link = (headers && (headers.get ? headers.get('link') : headers.Link || headers.link)) || '';
for (const part of String(link).split(',')) {
const url = part.split(';')[0].trim();
if (part.includes('rel="next"') && url.startsWith('<') && url.endsWith('>')) {
return url.slice(1, -1);
}
}
return null;
}
function headersFor(token) {
return {
Authorization: `Bearer ${token}`,
Accept: 'application/vnd.github+json',
'X-GitHub-Api-Version': '2022-11-28',
'User-Agent': UA,
};
}
async function get(token, url) {
const res = await fetch(url.startsWith('/') ? API + url : url, { headers: headersFor(token) });
let body = null;
try { body = await res.json(); } catch { body = null; }
return { status: res.status, body, headers: res.headers };
}
async function main() {
const token = process.env.GITHUB_TOKEN;
const repo = process.env.GITHUB_REPO;
const handled = handledSet(process.env.GITHUB_HANDLED_EVENTS || '');
if (!token || !repo || !repo.includes('/') || !handled.size) {
console.error('set GITHUB_TOKEN, GITHUB_REPO=owner/repo and GITHUB_HANDLED_EVENTS');
process.exitCode = 2;
return;
}
const [owner, name] = repo.split('/');
const hooksRes = await get(token, `/repos/${owner}/${name}/hooks?per_page=100`);
if (hooksRes.status !== 200 || !Array.isArray(hooksRes.body)) {
console.error(`GET hooks returned ${hooksRes.status}`);
process.exitCode = 2;
return;
}
const findings = [];
for (const hook of hooksRes.body) {
const events = subscribed(hook);
console.log(`hook ${hook.id} -> ${(hook.config || {}).url || '?'}`);
console.log(`subscribed: ${isWildcard(events) ? '* (wildcard)' : events.join(', ') || 'nothing'}`);
const rows = [];
let url = `/repos/${owner}/${name}/hooks/${hook.id}/deliveries?per_page=100`;
for (let page = 0; page < 8 && url; page += 1) {
const res = await get(token, url);
if (res.status !== 200 || !Array.isArray(res.body)) break;
rows.push(...res.body);
url = nextLink(res.headers);
}
const counts = tally(rows);
const [state, detail] = verdict(events, counts, handled);
console.log(`${state}: ${detail}`);
console.log(`repair: ${repair(state, handled, counts)}`);
findings.push({
hook_id: hook.id, events, wildcard: isWildcard(events), state,
counts, waste: waste(counts, handled), proposed_events: proposedEvents(handled),
});
}
console.log(JSON.stringify({ repo, handled: [...handled].sort(), hooks: findings }, null, 2));
process.exitCode = findings.every((f) => f.state === 'tight') ? 0 : 1;
}
// Guarded so importing this file from the test runner does not start a run.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The direction of the subtraction is the thing to pin, because the same two sets read the other way round are a different note entirely: an event the receiver handles and the hook does not carry is not this finding and must not be reported as one. After that, the wildcard has to stay a finding even in the window where every delivery happened to be wanted, the fraction has to survive an empty window without dividing by zero, and the proposed list has to keep a handled event that produced no deliveries at all.
from github_hook_event_volume import (
handled_set, is_wildcard, never_seen, next_link, normalize,
proposed_events, repair, subscribed, tally, verdict, waste,
)
HANDLES = "issues,pull_request,release"
STAR = {"id": 1, "events": ["*"], "config": {"url": "https://hooks.example.com/gh"}}
TIGHT = {"id": 2, "events": ["Issues", " pull_request "], "config": {}}
def test_event_names_are_normalised_narrowly():
assert normalize(" Pull_Request ") == "pull_request"
assert normalize(None) == ""
assert subscribed(TIGHT) == ["issues", "pull_request"]
assert subscribed({"events": "issues"}) == []
assert subscribed(None) == []
def test_the_wildcard_is_recognised_however_it_is_written():
assert is_wildcard(["*"])
assert is_wildcard(["push", " * "])
assert not is_wildcard(["push"])
assert not is_wildcard([])
def test_the_handled_set_drops_a_wildcard_it_is_given():
assert handled_set("issues, *, push") == {"issues", "push"}
assert handled_set(["Issues", "issues"]) == {"issues"}
assert handled_set("") == set()
def test_the_tally_counts_by_event_and_names_the_unknown():
rows = [{"event": "push"}, {"event": "Push"}, {"event": None}, "junk"]
assert tally(rows) == {"push": 2, "unknown": 1}
def test_the_waste_is_the_fraction_the_receiver_discards():
counts = {"push": 300, "status": 110, "issues": 90}
w = waste(counts, HANDLES)
assert w["total"] == 500
assert w["unhandled_deliveries"] == 410
assert w["share"] == 82.0
assert w["unhandled_events"] == ["push", "status"]
def test_an_empty_window_does_not_divide_by_zero():
assert waste({}, HANDLES) == {"total": 0, "unhandled_deliveries": 0,
"unhandled_events": [], "share": 0.0}
def test_a_wildcard_with_wasted_volume_is_the_headline_finding():
counts = {"push": 300, "status": 110, "issues": 90}
state, detail = verdict(subscribed(STAR), counts, HANDLES)
assert state == "wildcard"
assert "82.0%" in detail
assert "ships next" in detail
def test_a_wildcard_stays_a_finding_when_the_window_was_all_wanted():
state, detail = verdict(["*"], {"issues": 12}, HANDLES)
assert state == "wildcard-all-handled"
assert "luck rather than design" in detail
def test_a_wildcard_with_no_deliveries_is_still_reported():
state, detail = verdict(["*"], {}, HANDLES)
assert state == "wildcard-unmeasured"
assert "open ended" in detail
def test_events_the_receiver_handles_and_the_hook_omits_are_not_this_finding():
# release is handled and not subscribed; that is the other note's problem.
state, _ = verdict(["issues", "pull_request"], {"issues": 4}, HANDLES)
assert state == "tight"
def test_events_on_the_hook_and_not_in_the_code_are_this_finding():
state, detail = verdict(["issues", "push", "status"], {"push": 3}, HANDLES)
assert state == "over-subscribed"
assert "push, status" in detail
def test_an_empty_subscription_is_its_own_state():
assert verdict([], {}, HANDLES)[0] == "no-events"
def test_the_proposal_keeps_a_handled_event_that_never_fired():
assert proposed_events(HANDLES) == ["issues", "pull_request", "release"]
assert never_seen({"issues": 3}, HANDLES) == ["pull_request", "release"]
text = repair("wildcard", HANDLES, {"issues": 3})
assert '["issues", "pull_request", "release"]' in text
assert "Keep pull_request, release on the list" in text
def test_a_tight_hook_gets_no_repair():
assert repair("tight", HANDLES).startswith("nothing")
def test_the_cursor_is_read_from_the_link_header():
header = '<https://api.github.com/repos/a/b/hooks/1/deliveries?cursor=v2>; rel="next"'
assert next_link({"Link": header}).endswith("cursor=v2")
assert next_link({}) is None
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
handledSet, isWildcard, neverSeen, nextLink, normalize,
proposedEvents, repair, subscribed, tally, verdict, waste,
} from './github-hook-event-volume.mjs';
const HANDLES = 'issues,pull_request,release';
const STAR = { id: 1, events: ['*'], config: { url: 'https://hooks.example.com/gh' } };
const TIGHT = { id: 2, events: ['Issues', ' pull_request '], config: {} };
test('event names are normalised narrowly', () => {
assert.equal(normalize(' Pull_Request '), 'pull_request');
assert.equal(normalize(null), '');
assert.deepEqual(subscribed(TIGHT), ['issues', 'pull_request']);
assert.deepEqual(subscribed({ events: 'issues' }), []);
assert.deepEqual(subscribed(null), []);
});
test('the wildcard is recognised however it is written', () => {
assert.ok(isWildcard(['*']));
assert.ok(isWildcard(['push', ' * ']));
assert.ok(!isWildcard(['push']));
assert.ok(!isWildcard([]));
});
test('the handled set drops a wildcard it is given', () => {
assert.deepEqual([...handledSet('issues, *, push')].sort(), ['issues', 'push']);
assert.deepEqual([...handledSet(['Issues', 'issues'])], ['issues']);
assert.equal(handledSet('').size, 0);
});
test('the tally counts by event and names the unknown', () => {
const rows = [{ event: 'push' }, { event: 'Push' }, { event: null }, 'junk'];
assert.deepEqual(tally(rows), { push: 2, unknown: 1 });
});
test('the waste is the fraction the receiver discards', () => {
const counts = { push: 300, status: 110, issues: 90 };
const w = waste(counts, HANDLES);
assert.equal(w.total, 500);
assert.equal(w.unhandled_deliveries, 410);
assert.equal(w.share, 82.0);
assert.deepEqual(w.unhandled_events, ['push', 'status']);
});
test('an empty window does not divide by zero', () => {
assert.deepEqual(waste({}, HANDLES), {
total: 0, unhandled_deliveries: 0, unhandled_events: [], share: 0,
});
});
test('a wildcard with wasted volume is the headline finding', () => {
const counts = { push: 300, status: 110, issues: 90 };
const [state, detail] = verdict(subscribed(STAR), counts, HANDLES);
assert.equal(state, 'wildcard');
assert.match(detail, /82\.0%/);
assert.match(detail, /ships next/);
});
test('a wildcard stays a finding when the window was all wanted', () => {
const [state, detail] = verdict(['*'], { issues: 12 }, HANDLES);
assert.equal(state, 'wildcard-all-handled');
assert.match(detail, /luck rather than design/);
});
test('a wildcard with no deliveries is still reported', () => {
const [state, detail] = verdict(['*'], {}, HANDLES);
assert.equal(state, 'wildcard-unmeasured');
assert.match(detail, /open ended/);
});
test('events the receiver handles and the hook omits are not this finding', () => {
const [state] = verdict(['issues', 'pull_request'], { issues: 4 }, HANDLES);
assert.equal(state, 'tight');
});
test('events on the hook and not in the code are this finding', () => {
const [state, detail] = verdict(['issues', 'push', 'status'], { push: 3 }, HANDLES);
assert.equal(state, 'over-subscribed');
assert.match(detail, /push, status/);
});
test('an empty subscription is its own state', () => {
assert.equal(verdict([], {}, HANDLES)[0], 'no-events');
});
test('the proposal keeps a handled event that never fired', () => {
assert.deepEqual(proposedEvents(HANDLES), ['issues', 'pull_request', 'release']);
assert.deepEqual(neverSeen({ issues: 3 }, HANDLES), ['pull_request', 'release']);
const text = repair('wildcard', HANDLES, { issues: 3 });
assert.ok(text.includes('["issues","pull_request","release"]'));
assert.match(text, /Keep pull_request, release on the list/);
});
test('a tight hook gets no repair', () => {
assert.ok(repair('tight', HANDLES).startsWith('nothing'));
});
test('the cursor is read from the Link header', () => {
const header = '<https://api.github.com/repos/a/b/hooks/1/deliveries?cursor=v2>; rel="next"';
assert.ok(nextLink({ Link: header }).endsWith('cursor=v2'));
assert.equal(nextLink({}), null);
});
FAQ
Is a wildcard subscription ever the right answer?
Yes, for a receiver whose job is to see everything: an audit sink, an event lake, a mirror that archives activity without interpreting it. For those the open-ended property is the feature, because a new event type should join the archive automatically. What makes it wrong everywhere else is that the same property applies whether or not anybody wanted it, and a receiver with four handlers is not an archive. If you keep it deliberately, say so in the code, so the next person to audit the hook does not quietly undo the decision.
Nothing is failing. Why is this worth changing?
Because the cost is real and the trend is upward. Every unwanted delivery costs a connection, a body read, a signature computed over the raw bytes and a parse, before the handler can decide it does not care; that is most of the cost of a delivery that was wanted. It also means repository content, including commit messages and issue bodies, leaves GitHub for your endpoint whether or not anything reads it. And the volume grows on GitHub's release schedule, so the number in six months is not one anybody chose.
How is this different from the note about an unsubscribed event?
It is the same two sets subtracted the other way round. That note looks for events your handlers implement that the hook does not send, and its symptom is a handler that never runs. This one looks for events the hook sends that your handlers do not implement, and its symptom is nothing at all, which is why it has to be gone looking for rather than reported. A hook can easily be wrong in both directions at once, and the two findings have different repairs.
Can I build the replacement list from what actually arrived?
Only as a starting point, and the script deliberately does not. The delivery window covers a limited retention period, so an event that has not fired recently looks identical to one you do not need: a release handler on a quarterly release cycle disappears from any window shorter than a quarter. Build the list from what the receiver implements, use the tally to show what the wildcard is costing, and keep the handled events that produced nothing, which the script lists separately as a caution.
What about hooks on the organization rather than the repository?
The same wildcard applies and the same reasoning holds, with more volume behind it, since an org hook receives from every repository in the org. Reading them needs org-level access that a repository-scoped token does not have, which is a real blind spot: a repository audit that comes back clean says nothing about an org hook pointed at the same receiver. If you have the access, run the same check against the org hooks, and remember that a URL reached by both an org hook and a repo hook has a second problem as well.
Related field notes
- The hook is not subscribed to your event
- The same URL registered twice
- Deliveries failing where nobody reads the log
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Repository webhooks — GitHub REST API
- Webhook events and payloads — GitHub Docs
- Creating webhooks — GitHub Docs
- Best practices for using webhooks — GitHub Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.