Diagnostic Slack
The Request URL verified once, on a laptop that is now closed
The app configuration page has a green tick next to the Request URL. Somebody checked, twice. Event subscriptions are on, the scopes are right, the bot is in the channel, and nothing has arrived for four months.
Read the URL rather than the tick. It says https://a1b2c3d4.ngrok.io/slack/events, and it verified perfectly on the afternoon it was pasted in, from a laptop that has since been closed, reimaged and handed to somebody else. The tick records that the handshake succeeded once. It is not a claim about now.
A Request URL that verified is not a Request URL that works. Verification is a single exchange at configuration time; delivery is every day afterwards, to whatever that hostname resolves to today. Development tunnels satisfy Slack's requirements perfectly while they are running, so a URL pasted in during development verifies, gets forgotten, and leaves a production app whose configuration looks correct and whose events go nowhere.
The whole finding is in the string. The script exports the manifest, walks every surface that carries a URL — events, interactivity, message menu options, and each slash command — and classifies each host: not https, a known tunnel domain, an unroutable name or private address, a bare IP literal, or production-shaped. Then it lets Socket Mode downgrade the entire report, because an app on Socket Mode does not use these URLs at all. Nothing is sent to your endpoint. A detector that dials a production Request URL to see what answers has started participating in the incident it was asked to describe.
The problem in plain words
The thing that makes this expensive is that every screen you would think to check is green. The URL is present. It is https, usually. It was verified, and Slack remembers that it was. Event subscriptions are enabled and list the events you expect. There is no error state anywhere in the configuration, because from Slack's point of view nothing is wrong with the configuration: the URL is well formed and it passed its handshake.
And the app is often not new. It ran, in development, on the machine where the tunnel lived, and everybody who saw it saw it work. What shipped was the code, and the code was fine. What did not ship was the URL, because the URL is not in the repository and is not in the deployment and is not in the review. It is in a web form that one person filled in six months ago.
The variants all end the same way. A tunnel host that no longer resolves. A http:// URL that a colleague swears used to work and could never have verified. A hostname that resolves only inside the VPC, so it is reachable from every machine anybody tries it from and from nothing Slack owns. A certificate that expired last Tuesday. In each case the app is not broken and not misconfigured in any way an error message can express: it is pointed at an address that no longer answers.
Why it happens
Verification is a moment and delivery is a habit. Slack checks the URL once, when you save it, and then trusts it. There is no periodic revalidation, so the green tick is a fact about an afternoon in March and says nothing about today. Treating it as a health indicator is the mistake at the centre of this note.
A tunnel is indistinguishable from production while it is up. That is what makes it dangerous rather than obviously wrong. It presents a valid publicly trusted certificate, it answers the challenge, it delivers events, and every check anybody performs during development passes. The failure is deferred to the moment the process is killed, which is a moment nobody logs.
Slack has no notion of environments, so there is exactly one URL per app. There is no staging slot, no per-branch override, no environment variable. Whatever is in that field is what production uses. That single fact is why the repair is two apps rather than a better URL, and it is worth saying out loud because teams keep looking for the environment switch.
This is not the handshake failing. A URL that never verified is a different note with a different repair: you replay the exchange and find the redirect, the auth wall or the framework that swallowed the body. Here the exchange succeeded and the address went stale afterwards. Same field, opposite fault, and the diagnostic that solves one is useless for the other.
The detector must not send anything to the endpoint, and that is a design decision rather than a limitation. Your Request URL is a production endpoint that answers Slack. A script that dials it adds an unsigned request to whatever is already going wrong, may trip an alert, and proves less than the string does: a host that answers a bare GET can still be the wrong deployment. The classification and the records you already hold are both stronger evidence and free of side effects.
The fix, as a flow
Four months separate the third box from the fifth, and the third box is not a mistake: somebody closed a laptop. That is what makes this chain unusual to read, because there is no step in it anybody would have reviewed. The fix branch never leaves the manifest. Its two plain rows are both refusals: one because Socket Mode makes the whole question moot, and one because a hostname that looks right is not a hostname anybody dialled.
How to fix it
Export the manifest, or read one from a file
An app configuration token and apps.manifest.export return the whole configuration. If you do not have one, download the manifest from the app page and pass --manifest app.json; the check is entirely offline from that point, which also makes it something you can run in CI against a manifest in the repository.
Walk every surface, not just the events one
surfaces yields the event subscriptions URL, the interactivity URL, the message menu options URL and one row per slash command. These drift independently. The commonest half-repair in this whole area is fixing the events URL and leaving three slash commands pointed at a tunnel.
Classify each host from the string alone
url_verdict returns not-https, ephemeral-tunnel, unroutable, bare-ip or production-shaped. It sends nothing. production-shaped deliberately does not mean healthy; it means this script has no complaint about the address, which is a much smaller claim.
Let Socket Mode downgrade the whole report
If settings.socket_mode_enabled is true, none of these URLs is used for delivery: events, interactions and commands all arrive over the socket. The rows become dormant, still worth cleaning up before somebody turns Socket Mode off, but not the cause of today's silence.
Bring the records you already have
Two numbers from your own side settle it. --last-delivery is when your endpoint last logged a request from Slack, and --retry-reason is the X-Slack-Retry-Reason value on the last one it saw. connection_failed or ssl_error against a tunnel host is as conclusive as this gets.
Print the repair, including the part about two apps
Point every surface at the production hostname and re-verify. Then keep development in a separate Slack app with its own manifest, because there is no environment switch and the alternative is that somebody pastes a tunnel URL into the production app again next quarter.
How to check it worked
After the URLs are repointed and re-verified, re-run against the exported manifest. Every surface should read production-shaped, and the last delivery should be minutes old rather than months.
python3 slack_request_url_origin.py --manifest app.json --last-delivery 2026-08-31 \
--retry-reason none
# manifest 4 surface(s), socket mode off
# event subscriptions production-shaped hooks.acme.example is a routable public hostname
# interactivity production-shaped hooks.acme.example is a routable public hostname
# slash /deploy production-shaped hooks.acme.example is a routable public hostname
# records delivering a delivery was seen 0.0 day(s) ago
# verdict clear no configured URL points somewhere Slack cannot reachThe full code
Two pure functions and a manifest. url_verdict is the whole detector and it never opens a socket: it takes one string and returns which of six things that string is, which is more than a request to the endpoint would tell you and costs your production system nothing. surfaces exists because the events URL is the one everybody remembers and the slash commands are the ones that get left behind. record_verdict reads the two facts your own logs already hold, and verdict is where a suspicious hostname and a silent endpoint become one sentence.
"""Classify every Request URL an app has configured, without dialling any of them.
Read only, and deliberately more than that: this sends nothing at all to your
own endpoint. One optional apps.manifest.export on an app configuration token,
or a manifest file you exported yourself, and everything after that is string
work. A detector that dials a production Request URL to see what answers has
started participating in the incident it was asked to describe.
"""
import argparse
import datetime
import json
import logging
import os
import sys
import urllib.parse
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("slack_request_url_origin")
API = "https://slack.com/api/"
# Hosts handed out by development tunnels. Every one of these satisfies Slack
# completely while the tunnel is running, which is exactly why a URL pointing at
# one verifies, is forgotten, and then quietly stops existing.
TUNNEL_SUFFIXES = (
".ngrok.io", ".ngrok-free.app", ".ngrok.app", ".ngrok.dev",
".loca.lt", ".trycloudflare.com", ".serveo.net", ".localhost.run",
".lhr.life", ".telebit.io", ".pinggy.link", ".tunnelmole.net",
".devtunnels.ms", ".ngrok-free.dev",
)
# Names that resolve to something only you can see.
UNROUTABLE_NAMES = ("localhost", "localhost.localdomain")
UNROUTABLE_SUFFIXES = (".local", ".internal", ".localdomain", ".localhost",
".test", ".invalid", ".home.arpa")
# What Slack puts in X-Slack-Retry-Reason, and what each one means about the
# far end. These come out of your own access logs; none of them is produced by
# anything this script does.
RETRY_REASONS = {
"connection_failed": "the connection could not be established at all, which "
"is what a closed tunnel looks like from Slack's side",
"ssl_error": "the certificate could not be validated, which is what an "
"expired or self-signed certificate looks like",
"http_timeout": "the connection was accepted and nothing answered within "
"three seconds",
"http_error": "the endpoint answered with a status Slack does not accept",
}
RANK = {"not-https": 0, "unroutable": 1, "ephemeral-tunnel": 2, "bare-ip": 3,
"unparseable": 4, "no-url": 5, "production-shaped": 6}
FINDINGS = ("not-https", "unroutable", "ephemeral-tunnel", "bare-ip", "unparseable")
def _as_ip(host):
"""Parse an IP literal without a regex. Pure. -> (version, octets, text) or None"""
text = host.replace("[", "").replace("]", "")
parts = text.split(".")
if len(parts) == 4 and all(p.isdigit() and 0 < len(p) < 4 for p in parts):
nums = [int(p) for p in parts]
if all(n <= 255 for n in nums):
return (4, nums, text)
if ":" in text:
return (6, [], text)
return None
def _unroutable_ip(ip):
"""Whether an IP literal is one Slack could never reach. Pure.
The ranges are written out rather than taken from the standard library so
that this file and its Node counterpart give the same answer. They do not
otherwise: Python's private-address table has grown to include the
documentation ranges, and a reader comparing the two outputs on the same URL
deserves one verdict rather than an argument.
"""
version, nums, text = ip
if version == 6:
h = text.lower()
return (h in ("::1", "::") or h.startswith("fc") or h.startswith("fd")
or h.startswith("fe8") or h.startswith("fe9")
or h.startswith("fea") or h.startswith("feb"))
a, b = nums[0], nums[1]
return (a in (0, 10, 127) or (a == 192 and b == 168)
or (a == 169 and b == 254) or (a == 172 and 16 <= b <= 31))
def url_verdict(url):
"""Classify one configured URL from its text alone. Pure. -> (state, detail)
Six states, and production-shaped is the weakest of them on purpose: it
means this script has no complaint about the address, not that anything is
listening on it. The stronger claim would need a request, and making that
request is the one thing this script will not do.
"""
raw = str(url or "").strip()
if not raw:
return ("no-url", "no URL is configured for this surface")
try:
parsed = urllib.parse.urlsplit(raw)
host = (parsed.hostname or "").lower()
except ValueError:
return ("unparseable", "%s could not be parsed as a URL" % raw)
if not parsed.scheme or not host:
return ("unparseable", "%s is not a complete URL" % raw)
if parsed.scheme.lower() != "https":
return ("not-https",
"the scheme is %s. Slack requires https with a publicly trusted "
"certificate, so this URL cannot have verified in this form"
% parsed.scheme.lower())
for suffix in TUNNEL_SUFFIXES:
if host.endswith(suffix):
return ("ephemeral-tunnel",
"%s is a development tunnel host. It was reachable while the "
"tunnel was running, which is how it verified, and it stopped "
"existing when that process was killed" % host)
ip = _as_ip(host)
if ip is not None:
if _unroutable_ip(ip):
return ("unroutable",
"%s is not an address Slack can reach from the public "
"internet" % host)
return ("bare-ip",
"%s is a literal address rather than a name. Slack validates the "
"certificate for what it dials, and certificates issued to bare "
"addresses are rare enough to be worth confirming" % host)
if host in UNROUTABLE_NAMES or host.endswith(UNROUTABLE_SUFFIXES):
return ("unroutable",
"%s resolves inside a network Slack is not on" % host)
return ("production-shaped",
"%s is a routable public hostname. That is not the same as reachable, "
"and nothing was sent to it to find out" % host)
def surfaces(manifest):
"""Every place a manifest carries a URL. Pure. -> rows of (surface, url)
The events URL is the one everybody remembers. Slash commands are the ones
left behind, because there are several of them and they are on a different
screen, and a half-repair that fixes events and forgets three commands is
the commonest outcome in this whole area.
"""
m = manifest if isinstance(manifest, dict) else {}
settings = m.get("settings") or {}
features = m.get("features") or {}
rows = [
("event subscriptions",
(settings.get("event_subscriptions") or {}).get("request_url")),
("interactivity", (settings.get("interactivity") or {}).get("request_url")),
("message menu options",
(settings.get("interactivity") or {}).get("message_menu_options_url")),
]
for cmd in features.get("slash_commands") or []:
if not isinstance(cmd, dict):
continue
rows.append(("slash %s" % (cmd.get("command") or "?"), cmd.get("url")))
return [(name, url) for name, url in rows if url is not None]
def audit(manifest):
"""Classify every surface, and mark the rows Socket Mode makes moot. Pure.
Returns rows of (surface, url, state, detail, live), worst first. When
Socket Mode is on, events, interactions and commands all arrive over the
socket and none of these URLs is used, so a tunnel host left in the field is
a tidiness problem rather than the cause of today's silence.
"""
m = manifest if isinstance(manifest, dict) else {}
socket = bool((m.get("settings") or {}).get("socket_mode_enabled"))
rows = []
for name, url in surfaces(m):
state, detail = url_verdict(url)
if socket and state in FINDINGS:
detail += (". Socket Mode is on, so this URL is not used for delivery "
"and this is a cleanup rather than an outage")
rows.append((name, url, state, detail, not socket))
return sorted(rows, key=lambda r: (RANK.get(r[2], 9), r[0]))
def record_verdict(days_silent, retry_reason=None):
"""What your own records already say about the far end. Pure.
Both inputs come from your side of the connection: when your endpoint last
logged a request from Slack, and the X-Slack-Retry-Reason on the last one it
saw. Neither requires sending anything to the URL.
"""
reason = str(retry_reason or "").strip().lower()
if reason and reason not in ("none", "-"):
if reason in RETRY_REASONS:
return ("reason-recorded", "%s: %s" % (reason, RETRY_REASONS[reason]))
return ("unknown-reason",
"%s is not a retry reason this script knows; read it against the "
"Events API documentation rather than assuming it is benign" % reason)
if days_silent is None:
return ("no-records",
"no delivery record was supplied, so the classification above "
"stands on the URL alone")
try:
days = float(days_silent)
except (TypeError, ValueError):
return ("no-records", "the last delivery date could not be read")
if days >= 1.0:
return ("silent",
"nothing has been delivered for %.1f day(s)" % days)
return ("delivering", "a delivery was seen %.1f day(s) ago" % days)
def verdict(rows, record_state):
"""Turn a suspicious address and a silent endpoint into one sentence. Pure."""
live = [r for r in rows or [] if r[4] and r[2] in FINDINGS]
dormant = [r for r in rows or [] if not r[4] and r[2] in FINDINGS]
corroborated = record_state in ("silent", "reason-recorded")
if live and corroborated:
return ("dead-endpoint",
"%d surface(s) point somewhere Slack cannot reach, and your own "
"records agree that nothing is arriving" % len(live))
if live:
return ("suspect-origin",
"%d surface(s) point at an address that cannot serve production. "
"Pass --last-delivery to turn this into a closed case" % len(live))
if dormant:
return ("dormant",
"%d surface(s) carry an address that could not serve production, "
"but Socket Mode is on and none of them is used for delivery"
% len(dormant))
if corroborated:
return ("elsewhere",
"every configured URL is production-shaped and nothing is arriving "
"anyway. The endpoint, the handshake or a disabled subscription is "
"the next place to look")
return ("clear", "no configured URL points somewhere Slack cannot reach")
def get(session, method, params, label):
"""One GET, asserting on the body rather than the status line."""
r = session.get(API + method, params=params or {}, timeout=30)
body = r.json()
if body.get("ok") is not True:
log.warning("%-10s %-14s %s", label, "unavailable", body.get("error"))
return None
return body
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--manifest", default="",
help="path to a manifest exported from the app page")
ap.add_argument("--config-token-env", default="SLACK_CONFIG_TOKEN",
help="environment variable holding an app configuration token")
ap.add_argument("--app-id", default="", help="app ID, for the manifest read")
ap.add_argument("--last-delivery", default="",
help="YYYY-MM-DD your endpoint last logged a request from Slack")
ap.add_argument("--retry-reason", default="",
help="the X-Slack-Retry-Reason on the last delivery you saw")
args = ap.parse_args()
manifest = None
if args.manifest:
manifest = json.loads(open(args.manifest, encoding="utf-8").read())
manifest = manifest.get("manifest", manifest)
else:
token = os.environ.get(args.config_token_env)
if not token or not args.app_id:
log.error("supply --manifest, or set %s and pass --app-id; a bot token "
"cannot read app configuration", args.config_token_env)
return 2
s = requests.Session()
s.headers.update({"Authorization": "Bearer " + token})
body = get(s, "apps.manifest.export", {"app_id": args.app_id}, "manifest")
if not body:
return 2
manifest = body.get("manifest") or {}
rows = audit(manifest)
socket = bool((manifest.get("settings") or {}).get("socket_mode_enabled"))
log.info("manifest %d surface(s), socket mode %s", len(rows),
"on" if socket else "off")
for name, url, state, detail, live in rows:
line = "%-21s %-18s %s" % (name, state if live else "dormant", detail)
if state in FINDINGS and live:
log.warning(line)
else:
log.info(line)
log.info(" %-19s %s", "", url)
days = None
if args.last_delivery:
seen = datetime.datetime.strptime(args.last_delivery, "%Y-%m-%d").date()
days = (datetime.date.today() - seen).days
rstate, rdetail = record_verdict(days, args.retry_reason)
log.info("records %-14s %s", rstate, rdetail)
state, detail = verdict(rows, rstate)
if state in ("dead-endpoint", "suspect-origin"):
log.warning("verdict %-14s %s", state, detail)
log.warning(" repair: point every surface above at the production hostname "
"and re-verify; the tick records one exchange, not a health check")
log.warning(" repair: keep development in a separate Slack app with its own "
"manifest. Slack has no environments, so two apps is the only "
"clean separation")
log.warning(" repair: put certificate expiry on the same monitor that "
"watches the endpoint, because an expired certificate reads as "
"ssl_error and nothing else")
return 1
log.info("verdict %-14s %s", state, detail)
return 0
if __name__ == "__main__":
sys.exit(main())
/**
* Classify every Request URL an app has configured, without dialling any of them.
*
* Read only, and deliberately more than that: this sends nothing at all to your
* own endpoint. One optional apps.manifest.export on an app configuration
* token, or a manifest file you exported yourself, and everything after that is
* string work.
*/
import { readFile } from 'node:fs/promises';
const API = 'https://slack.com/api/';
// Hosts handed out by development tunnels. Every one satisfies Slack completely
// while the tunnel is running, which is why a URL pointing at one verifies, is
// forgotten, and then quietly stops existing.
const TUNNEL_SUFFIXES = [
'.ngrok.io', '.ngrok-free.app', '.ngrok.app', '.ngrok.dev',
'.loca.lt', '.trycloudflare.com', '.serveo.net', '.localhost.run',
'.lhr.life', '.telebit.io', '.pinggy.link', '.tunnelmole.net',
'.devtunnels.ms', '.ngrok-free.dev',
];
const UNROUTABLE_NAMES = ['localhost', 'localhost.localdomain'];
const UNROUTABLE_SUFFIXES = ['.local', '.internal', '.localdomain', '.localhost',
'.test', '.invalid', '.home.arpa'];
const RETRY_REASONS = new Map([
['connection_failed', 'the connection could not be established at all, which is ' +
"what a closed tunnel looks like from Slack's side"],
['ssl_error', 'the certificate could not be validated, which is what an expired ' +
'or self-signed certificate looks like'],
['http_timeout', 'the connection was accepted and nothing answered within three ' +
'seconds'],
['http_error', 'the endpoint answered with a status Slack does not accept'],
]);
const RANK = {
'not-https': 0, unroutable: 1, 'ephemeral-tunnel': 2, 'bare-ip': 3,
unparseable: 4, 'no-url': 5, 'production-shaped': 6,
};
const FINDINGS = ['not-https', 'unroutable', 'ephemeral-tunnel', 'bare-ip',
'unparseable'];
/** Parse an IP literal without a regex. Pure. -> {v, nums} or null */
function asIp(host) {
const h = host.split('[').join('').split(']').join('');
const parts = h.split('.');
const digits = (p) => p.length > 0 && p.length < 4
&& [...p].every((c) => c >= '0' && c <= '9');
if (parts.length === 4 && parts.every(digits)) {
const nums = parts.map(Number);
if (nums.every((n) => n <= 255)) return { v: 4, nums, host: h };
}
if (h.includes(':')) return { v: 6, nums: [], host: h };
return null;
}
/** Whether an IP literal is one Slack could never reach. Pure. */
function unroutableIp(ip) {
if (ip.v === 6) {
const h = ip.host.toLowerCase();
return h === '::1' || h === '::' || h.startsWith('fc') || h.startsWith('fd')
|| h.startsWith('fe8') || h.startsWith('fe9') || h.startsWith('fea')
|| h.startsWith('feb');
}
const [a, b] = ip.nums;
return a === 10 || a === 127 || a === 0
|| (a === 192 && b === 168) || (a === 169 && b === 254)
|| (a === 172 && b >= 16 && b <= 31);
}
/**
* Classify one configured URL from its text alone. Pure. -> [state, detail]
* production-shaped is the weakest state on purpose: it means this script has
* no complaint about the address, not that anything is listening on it.
*/
export function urlVerdict(url) {
const raw = String(url ?? '').trim();
if (!raw) return ['no-url', 'no URL is configured for this surface'];
let parsed;
try {
parsed = new URL(raw);
} catch {
return ['unparseable', `${raw} could not be parsed as a URL`];
}
const host = (parsed.hostname ?? '').toLowerCase();
if (!host) return ['unparseable', `${raw} is not a complete URL`];
const scheme = parsed.protocol.replace(':', '').toLowerCase();
if (scheme !== 'https') {
return ['not-https',
`the scheme is ${scheme}. Slack requires https with a publicly trusted ` +
'certificate, so this URL cannot have verified in this form'];
}
if (TUNNEL_SUFFIXES.some((s) => host.endsWith(s))) {
return ['ephemeral-tunnel',
`${host} is a development tunnel host. It was reachable while the tunnel ` +
'was running, which is how it verified, and it stopped existing when that ' +
'process was killed'];
}
const ip = asIp(host);
if (ip) {
if (unroutableIp(ip)) {
return ['unroutable',
`${host} is not an address Slack can reach from the public internet`];
}
return ['bare-ip',
`${host} is a literal address rather than a name. Slack validates the ` +
'certificate for what it dials, and certificates issued to bare addresses ' +
'are rare enough to be worth confirming'];
}
if (UNROUTABLE_NAMES.includes(host)
|| UNROUTABLE_SUFFIXES.some((s) => host.endsWith(s))) {
return ['unroutable', `${host} resolves inside a network Slack is not on`];
}
return ['production-shaped',
`${host} is a routable public hostname. That is not the same as reachable, ` +
'and nothing was sent to it to find out'];
}
/**
* Every place a manifest carries a URL. Pure. -> rows of [surface, url]
* Slash commands are the ones left behind, because there are several of them
* and they are on a different screen.
*/
export function surfaces(manifest) {
const m = (manifest && typeof manifest === 'object') ? manifest : {};
const settings = m.settings ?? {};
const features = m.features ?? {};
const rows = [
['event subscriptions', settings.event_subscriptions?.request_url],
['interactivity', settings.interactivity?.request_url],
['message menu options', settings.interactivity?.message_menu_options_url],
];
for (const cmd of features.slash_commands ?? []) {
if (!cmd || typeof cmd !== 'object') continue;
rows.push([`slash ${cmd.command ?? '?'}`, cmd.url]);
}
return rows.filter(([, url]) => url !== undefined && url !== null);
}
/**
* Classify every surface, and mark the rows Socket Mode makes moot. Pure.
* Returns rows of [surface, url, state, detail, live], worst first.
*/
export function audit(manifest) {
const m = (manifest && typeof manifest === 'object') ? manifest : {};
const socket = Boolean(m.settings?.socket_mode_enabled);
const rows = surfaces(m).map(([name, url]) => {
const [state, base] = urlVerdict(url);
const detail = (socket && FINDINGS.includes(state))
? `${base}. Socket Mode is on, so this URL is not used for delivery and ` +
'this is a cleanup rather than an outage'
: base;
return [name, url, state, detail, !socket];
});
return rows.sort((a, b) => (RANK[a[2]] ?? 9) - (RANK[b[2]] ?? 9)
|| a[0].localeCompare(b[0]));
}
/** What your own records already say about the far end. Pure. */
export function recordVerdict(daysSilent, retryReason = null) {
const reason = String(retryReason ?? '').trim().toLowerCase();
if (reason && reason !== 'none' && reason !== '-') {
if (RETRY_REASONS.has(reason)) {
return ['reason-recorded', `${reason}: ${RETRY_REASONS.get(reason)}`];
}
return ['unknown-reason',
`${reason} is not a retry reason this script knows; read it against the ` +
'Events API documentation rather than assuming it is benign'];
}
if (daysSilent === null || daysSilent === undefined || daysSilent === '') {
return ['no-records',
'no delivery record was supplied, so the classification above stands on ' +
'the URL alone'];
}
const days = Number(daysSilent);
if (!Number.isFinite(days)) {
return ['no-records', 'the last delivery date could not be read'];
}
if (days >= 1) {
return ['silent', `nothing has been delivered for ${days.toFixed(1)} day(s)`];
}
return ['delivering', `a delivery was seen ${days.toFixed(1)} day(s) ago`];
}
/** Turn a suspicious address and a silent endpoint into one sentence. Pure. */
export function verdict(rows, recordState) {
const live = (rows ?? []).filter((r) => r[4] && FINDINGS.includes(r[2]));
const dormant = (rows ?? []).filter((r) => !r[4] && FINDINGS.includes(r[2]));
const corroborated = recordState === 'silent' || recordState === 'reason-recorded';
if (live.length && corroborated) {
return ['dead-endpoint',
`${live.length} surface(s) point somewhere Slack cannot reach, and your own ` +
'records agree that nothing is arriving'];
}
if (live.length) {
return ['suspect-origin',
`${live.length} surface(s) point at an address that cannot serve ` +
'production. Pass --last-delivery to turn this into a closed case'];
}
if (dormant.length) {
return ['dormant',
`${dormant.length} surface(s) carry an address that could not serve ` +
'production, but Socket Mode is on and none of them is used for delivery'];
}
if (corroborated) {
return ['elsewhere',
'every configured URL is production-shaped and nothing is arriving anyway. ' +
'The endpoint, the handshake or a disabled subscription is the next place ' +
'to look'];
}
return ['clear', 'no configured URL points somewhere Slack cannot reach'];
}
async function get(token, method, params, label) {
const qs = new URLSearchParams(params ?? {});
const res = await fetch(`${API}${method}?${qs}`, {
headers: { Authorization: `Bearer ${token}` },
});
const body = await res.json();
if (body.ok !== true) {
console.warn(`${label.padEnd(10)} ${'unavailable'.padEnd(14)} ${body.error}`);
return null;
}
return body;
}
function arg(args, name, fallback = null) {
const i = args.indexOf(name);
return i === -1 ? fallback : args[i + 1];
}
async function main() {
const args = process.argv.slice(2);
const manifestPath = arg(args, '--manifest', '');
const configTokenEnv = arg(args, '--config-token-env', 'SLACK_CONFIG_TOKEN');
const appId = arg(args, '--app-id', '');
let manifest = null;
if (manifestPath) {
const parsed = JSON.parse(await readFile(manifestPath, 'utf8'));
manifest = parsed.manifest ?? parsed;
} else {
const token = process.env[configTokenEnv];
if (!token || !appId) {
console.error(`supply --manifest, or set ${configTokenEnv} and pass --app-id; ` +
'a bot token cannot read app configuration');
process.exitCode = 2;
return;
}
const body = await get(token, 'apps.manifest.export', { app_id: appId },
'manifest');
if (!body) { process.exitCode = 2; return; }
manifest = body.manifest ?? {};
}
const rows = audit(manifest);
const socket = Boolean(manifest.settings?.socket_mode_enabled);
console.log(`manifest ${rows.length} surface(s), socket mode ` +
`${socket ? 'on' : 'off'}`);
for (const [name, url, state, detail, live] of rows) {
const line = `${name.padEnd(21)} ${(live ? state : 'dormant').padEnd(18)} ${detail}`;
if (FINDINGS.includes(state) && live) console.warn(line); else console.log(line);
console.log(` ${''.padEnd(19)} ${url}`);
}
const lastDelivery = arg(args, '--last-delivery', '');
let days = null;
if (lastDelivery) {
days = (Date.now() - Date.parse(`${lastDelivery}T00:00:00Z`)) / 86400000;
}
const [rstate, rdetail] = recordVerdict(days, arg(args, '--retry-reason', ''));
console.log(`records ${rstate.padEnd(14)} ${rdetail}`);
const [state, detail] = verdict(rows, rstate);
if (state === 'dead-endpoint' || state === 'suspect-origin') {
console.warn(`verdict ${state.padEnd(14)} ${detail}`);
console.warn(' repair: point every surface above at the production hostname ' +
'and re-verify; the tick records one exchange, not a health check');
console.warn(' repair: keep development in a separate Slack app with its own ' +
'manifest. Slack has no environments, so two apps is the only clean ' +
'separation');
console.warn(' repair: put certificate expiry on the same monitor that watches ' +
'the endpoint, because an expired certificate reads as ssl_error and ' +
'nothing else');
process.exitCode = 1;
} else {
console.log(`verdict ${state.padEnd(14)} ${detail}`);
}
}
// Only run when invoked directly, so importing this module in the tests does
// not execute main() and fail the file on a missing manifest.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The assertions are mostly about not overclaiming. A public hostname has to come back production-shaped and nothing stronger, because the script did not dial it and must not imply that it did. Socket Mode has to demote every finding to dormant, since an app on a socket is not using these fields at all and reporting an outage there would be a fabrication. And a slash command URL has to be classified alongside the events URL, because the half-repair that fixes one and forgets the others is the failure mode this note exists to prevent twice.
from slack_request_url_origin import audit, record_verdict, surfaces, url_verdict, verdict
def manifest(events=None, socket=False, commands=None, interactivity=None):
return {
"settings": {
"event_subscriptions": {"request_url": events} if events else {},
"interactivity": interactivity or {},
"socket_mode_enabled": socket,
},
"features": {"slash_commands": commands or []},
}
def test_a_tunnel_host_is_the_finding():
state, detail = url_verdict("https://a1b2c3d4.ngrok.io/slack/events")
assert state == "ephemeral-tunnel"
assert "stopped existing" in detail
def test_every_tunnel_vendor_in_the_table_is_recognised():
for host in ("x.ngrok-free.app", "y.loca.lt", "z.trycloudflare.com",
"q.serveo.net", "r.devtunnels.ms"):
assert url_verdict("https://%s/slack/events" % host)[0] == "ephemeral-tunnel"
def test_plain_http_could_never_have_verified():
state, detail = url_verdict("http://hooks.acme.example/slack/events")
assert state == "not-https"
assert "cannot have verified" in detail
def test_localhost_and_private_addresses_are_unroutable():
assert url_verdict("https://localhost:3000/slack/events")[0] == "unroutable"
assert url_verdict("https://api.internal/slack/events")[0] == "unroutable"
assert url_verdict("https://10.0.4.12/slack/events")[0] == "unroutable"
assert url_verdict("https://192.168.1.9/slack/events")[0] == "unroutable"
assert url_verdict("https://172.16.0.1/slack/events")[0] == "unroutable"
assert url_verdict("https://127.0.0.1/slack/events")[0] == "unroutable"
def test_a_public_ip_literal_is_flagged_for_its_certificate_not_its_routing():
state, detail = url_verdict("https://203.0.113.44/slack/events")
assert state == "bare-ip"
assert "certificate" in detail
def test_a_public_hostname_is_never_called_healthy():
state, detail = url_verdict("https://hooks.acme.example/slack/events")
assert state == "production-shaped"
assert "not the same as reachable" in detail
assert "nothing was sent to it" in detail
def test_an_empty_or_broken_url_is_said_so_rather_than_guessed():
assert url_verdict("")[0] == "no-url"
assert url_verdict(None)[0] == "no-url"
assert url_verdict("not a url at all")[0] == "unparseable"
def test_slash_commands_are_surfaces_too():
m = manifest(events="https://hooks.acme.example/e",
commands=[{"command": "/deploy",
"url": "https://a1.ngrok.io/cmd"}])
names = [n for n, _ in surfaces(m)]
assert "event subscriptions" in names
assert "slash /deploy" in names
def test_the_forgotten_slash_command_outranks_the_healthy_events_url():
m = manifest(events="https://hooks.acme.example/e",
commands=[{"command": "/deploy", "url": "https://a1.ngrok.io/cmd"}])
rows = audit(m)
assert rows[0][0] == "slash /deploy"
assert rows[0][2] == "ephemeral-tunnel"
def test_interactivity_and_menu_urls_are_read_as_well():
m = manifest(interactivity={"request_url": "https://a1.ngrok.io/i",
"message_menu_options_url": "http://x.example/m"})
states = {n: s for n, _, s, _, _ in audit(m)}
assert states["interactivity"] == "ephemeral-tunnel"
assert states["message menu options"] == "not-https"
def test_socket_mode_demotes_every_finding_to_dormant():
m = manifest(events="https://a1.ngrok.io/slack/events", socket=True)
rows = audit(m)
assert rows[0][2] == "ephemeral-tunnel"
assert rows[0][4] is False
assert "cleanup rather than an outage" in rows[0][3]
assert verdict(rows, "no-records")[0] == "dormant"
def test_a_retry_reason_is_translated_rather_than_echoed():
state, detail = record_verdict(None, "connection_failed")
assert state == "reason-recorded"
assert "closed tunnel" in detail
assert record_verdict(None, "ssl_error")[0] == "reason-recorded"
def test_an_unknown_retry_reason_is_not_waved_through():
state, detail = record_verdict(None, "something_new")
assert state == "unknown-reason"
assert "rather than assuming it is benign" in detail
def test_silence_is_measured_in_days_and_none_means_no_records():
assert record_verdict(120)[0] == "silent"
assert record_verdict(0.0)[0] == "delivering"
assert record_verdict(None)[0] == "no-records"
assert record_verdict("yesterday")[0] == "no-records"
def test_a_dead_tunnel_plus_a_silent_endpoint_closes_the_case():
rows = audit(manifest(events="https://a1.ngrok.io/slack/events"))
assert verdict(rows, "silent")[0] == "dead-endpoint"
assert verdict(rows, "reason-recorded")[0] == "dead-endpoint"
def test_a_suspect_url_alone_asks_for_the_records():
rows = audit(manifest(events="https://a1.ngrok.io/slack/events"))
state, detail = verdict(rows, "no-records")
assert state == "suspect-origin"
assert "--last-delivery" in detail
def test_a_clean_url_and_a_silent_endpoint_is_somebody_else_s_note():
rows = audit(manifest(events="https://hooks.acme.example/slack/events"))
state, detail = verdict(rows, "silent")
assert state == "elsewhere"
assert "handshake" in detail
def test_a_clean_url_and_live_delivery_is_clear():
rows = audit(manifest(events="https://hooks.acme.example/slack/events"))
assert verdict(rows, "delivering")[0] == "clear"
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
audit, recordVerdict, surfaces, urlVerdict, verdict,
} from './slack-request-url-origin.mjs';
const manifest = ({
events = null, socket = false, commands = [], interactivity = {},
} = {}) => ({
settings: {
event_subscriptions: events ? { request_url: events } : {},
interactivity,
socket_mode_enabled: socket,
},
features: { slash_commands: commands },
});
test('a tunnel host is the finding', () => {
const [state, detail] = urlVerdict('https://a1b2c3d4.ngrok.io/slack/events');
assert.equal(state, 'ephemeral-tunnel');
assert.match(detail, /stopped existing/);
});
test('every tunnel vendor in the table is recognised', () => {
for (const host of ['x.ngrok-free.app', 'y.loca.lt', 'z.trycloudflare.com',
'q.serveo.net', 'r.devtunnels.ms']) {
assert.equal(urlVerdict(`https://${host}/slack/events`)[0], 'ephemeral-tunnel');
}
});
test('plain http could never have verified', () => {
const [state, detail] = urlVerdict('http://hooks.acme.example/slack/events');
assert.equal(state, 'not-https');
assert.match(detail, /cannot have verified/);
});
test('localhost and private addresses are unroutable', () => {
for (const url of ['https://localhost:3000/slack/events',
'https://api.internal/slack/events', 'https://10.0.4.12/slack/events',
'https://192.168.1.9/slack/events', 'https://172.16.0.1/slack/events',
'https://127.0.0.1/slack/events']) {
assert.equal(urlVerdict(url)[0], 'unroutable', url);
}
});
test('a public ip literal is flagged for its certificate not its routing', () => {
const [state, detail] = urlVerdict('https://203.0.113.44/slack/events');
assert.equal(state, 'bare-ip');
assert.match(detail, /certificate/);
});
test('a public hostname is never called healthy', () => {
const [state, detail] = urlVerdict('https://hooks.acme.example/slack/events');
assert.equal(state, 'production-shaped');
assert.match(detail, /not the same as reachable/);
assert.match(detail, /nothing was sent to it/);
});
test('an empty or broken url is said so rather than guessed', () => {
assert.equal(urlVerdict('')[0], 'no-url');
assert.equal(urlVerdict(null)[0], 'no-url');
assert.equal(urlVerdict('not a url at all')[0], 'unparseable');
});
test('slash commands are surfaces too', () => {
const m = manifest({
events: 'https://hooks.acme.example/e',
commands: [{ command: '/deploy', url: 'https://a1.ngrok.io/cmd' }],
});
const names = surfaces(m).map(([n]) => n);
assert.ok(names.includes('event subscriptions'));
assert.ok(names.includes('slash /deploy'));
});
test('the forgotten slash command outranks the healthy events url', () => {
const m = manifest({
events: 'https://hooks.acme.example/e',
commands: [{ command: '/deploy', url: 'https://a1.ngrok.io/cmd' }],
});
const rows = audit(m);
assert.equal(rows[0][0], 'slash /deploy');
assert.equal(rows[0][2], 'ephemeral-tunnel');
});
test('interactivity and menu urls are read as well', () => {
const m = manifest({
interactivity: {
request_url: 'https://a1.ngrok.io/i',
message_menu_options_url: 'http://x.example/m',
},
});
const states = Object.fromEntries(audit(m).map(([n, , s]) => [n, s]));
assert.equal(states.interactivity, 'ephemeral-tunnel');
assert.equal(states['message menu options'], 'not-https');
});
test('socket mode demotes every finding to dormant', () => {
const rows = audit(manifest({
events: 'https://a1.ngrok.io/slack/events', socket: true,
}));
assert.equal(rows[0][2], 'ephemeral-tunnel');
assert.equal(rows[0][4], false);
assert.match(rows[0][3], /cleanup rather than an outage/);
assert.equal(verdict(rows, 'no-records')[0], 'dormant');
});
test('a retry reason is translated rather than echoed', () => {
const [state, detail] = recordVerdict(null, 'connection_failed');
assert.equal(state, 'reason-recorded');
assert.match(detail, /closed tunnel/);
assert.equal(recordVerdict(null, 'ssl_error')[0], 'reason-recorded');
});
test('an unknown retry reason is not waved through', () => {
const [state, detail] = recordVerdict(null, 'something_new');
assert.equal(state, 'unknown-reason');
assert.match(detail, /rather than assuming it is benign/);
});
test('silence is measured in days and none means no records', () => {
assert.equal(recordVerdict(120)[0], 'silent');
assert.equal(recordVerdict(0)[0], 'delivering');
assert.equal(recordVerdict(null)[0], 'no-records');
assert.equal(recordVerdict('yesterday')[0], 'no-records');
});
test('a dead tunnel plus a silent endpoint closes the case', () => {
const rows = audit(manifest({ events: 'https://a1.ngrok.io/slack/events' }));
assert.equal(verdict(rows, 'silent')[0], 'dead-endpoint');
assert.equal(verdict(rows, 'reason-recorded')[0], 'dead-endpoint');
});
test('a suspect url alone asks for the records', () => {
const rows = audit(manifest({ events: 'https://a1.ngrok.io/slack/events' }));
const [state, detail] = verdict(rows, 'no-records');
assert.equal(state, 'suspect-origin');
assert.match(detail, /--last-delivery/);
});
test('a clean url and a silent endpoint is somebody elses note', () => {
const rows = audit(manifest({ events: 'https://hooks.acme.example/slack/events' }));
const [state, detail] = verdict(rows, 'silent');
assert.equal(state, 'elsewhere');
assert.match(detail, /handshake/);
});
test('a clean url and live delivery is clear', () => {
const rows = audit(manifest({ events: 'https://hooks.acme.example/slack/events' }));
assert.equal(verdict(rows, 'delivering')[0], 'clear');
});
FAQ
The URL is verified and has a green tick. Does that not mean it works?
It means the handshake succeeded on the day it was saved. Slack does not revalidate a Request URL on a schedule, so the tick is a record of one exchange rather than a health check. A URL that verified on a development tunnel keeps its tick forever, including for the four months after the tunnel stopped existing.
Why will this script not just make a request to my endpoint and tell me?
Because your Request URL is a production endpoint and a detector should not add traffic to an incident it was asked to describe. An unsigned request could trip an alert or a rate limiter, and it proves less than you would think: a host that answers a bare GET can still be the wrong deployment, and one that refuses it can be perfectly healthy behind a signature check. The hostname and your own access log are better evidence and cost nothing.
How do I run development and production without swapping this field?
Two Slack apps. Slack has no notion of environments, no staging slot and no per-branch override, so the only clean separation is a second app with its own manifest carrying the tunnel URL while the production app carries the production hostname. Any workflow that involves editing the production app's URL to debug something will eventually leave it edited.
Everything says production-shaped and events still are not arriving. Now what?
Then the address is not the problem and the script says so rather than inventing one. Three things sit behind it: the endpoint never passed the handshake at all, which is a separate note about the challenge exchange; Slack disabled your event subscriptions after sustained failures, which is another; or the events you expect were never subscribed or never scoped. The clear verdict is meant to move you along.
What is ssl_error and why does it appear months after everything was fine?
It is the retry reason Slack records when it cannot validate your certificate, and the usual cause is expiry rather than misconfiguration. The certificate was valid when the URL verified and stopped being valid at renewal time, so delivery fails without a single line of your configuration changing. Put certificate expiry on the same monitor that watches the endpoint.
Related field notes
- the handshake this URL already passed, once
- where an endpoint that stops answering ends up
- the credential that reads the manifest, and its twelve hours
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- The Events API - Slack Docs
- apps.manifest.export method reference - Slack Docs
- Handling user interaction - Slack Docs
- Security best practices - Slack Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.