Diagnostic Slack
ekm_access_denied: the customer holds the keys
{"ok": false, "error": "ekm_access_denied"}, and the human-readable version Slack prints beside it: “Your message couldn't be sent because your admins have disabled sending messages to this channel.” It affects eleven channels out of two hundred. It started on a Thursday. Nothing you deployed went out that week.
Every instinct says scope. It is not a scope, and no grant you can request will change it, because this refusal is not about your app at all: it is about a key. The customer's organization holds its own encryption keys for its Slack content, somebody revoked access to some of them, and Slack is now declining to decrypt that content for anybody — including you, with a perfectly valid token and every scope you asked for.
Enterprise Key Management lets an organization encrypt its Slack content with keys it controls: messages, files, the search index, and the messages and files generated by apps and bots. When access to a key is revoked, Slack refuses the content it protects. The error is ekm_access_denied, and it appears in the documented error list of read methods as well as writes — conversations.history, conversations.info and admin.teams.list all carry it, alongside chat.postMessage.
That matters, because it means the extent is measurable read-only. Two reads per channel — conversations.info for the metadata and conversations.history?limit=1 for the content — produce a map of what your app can still see, and the two layers can disagree: a channel whose metadata answers while its history is refused is the characteristic shape of content held under a key you have lost access to.
Do not retry. This is a decision recorded in a customer's key management, not a condition that clears. The tenth attempt is refused identically, and a retry loop turns a policy into rate-limit pressure and a job that never finishes. Skip the affected channels, complete the run over the rest, and report the affected set once.
And be careful about the words used for the extent. Slack documents that access can be revoked granularly and does not publish the exact levels, so the shape in the output below is an inference from what was observed, and labelled as one.
The problem in plain words
The first difficulty is that it reads like every other refusal. ekm_access_denied arrives in the same envelope as missing_scope and not_in_channel, inside an HTTP 200, and lands in the same catch block. A team that has been trained by this API to respond to a refusal by checking the scope list will check the scope list, find nothing wrong, reinstall, find nothing changed, and start looking for the bug they introduced. There is no bug. There is nothing on your side to find.
The second is the partial extent. If everything failed you would have an outage and an obvious cause; instead eleven channels out of two hundred fail while the rest are fine, which looks exactly like a data problem in your own code. Engineers reach for the channels themselves — are they archived, private, externally shared, unusually old — and the shared property is not a property of the channels at all, it is which key protects them.
The third is retry behaviour, and it is the expensive one. A refusal that arrives intermittently across a run trains a queue into retrying, and the retries are refused identically because nothing about the situation changes between attempts. A job that would have finished in four minutes runs for an hour, burns quota that other calls needed, and ends with the same eleven failures it started with. Worse, the retry noise buries a genuine transient failure somewhere in the same log.
The fourth is escalation. This is one of the few problems in the section where the correct response is a message to somebody else's administrator, and the message has to contain enough for them to act: which channels, which workspace, and the specific error string, because from their side the question is which key was revoked and when. A monitoring alert that says "Slack errors elevated" cannot start that conversation. A line that names eleven channels in one workspace and one error string can.
Why it happens
Two reads per channel, because the two layers answer different questions. conversations.info asks whether the channel's metadata is visible; conversations.history?limit=1 asks whether its content is. Both document ekm_access_denied. When the first answers and the second does not, you are looking at content protected by a key rather than at a channel you cannot see, and that distinction is what stops this being misfiled as a membership or scope problem.
The other refusals are separated out, and one of them stays ambiguous on purpose. not_in_channel means invite the bot; missing_scope means grant the scope. channel_not_found is genuinely undecidable from outside: a channel that does not exist and a private channel your token cannot see return the same thing, and a script that picks one is confidently wrong about half the time. It is reported as ambiguous and left that way.
The extent is an inference and is labelled an inference. Slack documents that an organization can revoke granular access to its keys and does not publish the granularity, so “this looks workspace-wide” is a description of the rows collected, not a claim about how the customer's key policy is configured. The function that produces it returns a confidence alongside the shape, and the confidence is always the same value, which is the honest one.
Retry classification is a first-class output rather than a note in the prose. retry_class puts ekm_access_denied in do-not-retry next to the errors that genuinely should back off, so the difference is expressed in code that your job can import rather than in a paragraph somebody has to remember. A policy is not a transient condition and treating it as one is the most common expensive mistake here.
The output is built to be forwarded. The affected set, the workspace, the error string and the shape are printed as a block because the next step is a message to the customer's administrator, and the useful version of that message is specific. No app-side change affects this, so anything the script prints about your configuration would be a distraction.
Everything is a read, and the write that produced the original symptom is not repeated. The complaint that starts this investigation is usually a failed chat.postMessage. The script never posts to find out whether posting works; it maps what can be read, which is enough to establish the extent and costs the workspace nothing.
The fix, as a flow
The red arrow is in the middle and it separates two things that look identical from inside a catch block. Everything to the left is an ordinary working integration; the arrow is a key being revoked somewhere inside the customer's organization, which is an event your app has no view of and no part in. What follows is the expensive habit rather than the refusal itself: a job that retries a decision runs for an hour, burns quota that other calls needed, and ends with the same eleven failures. The fix branch sorts channels by what each of two reads returned, because the pair is the diagnosis: a channel whose metadata answers while its content is refused is plainly there, and what cannot be served is what is inside it.
How to fix it
Take the channel list from your app, not from the workspace
Pass --channels with the ids your integration actually works in. Sweeping every channel in the workspace produces a bigger map that answers a question nobody asked, and costs a great deal more quota.
Ask each channel twice
conversations.info then conversations.history?limit=1. One request each, and the pair is what distinguishes content held under a revoked key from a channel that is simply invisible to this token.
Classify each response rather than counting failures
read_outcome returns ok, blocked, ambiguous or other, and names the error. Only ekm_access_denied becomes blocked; channel_not_found becomes ambiguous and stays there.
Read the two layers together
layer combines the metadata outcome with the content outcome: content-blocked is the characteristic EKM shape, both-blocked is broader, and clear is a channel with nothing wrong with it.
Describe the extent, and say that it is inferred
extent_shape reports whether the blocked channels are scattered, confined to one workspace, or everything measured. It returns a confidence, and that confidence is always inferred, because the granularity of key revocation is not published.
Skip, do not retry, and send the block to an administrator
skip_plan lists the channels the job should pass over so the run completes. retry_class keeps ekm_access_denied out of your backoff path. Then send the affected list, the workspace and the error string to the customer's Slack administrators, which is where the only available repair lives.
How to check it worked
After the customer's administrators restore access, re-run over the same list. Every channel should read clear, and the skip plan should be empty.
python3 slack_ekm_extent.py --channels C001,C002,C003,C004
# channel C001 metadata=ok content=ok clear
# channel C002 metadata=ok content=blocked content-blocked
# channel C003 metadata=blocked content=blocked both-blocked
# channel C004 metadata=ok content=ambiguous unknown
# note channel_not_found is undecidable from here: absent, or
# invisible to this token
# extent partial inferred 2 of 4 channel(s) blocked, all in T0123
# retry do-not-retry ekm_access_denied is a decision, not a condition
# skip C002,C003 skip these so the run completes over the rest
# verdict 2 finding(s) escalate to the customer's Slack administrators
# repair: no app side change affects this; access to the key is theirs to restore
# repair: treat ekm_access_denied as a permanent per channel failure and skip it
# repair: alert on it distinctly, so it is never read as a scope problem againThe full code
Two reads per channel and one classifier per layer. read_outcome is where the discipline sits: only ekm_access_denied becomes blocked, channel_not_found becomes ambiguous and is never resolved into a guess, and every other refusal is named and handed on. layer reads the metadata and content outcomes as a pair, which is what makes the EKM shape recognisable rather than just another failure. extent_shape returns a confidence with its shape and that confidence never rises above inferred, because the granularity of key revocation is not published. retry_class and skip_plan exist so a job can import the two behaviours that matter.
"""Map how far an Enterprise Key Management refusal reaches, read only.
ekm_access_denied is not about your app. An organization using Enterprise Key
Management holds its own encryption keys for its Slack content, and when
access to a key is revoked Slack declines to serve the content it protects to
anybody, with any token, holding any scope. There is no app side repair.
What an app can usefully do is measure the extent and stop retrying. Two reads
per channel: conversations.info for the metadata layer and
conversations.history with limit=1 for the content layer. Both document
ekm_access_denied in their error lists, and the two layers can disagree, which
is what makes this recognisable rather than just another refusal.
The extent is reported as an inference. Slack documents that an organization
can revoke granular access to its keys and does not publish the exact
granularity, so the shape below describes the rows that were collected and
makes no claim about how the customer's policy is configured.
"""
import argparse
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("slack_ekm_extent")
API = "https://slack.com/api/"
EKM_ERROR = "ekm_access_denied"
# Refusals that are somebody else's note. Named so a sweep that meets one hands
# the reader on rather than filing it under encryption.
ELSEWHERE = {
"not_in_channel": "/slack/bot-not-in-channel/",
"missing_scope": "/slack/missing-scope-on-read/",
"is_archived": "/slack/archived-channel-target/",
"accesslimited": "/slack/accesslimited-ip-allowlist/",
"not_allowed_token_type": "/slack/admin-method-needs-user-token/",
}
# Errors that a job should back off on, as against the one it must not.
TRANSIENT = ("ratelimited", "internal_error", "service_unavailable", "fatal_error",
"request_timeout")
def read_outcome(body):
"""Read one response as an outcome for one layer. Pure.
Returns (state, detail). Four states:
ok the layer answered.
blocked ekm_access_denied, and nothing else produces this.
ambiguous channel_not_found, which is undecidable from outside: absent,
or present and invisible to this token. Left undecided.
other any other refusal, named, and handed on where a note owns it.
"""
doc = body or {}
if doc.get("ok") is True:
return ("ok", "the layer answered")
error = str(doc.get("error") or "unknown_error")
if error == EKM_ERROR:
return ("blocked", "ekm_access_denied; the content is protected by a key this "
"token no longer has access to")
if error == "channel_not_found":
return ("ambiguous", "channel_not_found is undecidable from here: absent, or "
"invisible to this token")
pointer = ELSEWHERE.get(error)
if pointer:
return ("other", "%s; that refusal is owned by %s" % (error, pointer))
return ("other", "%s; not an encryption refusal" % error)
def layer(metadata_state, content_state):
"""Read the two layers together. Pure.
The pair is the diagnosis. Metadata that answers while content is refused
is the characteristic shape of a key revocation, because the channel is
plainly there and its contents are what cannot be served.
"""
if metadata_state == "blocked" and content_state == "blocked":
return ("both-blocked", "neither the channel's metadata nor its content can "
"be served")
if content_state == "blocked":
return ("content-blocked", "the channel is visible and its content is not; "
"this is the characteristic shape")
if metadata_state == "blocked":
return ("metadata-blocked", "the metadata is refused while content answered, "
"which is unusual and worth reporting as seen")
if "ambiguous" in (metadata_state, content_state):
return ("unknown", "one layer was undecidable, so this channel is neither "
"cleared nor counted")
if metadata_state == "ok" and content_state == "ok":
return ("clear", "both layers answered")
return ("other", "a refusal that is not about encryption")
def extent_shape(rows):
"""Describe how far the blocking reaches. Pure.
rows: [(channel, team_id, layer_state), ...]. Returns (shape, confidence,
detail). The confidence is always "inferred": Slack documents granular
revocation and does not publish the granularity, so this describes the
rows collected rather than the customer's policy.
"""
total = len(rows or [])
blocked = [r for r in (rows or []) if r[2] in ("content-blocked", "both-blocked",
"metadata-blocked")]
if not total:
return ("none-measured", "inferred", "no channels were read")
if not blocked:
return ("clear", "inferred", "no channel in this set was refused")
teams = {r[1] for r in blocked if r[1]}
where = "all in %s" % list(teams)[0] if len(teams) == 1 else \
"across %d workspace(s)" % len(teams)
detail = "%d of %d channel(s) blocked, %s" % (len(blocked), total, where)
if len(blocked) == total:
return ("everything-measured", "inferred", detail)
return ("partial", "inferred", detail)
def retry_class(error):
"""What should a job do with this error? Pure.
The one behaviour worth importing. ekm_access_denied is a decision recorded
in a customer's key management, not a condition that clears, and retrying
it converts a policy into rate limit pressure and a job that never
finishes.
"""
name = str(error or "").strip()
if name == EKM_ERROR:
return ("do-not-retry", "a decision, not a condition; the tenth attempt is "
"refused identically")
if name in TRANSIENT:
return ("backoff", "a condition that may clear; retry with backoff")
if name in ELSEWHERE:
return ("do-not-retry", "a configuration problem; retrying cannot fix it")
if not name:
return ("proceed", "no error")
return ("retry-once", "unrecognised; one retry, then report it rather than "
"looping")
def skip_plan(rows):
"""The channels a run should pass over so it completes. Pure."""
return [r[0] for r in (rows or []) if r[2] in ("content-blocked", "both-blocked",
"metadata-blocked")]
def alerting(shape):
"""How should this be surfaced? Pure.
Deliberately never a page. Nothing on call can act on a customer's key
policy at three in the morning, and an alert that wakes somebody up to
tell them a decision was made is training them to ignore alerts.
"""
if shape in ("clear", "none-measured"):
return ("none", "nothing to report")
if shape == "everything-measured":
return ("escalate", "every channel measured is refused; tell the customer's "
"administrators today, and do not page your own on call")
return ("notify", "report the affected set distinctly, so it is never read as a "
"scope problem; this is not an outage on your side")
def verdict(rows):
"""One line for the sweep. Pure."""
blocked = skip_plan(rows)
unknown = [r[0] for r in (rows or []) if r[2] == "unknown"]
if not rows:
return ("inconclusive", "no channels were read")
if not blocked:
return ("clean", "no channel in this set is refused on encryption grounds")
return ("%d finding(s)" % len(blocked),
"escalate to the customer's Slack administrators%s"
% ("; %d channel(s) undecided" % len(unknown) if unknown else ""))
def get(session, method, params):
"""One GET. Returns the parsed body."""
r = session.get(API + method, params=params, timeout=30)
try:
return r.json()
except ValueError:
return {"ok": False, "error": "unparseable_body"}
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--channels", required=True,
help="comma separated channel ids your app works in")
ap.add_argument("--token-env", default="SLACK_BOT_TOKEN",
help="environment variable holding a read scoped token")
args = ap.parse_args()
token = os.environ.get(args.token_env)
if not token:
log.error("token missing set %s", args.token_env)
return 2
session = requests.Session()
session.headers.update({"Authorization": "Bearer " + token})
auth = get(session, "auth.test", {})
team_id = auth.get("team_id") or ""
if auth.get("enterprise_id"):
log.info("context %-16s enterprise_id=%s, so key management is available "
"to this organization", team_id, auth.get("enterprise_id"))
else:
log.info("context %-16s no enterprise_id on auth.test", team_id or "?")
rows = []
for channel in [c.strip() for c in args.channels.split(",") if c.strip()]:
meta_state, meta_why = read_outcome(
get(session, "conversations.info", {"channel": channel}))
content_state, content_why = read_outcome(
get(session, "conversations.history", {"channel": channel, "limit": 1}))
state, why = layer(meta_state, content_state)
level = log.info if state == "clear" else log.warning
level("channel %-16s metadata=%-9s content=%-9s %s", channel, meta_state,
content_state, state)
if state != "clear":
log.info(" note %s", why)
for detail in (meta_why, content_why):
if "owned by" in detail or "undecidable" in detail:
log.info(" note %s", detail)
rows.append((channel, team_id, state))
shape, confidence, detail = extent_shape(rows)
(log.info if shape == "clear" else log.warning)(
"extent %-16s %-9s %s", shape, confidence, detail)
action, why = retry_class(EKM_ERROR)
log.info("retry %-16s %s", action, why)
skip = skip_plan(rows)
if skip:
log.warning("skip %-16s skip these so the run completes over the rest",
",".join(skip[:8]))
route, route_why = alerting(shape)
log.info("alert %-16s %s", route, route_why)
final, summary = verdict(rows)
if final == "clean":
log.info("verdict clean %s", summary)
return 0
log.warning("verdict %-16s %s", final, summary)
log.warning(" repair: no app side change affects this; access to the key is the "
"customer's to restore")
log.warning(" repair: treat %s as a permanent per channel failure and skip the "
"affected channels rather than blocking the run", EKM_ERROR)
log.warning(" repair: alert on it distinctly, so it is never read as a scope "
"problem again")
return 1
if __name__ == "__main__":
sys.exit(main())
/**
* Map how far an Enterprise Key Management refusal reaches, read only.
*
* ekm_access_denied is not about your app. An organization using Enterprise
* Key Management holds its own encryption keys for its Slack content, and when
* access to a key is revoked Slack declines to serve the content it protects
* to anybody, with any token, holding any scope. There is no app side repair.
*
* Two reads per channel: conversations.info for the metadata layer and
* conversations.history with limit=1 for the content layer. Both document
* ekm_access_denied, and the two layers can disagree, which is what makes this
* recognisable rather than just another refusal.
*
* The extent is reported as an inference, because Slack documents granular
* key revocation without publishing the granularity.
*/
const API = 'https://slack.com/api/';
export const EKM_ERROR = 'ekm_access_denied';
export const ELSEWHERE = {
not_in_channel: '/slack/bot-not-in-channel/',
missing_scope: '/slack/missing-scope-on-read/',
is_archived: '/slack/archived-channel-target/',
accesslimited: '/slack/accesslimited-ip-allowlist/',
not_allowed_token_type: '/slack/admin-method-needs-user-token/',
};
export const TRANSIENT = ['ratelimited', 'internal_error', 'service_unavailable',
'fatal_error', 'request_timeout'];
export function readOutcome(body) {
const doc = body ?? {};
if (doc.ok === true) return ['ok', 'the layer answered'];
const error = String(doc.error ?? 'unknown_error');
if (error === EKM_ERROR) {
return ['blocked', 'ekm_access_denied; the content is protected by a key this '
+ 'token no longer has access to'];
}
if (error === 'channel_not_found') {
return ['ambiguous', 'channel_not_found is undecidable from here: absent, or '
+ 'invisible to this token'];
}
const pointer = ELSEWHERE[error];
if (pointer) return ['other', `${error}; that refusal is owned by ${pointer}`];
return ['other', `${error}; not an encryption refusal`];
}
// The pair is the diagnosis.
export function layer(metadataState, contentState) {
if (metadataState === 'blocked' && contentState === 'blocked') {
return ['both-blocked', "neither the channel's metadata nor its content can be served"];
}
if (contentState === 'blocked') {
return ['content-blocked', 'the channel is visible and its content is not; this is '
+ 'the characteristic shape'];
}
if (metadataState === 'blocked') {
return ['metadata-blocked', 'the metadata is refused while content answered, which '
+ 'is unusual and worth reporting as seen'];
}
if (metadataState === 'ambiguous' || contentState === 'ambiguous') {
return ['unknown', 'one layer was undecidable, so this channel is neither cleared '
+ 'nor counted'];
}
if (metadataState === 'ok' && contentState === 'ok') return ['clear', 'both layers answered'];
return ['other', 'a refusal that is not about encryption'];
}
// The confidence is always "inferred".
export function extentShape(rows) {
const all = rows ?? [];
const blocked = all.filter(([, , state]) => ['content-blocked', 'both-blocked',
'metadata-blocked'].includes(state));
if (!all.length) return ['none-measured', 'inferred', 'no channels were read'];
if (!blocked.length) {
return ['clear', 'inferred', 'no channel in this set was refused'];
}
const teams = [...new Set(blocked.map(([, team]) => team).filter(Boolean))];
const where = teams.length === 1 ? `all in ${teams[0]}`
: `across ${teams.length} workspace(s)`;
const detail = `${blocked.length} of ${all.length} channel(s) blocked, ${where}`;
if (blocked.length === all.length) return ['everything-measured', 'inferred', detail];
return ['partial', 'inferred', detail];
}
// The one behaviour worth importing.
export function retryClass(error) {
const name = String(error ?? '').trim();
if (name === EKM_ERROR) {
return ['do-not-retry', 'a decision, not a condition; the tenth attempt is refused '
+ 'identically'];
}
if (TRANSIENT.includes(name)) {
return ['backoff', 'a condition that may clear; retry with backoff'];
}
if (name in ELSEWHERE) {
return ['do-not-retry', 'a configuration problem; retrying cannot fix it'];
}
if (!name) return ['proceed', 'no error'];
return ['retry-once', 'unrecognised; one retry, then report it rather than looping'];
}
export function skipPlan(rows) {
return (rows ?? []).filter(([, , state]) => ['content-blocked', 'both-blocked',
'metadata-blocked'].includes(state)).map(([channel]) => channel);
}
// Deliberately never a page.
export function alerting(shape) {
if (shape === 'clear' || shape === 'none-measured') return ['none', 'nothing to report'];
if (shape === 'everything-measured') {
return ['escalate', "every channel measured is refused; tell the customer's "
+ 'administrators today, and do not page your own on call'];
}
return ['notify', 'report the affected set distinctly, so it is never read as a scope '
+ 'problem; this is not an outage on your side'];
}
export function verdict(rows) {
const blocked = skipPlan(rows);
const unknown = (rows ?? []).filter(([, , state]) => state === 'unknown');
if (!(rows ?? []).length) return ['inconclusive', 'no channels were read'];
if (!blocked.length) {
return ['clean', 'no channel in this set is refused on encryption grounds'];
}
return [`${blocked.length} finding(s)`,
"escalate to the customer's Slack administrators"
+ (unknown.length ? `; ${unknown.length} channel(s) undecided` : '')];
}
function arg(args, name, fallback = '') {
const i = args.indexOf(name);
return i === -1 ? fallback : args[i + 1];
}
async function read(token, method, params) {
const url = new URL(API + method);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, String(v));
const r = await fetch(url, { headers: { Authorization: `Bearer ${token}` } });
try {
return await r.json();
} catch {
return { ok: false, error: 'unparseable_body' };
}
}
async function main() {
const args = process.argv.slice(2);
const channels = arg(args, '--channels').split(',').map((c) => c.trim()).filter(Boolean);
if (!channels.length) {
console.error('pass --channels with the channel ids your app works in');
process.exitCode = 2;
return;
}
const tokenEnv = arg(args, '--token-env', 'SLACK_BOT_TOKEN');
const token = process.env[tokenEnv];
if (!token) {
console.error(`token missing set ${tokenEnv}`);
process.exitCode = 2;
return;
}
const auth = await read(token, 'auth.test', {});
const teamId = auth.team_id ?? '';
if (auth.enterprise_id) {
console.log(`context ${String(teamId).padEnd(16)} enterprise_id=`
+ `${auth.enterprise_id}, so key management is available to this organization`);
} else {
console.log(`context ${String(teamId || '?').padEnd(16)} no enterprise_id on auth.test`);
}
const rows = [];
for (const channel of channels) {
// eslint-disable-next-line no-await-in-loop
const [metaState, metaWhy] = readOutcome(await read(token, 'conversations.info',
{ channel }));
// eslint-disable-next-line no-await-in-loop
const [contentState, contentWhy] = readOutcome(await read(token,
'conversations.history', { channel, limit: 1 }));
const [state, why] = layer(metaState, contentState);
const line = `channel ${channel.padEnd(16)} metadata=${metaState.padEnd(9)} `
+ `content=${contentState.padEnd(9)} ${state}`;
if (state === 'clear') console.log(line);
else console.warn(line);
if (state !== 'clear') console.log(` note ${why}`);
for (const detail of [metaWhy, contentWhy]) {
if (detail.includes('owned by') || detail.includes('undecidable')) {
console.log(` note ${detail}`);
}
}
rows.push([channel, teamId, state]);
}
const [shape, confidence, detail] = extentShape(rows);
const extentLine = `extent ${shape.padEnd(16)} ${confidence.padEnd(9)} ${detail}`;
if (shape === 'clear') console.log(extentLine);
else console.warn(extentLine);
const [action, why] = retryClass(EKM_ERROR);
console.log(`retry ${action.padEnd(16)} ${why}`);
const skip = skipPlan(rows);
if (skip.length) {
console.warn(`skip ${skip.slice(0, 8).join(',').padEnd(16)} skip these so the `
+ 'run completes over the rest');
}
const [route, routeWhy] = alerting(shape);
console.log(`alert ${route.padEnd(16)} ${routeWhy}`);
const [final, summary] = verdict(rows);
if (final === 'clean') {
console.log(`verdict clean ${summary}`);
return;
}
console.warn(`verdict ${final.padEnd(16)} ${summary}`);
console.warn(' repair: no app side change affects this; access to the key is the '
+ "customer's to restore");
console.warn(` repair: treat ${EKM_ERROR} as a permanent per channel failure and `
+ 'skip the affected channels rather than blocking the run');
console.warn(' repair: alert on it distinctly, so it is never read as a scope '
+ 'problem again');
process.exitCode = 1;
}
// Only run when invoked directly, so importing this module in the tests does
// not execute main().
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The assertions that carry this note are about restraint again, in three places. channel_not_found has to stay ambiguous and must never be resolved into either an absence or a block, because Slack genuinely does not say which it is. ekm_access_denied has to land in do-not-retry, next to a test that puts ratelimited in backoff, so the difference between a policy and a condition is checked rather than described. And extent_shape has to return inferred for every input, including the ones where the pattern looks obvious, because the granularity of key revocation is not published and a script that says workspace-level with confidence is asserting something nobody outside the customer's organization knows.
from slack_ekm_extent import (
alerting, extent_shape, layer, read_outcome, retry_class, skip_plan, verdict,
)
BLOCKED = {"ok": False, "error": "ekm_access_denied"}
FINE = {"ok": True, "channel": {"id": "C1"}}
def test_only_the_encryption_error_is_blocked():
assert read_outcome(BLOCKED)[0] == "blocked"
assert read_outcome(FINE)[0] == "ok"
assert read_outcome({"ok": False, "error": "not_in_channel"})[0] == "other"
def test_channel_not_found_stays_undecided():
state, why = read_outcome({"ok": False, "error": "channel_not_found"})
assert state == "ambiguous"
assert "undecidable" in why
def test_refusals_owned_elsewhere_are_handed_over_by_name():
_state, why = read_outcome({"ok": False, "error": "missing_scope"})
assert "/slack/missing-scope-on-read/" in why
def test_visible_metadata_with_refused_content_is_the_shape():
state, why = layer("ok", "blocked")
assert state == "content-blocked"
assert "characteristic shape" in why
def test_both_layers_refused_is_broader():
assert layer("blocked", "blocked")[0] == "both-blocked"
def test_a_channel_that_answers_twice_is_clear():
assert layer("ok", "ok")[0] == "clear"
def test_an_ambiguous_layer_leaves_the_channel_uncounted():
assert layer("ok", "ambiguous")[0] == "unknown"
def test_the_extent_is_always_inferred():
rows = [("C1", "T1", "clear"), ("C2", "T1", "content-blocked")]
shape, confidence, detail = extent_shape(rows)
assert shape == "partial"
assert confidence == "inferred"
assert "all in T1" in detail
def test_the_extent_is_inferred_even_when_everything_is_blocked():
rows = [("C1", "T1", "both-blocked"), ("C2", "T1", "content-blocked")]
shape, confidence, _detail = extent_shape(rows)
assert shape == "everything-measured"
assert confidence == "inferred"
def test_a_clean_set_reports_clear_and_still_says_inferred():
assert extent_shape([("C1", "T1", "clear")]) == (
"clear", "inferred", "no channel in this set was refused")
def test_nothing_measured_is_not_a_clean_result():
assert extent_shape([])[0] == "none-measured"
def test_the_encryption_error_is_never_retried():
action, why = retry_class("ekm_access_denied")
assert action == "do-not-retry"
assert "not a condition" in why
def test_a_genuine_transient_still_backs_off():
assert retry_class("ratelimited")[0] == "backoff"
def test_an_unrecognised_error_is_tried_once_and_then_reported():
assert retry_class("something_new")[0] == "retry-once"
def test_the_skip_plan_lists_the_channels_a_run_should_pass_over():
rows = [("C1", "T1", "clear"), ("C2", "T1", "content-blocked"),
("C3", "T1", "both-blocked"), ("C4", "T1", "unknown")]
assert skip_plan(rows) == ["C2", "C3"]
def test_this_is_never_a_page():
assert alerting("partial")[0] == "notify"
assert alerting("everything-measured")[0] == "escalate"
assert alerting("clear")[0] == "none"
def test_the_verdict_counts_blocked_channels_and_flags_the_undecided():
state, summary = verdict([("C1", "T1", "content-blocked"), ("C2", "T1", "unknown")])
assert state == "1 finding(s)"
assert "1 channel(s) undecided" in summary
def test_a_clean_sweep_says_clean():
assert verdict([("C1", "T1", "clear")])[0] == "clean"
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
alerting, extentShape, layer, readOutcome, retryClass, skipPlan, verdict,
} from './slack-ekm-extent.mjs';
const BLOCKED = { ok: false, error: 'ekm_access_denied' };
const FINE = { ok: true, channel: { id: 'C1' } };
test('only the encryption error is blocked', () => {
assert.equal(readOutcome(BLOCKED)[0], 'blocked');
assert.equal(readOutcome(FINE)[0], 'ok');
assert.equal(readOutcome({ ok: false, error: 'not_in_channel' })[0], 'other');
});
test('channel_not_found stays undecided', () => {
const [state, why] = readOutcome({ ok: false, error: 'channel_not_found' });
assert.equal(state, 'ambiguous');
assert.match(why, /undecidable/);
});
test('refusals owned elsewhere are handed over by name', () => {
const [, why] = readOutcome({ ok: false, error: 'missing_scope' });
assert.match(why, /\/slack\/missing-scope-on-read\//);
});
test('visible metadata with refused content is the shape', () => {
const [state, why] = layer('ok', 'blocked');
assert.equal(state, 'content-blocked');
assert.match(why, /characteristic shape/);
});
test('both layers refused is broader', () => {
assert.equal(layer('blocked', 'blocked')[0], 'both-blocked');
});
test('a channel that answers twice is clear', () => {
assert.equal(layer('ok', 'ok')[0], 'clear');
});
test('an ambiguous layer leaves the channel uncounted', () => {
assert.equal(layer('ok', 'ambiguous')[0], 'unknown');
});
test('the extent is always inferred', () => {
const rows = [['C1', 'T1', 'clear'], ['C2', 'T1', 'content-blocked']];
const [shape, confidence, detail] = extentShape(rows);
assert.equal(shape, 'partial');
assert.equal(confidence, 'inferred');
assert.match(detail, /all in T1/);
});
test('the extent is inferred even when everything is blocked', () => {
const rows = [['C1', 'T1', 'both-blocked'], ['C2', 'T1', 'content-blocked']];
const [shape, confidence] = extentShape(rows);
assert.equal(shape, 'everything-measured');
assert.equal(confidence, 'inferred');
});
test('a clean set reports clear and still says inferred', () => {
assert.deepEqual(extentShape([['C1', 'T1', 'clear']]),
['clear', 'inferred', 'no channel in this set was refused']);
});
test('nothing measured is not a clean result', () => {
assert.equal(extentShape([])[0], 'none-measured');
});
test('the encryption error is never retried', () => {
const [action, why] = retryClass('ekm_access_denied');
assert.equal(action, 'do-not-retry');
assert.match(why, /not a condition/);
});
test('a genuine transient still backs off', () => {
assert.equal(retryClass('ratelimited')[0], 'backoff');
});
test('an unrecognised error is tried once and then reported', () => {
assert.equal(retryClass('something_new')[0], 'retry-once');
});
test('the skip plan lists the channels a run should pass over', () => {
const rows = [['C1', 'T1', 'clear'], ['C2', 'T1', 'content-blocked'],
['C3', 'T1', 'both-blocked'], ['C4', 'T1', 'unknown']];
assert.deepEqual(skipPlan(rows), ['C2', 'C3']);
});
test('this is never a page', () => {
assert.equal(alerting('partial')[0], 'notify');
assert.equal(alerting('everything-measured')[0], 'escalate');
assert.equal(alerting('clear')[0], 'none');
});
test('the verdict counts blocked channels and flags the undecided', () => {
const [state, summary] = verdict([['C1', 'T1', 'content-blocked'],
['C2', 'T1', 'unknown']]);
assert.equal(state, '1 finding(s)');
assert.match(summary, /1 channel\(s\) undecided/);
});
test('a clean sweep says clean', () => {
assert.equal(verdict([['C1', 'T1', 'clear']])[0], 'clean');
});
FAQ
Which scope fixes ekm_access_denied?
None of them, and that is the first thing to internalise about it. The refusal is not about your token's permissions; it is about a key held by the customer's organization under Enterprise Key Management. Slack is declining to serve content encrypted with a key whose access has been revoked, and it will decline for any caller with any grant. The only path to it being served again runs through the customer's own administrators.
Can I tell whether this is one channel, one workspace or the whole organization?
You can describe what you measured, and the script is careful to phrase it that way. Slack documents that an organization can revoke granular access to its encryption keys and does not publish the exact levels at which that happens, so a map showing eleven blocked channels in one workspace is a description of eleven channels, not a statement about the customer's key policy. The confidence value in the output is always inferred for exactly that reason.
Should the job retry these?
No, and the retry classifier is in the module so your job can import the decision rather than reimplement it. A revoked key is a decision that persists until somebody changes it, so the second attempt and the two-hundredth are refused identically. All a retry loop adds is quota pressure, a job that runs for an hour instead of four minutes, and a log in which a genuinely transient failure is buried among identical permanent ones.
Why read conversations.info as well as conversations.history?
Because the pair is the diagnosis. Both methods document ekm_access_denied, and when the metadata answers while the content is refused you know the channel is there and visible and that what cannot be served is its contents, which is the characteristic shape of a key problem. If you only called one method you would have a failure that looks like every other failure, and the natural conclusion would be that the bot is not in the channel.
What exactly should I send the customer?
The specific block: the error string ekm_access_denied, the affected channel ids, the workspace they are in, and roughly when the failures began. Their administrators can look up which key covers that content and whether its access was revoked, which is a question they can answer in minutes and you cannot answer at all. A generic alert that Slack errors are elevated cannot start that conversation, which is why the script prints the affected set as a block rather than a count.
Related field notes
- an administrator's decision, made in the channel
- a refusal decided by the network instead
- a policy aimed at the app rather than the content
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Slack Enterprise Key Management - Slack Help Center
- conversations.history method reference - Slack Docs
- conversations.info method reference - Slack Docs
- chat.postMessage method reference - Slack Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.