Diagnostic Slack
Edits, joins and deletes handled as brand new messages
Somebody fixes a typo in a message they posted an hour ago, and the bot answers it again. Somebody joins the channel, and the ticket pipeline opens a ticket for “Ana has joined the channel”. Somebody deletes a message, and it turns up in the archive anyway, which is the version of this bug that ends up in a compliance conversation.
The handler is reading event.text on every message event it receives. Most message events are not new messages.
message is not one event. It is a family, distinguished by a subtype field: message_changed, message_deleted, channel_join, channel_leave, bot_message, file_share, thread_broadcast, message_replied, tombstone and more. A handler that reads event.text and replies treats every one of them as a person saying something new.
The nastiest member is message_changed, because the payload is a different shape: the current text is at event.message.text and the previous version at event.previous_message.text, so event.text is missing entirely and code that reads it gets None rather than an error. The script reads conversations.history, where the same edit shows up as an edited block on the message, counts what share of recent traffic is not a plain new message, and then looks for the proof: app replies clustered after an edited.ts instead of the original ts.
The problem in plain words
The first version of a Slack handler is four lines long and it works. It subscribes to message.channels, reads event.text, matches a keyword and replies. Every message anybody sends in the test channel is a plain new message, so the four lines are correct for the entire time anybody is watching.
Then the app meets a real channel. People join and leave, and Slack writes those into the conversation as messages with a subtype and a text that reads "Ana has joined the channel" - which means a keyword matcher will happily match on it. People edit messages, and the edit arrives as a whole new delivery with a nested payload, so the reply pipeline fires a second time on content it has already answered. People delete messages, and an archiver that treats the deletion as an arrival stores the thing that was just withdrawn. Files get shared, and a message with an empty text and everything of interest in event.files gets logged as a blank line.
None of this errors. Every one of those payloads is a valid message event, delivered correctly, and the handler processes each of them without complaint. The symptom is downstream and looks like a data problem: the archive has three copies of one message, the ticket queue has an entry with somebody's join line in it, the reply log has a response to a message nobody sent. By the time it is noticed the pipeline has months of it, and reconstructing which rows are real means knowing which subtypes were being swallowed.
Why it happens
This is not the echo loop. The echo loop is the app reacting to its own output, and it is loud, fast and visible as a run of self-authored messages. This is the app mis-reading traffic from people, one message at a time, at a rate nobody notices. The two overlap on exactly one subtype, bot_message, and this script hands that row to the other note rather than claiming it.
message_changed is a different payload shape, not a flag on the same one. The text moves to event.message.text, the prior version appears at event.previous_message.text, and event.text is simply not there. In Python that is a None; in JavaScript it is undefined, which then gets concatenated into a string and stored. Neither one raises.
Join and leave lines carry text a matcher will match. They are not empty and they are not marked in any way that a naive read notices. Anything doing keyword matching, sentiment, indexing or ticket creation will act on them, which is why joins show up as the first reported symptom more often than edits do.
History shows edits differently from the event stream, and that is what makes this detectable. Over the Events API an edit arrives as message_changed. In conversations.history the same edit is a plain message carrying an edited: {user, ts} object. So a read-only script can find the edits and then check whether your app replied at the edit time rather than the original time, which is the closest thing to proof available without seeing your handler.
Bolt filters one of these for you and not the rest. app.message() drops bot_message, which is enough to stop the loop and is frequently mistaken for subtype handling in general. Joins, leaves, edits and deletions all still arrive.
The fix, as a flow
This chain goes red at the second box, earlier than the other three, because the mistake is an assumption made while the code still worked perfectly. Everything after it is correct behaviour acting on a wrong premise. The fix branch is a lookup rather than a diagnosis, and one of its plain rows is a row this note deliberately declines to own.
How to fix it
Sample real history rather than a test channel
A test channel is all plain messages, which is why this survives review. conversations.history on a channel people actually use will contain joins, edits and file shares within the first two hundred messages, and those are the payload shapes your handler has been seeing.
Classify every message and say where its text actually lives
classify returns the kind, the field the real content is in, and what a handler should do with it. The text_path is the useful column: message.text for an edit, previous_message.text for a deletion, files for a share. It is the list somebody needs open while they write the branch.
Count the share of traffic that is not a plain new message
profile turns the sample into one number. A channel where a fifth of recent messages carry a subtype or an edit is a channel where a handler with no subtype branch is wrong a fifth of the time, and that number is what makes the case to prioritise the fix.
Look for replies that land at the edit time instead of the original time
edit_echo is the proof. If your app replied within a short window after an edited.ts but not after the original ts, it processed the edit as a new message. If it replied after both, that is inconclusive and gets reported as inconclusive rather than as a finding.
Hand bot_message to the note that owns it
Self-authored messages are classified as elsewhere, with a pointer. Two audits reporting the same row is how a team ends up fixing it twice, or arguing about which report is right, so this one declines the overlap explicitly.
Print the branch, with the default at the bottom
The repair is a switch on subtype where the absent case is the plain new message and everything else is named: read message.text for message_changed, ignore channel_join, channel_leave and message_deleted unless you want them, and treat file_share and thread_broadcast as real content.
How to check it worked
After the branch ships, re-run against the same channel. The profile will be unchanged, because it describes the workspace rather than your code, and the edit correlation should be the part that goes quiet.
python3 slack_message_subtypes.py --channel C01ABCDE9
# identity U0APPBOT11 in acme
# profile 200 message(s): 148 plain, 41 subtyped, 11 edited, 26.0% not a new message
# kind channel_join 22 ignore joins carry text a keyword matcher will match
# kind file_share 9 content real content, and event.text is often empty
# kind message_changed 0 branch the text is at event.message.text
# kind bot_message 10 elsewhere self authored; the echo loop note owns this row
# edits clean 0 of 11 edit(s) were followed by a replyThe full code
One read method and three pure functions. classify is the table somebody wants open while writing the branch: for each subtype, where the content actually lives and what to do with it. profile reduces a page of history to the share of traffic a subtype-blind handler gets wrong. edit_echo is the only part that tries to prove anything, correlating your app's replies against edit timestamps rather than original ones, and it is written to return inconclusive far more often than it returns a finding.
"""Measure how much of a channel's traffic is not a plain new message.
Read only. auth.test for identity and conversations.history per channel.
Nothing is posted, edited or deleted: this reports which subtypes your handler
has been receiving, where the real content lives in each of them, and whether
your app's replies line up with edit timestamps rather than original ones.
"""
import argparse
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("slack_message_subtypes")
API = "https://slack.com/api/"
# subtype -> (where the real content is, what to do with it, why).
# The disposition column is the one to read. "elsewhere" appears once, for
# bot_message, because the loop that subtype causes is a different note and two
# audits reporting one row is how a team fixes it twice.
SUBTYPES = {
"message_changed": ("message.text", "branch",
"the current text is at event.message.text and the previous "
"version at event.previous_message.text; event.text is absent"),
"message_deleted": ("previous_message.text", "ignore",
"the message was withdrawn; an archiver that stores this "
"stores the thing that was just deleted"),
"message_replied": ("message.text", "ignore",
"a metadata update on a thread parent, not a new message"),
"channel_join": ("text", "ignore",
"joins carry text a keyword matcher will match"),
"channel_leave": ("text", "ignore", "the same, for leaving"),
"channel_topic": ("text", "ignore", "a topic change written as a message"),
"channel_purpose": ("text", "ignore", "a purpose change written as a message"),
"channel_name": ("text", "ignore", "a rename written as a message"),
"channel_archive": ("text", "ignore", "an archive written as a message"),
"channel_unarchive": ("text", "ignore", "an unarchive written as a message"),
"tombstone": ("text", "ignore", "a placeholder left where a message was removed"),
"bot_message": ("text", "elsewhere",
"an app authored message; the loop this causes has its own note "
"and this script does not claim the row"),
"file_share": ("files", "content",
"real content, and event.text is often empty because the point "
"of the message is in event.files"),
"thread_broadcast": ("text", "content",
"a real threaded message also sent to the channel; expect "
"to see it twice if you read replies as well"),
"me_message": ("text", "content", "a real message typed with /me"),
}
def classify(message):
"""What kind of message event is this, and where is its content? Pure.
Returns (kind, text_path, disposition, note). The kind that matters most is
edited-in-place: in conversations.history an edit appears as a plain
message carrying an edited block, while the Events API delivered the very
same edit to your handler as a message_changed with a different shape.
"""
m = message or {}
subtype = str(m.get("subtype") or "").strip()
if not subtype:
if m.get("edited"):
return ("edited-in-place", "message.text", "branch",
"history shows this as a plain message with an edited block; "
"over the Events API the same edit arrived as message_changed, "
"which is the payload your handler actually saw")
return ("plain", "text", "new-message",
"a genuine new message from a person, and the only kind event.text "
"is the right field for")
known = SUBTYPES.get(subtype)
if not known:
return (subtype, "text", "unknown-subtype",
"not in this script's table; read the message event reference "
"before letting it fall through to the new message path")
path, disposition, note = known
return (subtype, path, disposition, note)
def profile(messages):
"""Reduce a page of history to the share a subtype-blind handler gets wrong. Pure.
Returns a dict with the counts and one percentage. The percentage is the
argument: a handler with no subtype branch is wrong that often, in this
channel, today.
"""
kinds = {}
plain = subtyped = edited = 0
rows = list(messages or [])
for m in rows:
kind, _, disposition, note = classify(m)
entry = kinds.setdefault(kind, {"count": 0, "disposition": disposition,
"note": note})
entry["count"] += 1
if kind == "plain":
plain += 1
elif kind == "edited-in-place":
edited += 1
else:
subtyped += 1
total = len(rows)
off = total - plain
return {"total": total, "plain": plain, "subtyped": subtyped, "edited": edited,
"share_percent": round(off * 100.0 / total, 1) if total else 0.0,
"kinds": kinds}
def edit_echo(messages, bot_id, bot_user, window=90.0):
"""Did the app reply at the edit time rather than at the original time? Pure.
Returns rows of (edited_ts, original_ts, verdict, detail). Written to
return inconclusive readily: a reply after both timestamps proves nothing,
and an edit made seconds after the original cannot be separated from it at
all.
"""
replies = []
edits = []
for m in messages or []:
ts = float(m.get("ts") or 0)
ours = bool(bot_id and m.get("bot_id") == bot_id) or bool(
bot_user and m.get("user") == bot_user)
if ours:
replies.append(ts)
continue
edited = m.get("edited") or {}
if edited.get("ts"):
edits.append((float(edited["ts"]), ts))
replies.sort()
w = float(window)
out = []
for edited_ts, original_ts in sorted(edits):
if edited_ts - original_ts <= w:
out.append((edited_ts, original_ts, "too-close",
"the edit landed within the correlation window of the "
"original, so a reply cannot be attributed to either"))
continue
after_edit = [r for r in replies if edited_ts <= r <= edited_ts + w]
after_original = [r for r in replies if original_ts <= r <= original_ts + w]
if after_edit and not after_original:
out.append((edited_ts, original_ts, "reprocessed",
"%d reply(ies) within %.0fs of the edit and none within "
"%.0fs of the original message: the edit was handled as a "
"new message" % (len(after_edit), w, w)))
elif after_edit and after_original:
out.append((edited_ts, original_ts, "inconclusive",
"replies near both timestamps; a long thread can produce "
"this without any subtype bug"))
else:
out.append((edited_ts, original_ts, "clean",
"no reply followed this edit"))
return out
def get(session, method, params, label):
"""One GET, asserting on the body rather than the status line."""
r = session.get(API + method, params=params or {}, timeout=30)
body = r.json()
if body.get("ok") is not True:
log.warning("%-10s %-14s %s", label, "unavailable", body.get("error"))
return None
return body
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--token-env", default="SLACK_BOT_TOKEN",
help="environment variable holding the bot token")
ap.add_argument("--channel", action="append", default=[],
help="a channel ID your handler serves; repeatable")
ap.add_argument("--limit", type=int, default=200,
help="messages sampled per channel")
ap.add_argument("--window", type=float, default=90.0,
help="seconds after an edit in which a reply counts as caused by it")
args = ap.parse_args()
token = os.environ.get(args.token_env)
if not token:
log.error("set %s (channels:history is enough)", args.token_env)
return 2
if not args.channel:
log.error("pass at least one --channel; a test channel is all plain "
"messages and will show you nothing")
return 2
s = requests.Session()
s.headers.update({"Authorization": "Bearer " + token})
who = get(s, "auth.test", {}, "auth.test")
if not who:
return 2
bot_user, bot_id = who.get("user_id") or "", who.get("bot_id") or ""
log.info("identity %s in %s", bot_user, who.get("team"))
findings = 0
for channel in args.channel:
body = get(s, "conversations.history",
{"channel": channel, "limit": str(args.limit)}, "history")
if not body:
continue
messages = body.get("messages") or []
p = profile(messages)
(log.warning if p["share_percent"] >= 10.0 else log.info)(
"profile %d message(s): %d plain, %d subtyped, %d edited, %.1f%% "
"not a new message", p["total"], p["plain"], p["subtyped"],
p["edited"], p["share_percent"])
for kind, info in sorted(p["kinds"].items(),
key=lambda kv: (-kv[1]["count"], kv[0])):
if kind == "plain":
continue
(log.warning if info["disposition"] in ("branch", "unknown-subtype")
else log.info)("kind %-16s %3d %-10s %s", kind, info["count"],
info["disposition"], info["note"])
if info["disposition"] in ("branch", "unknown-subtype"):
findings += 1
rows = edit_echo(messages, bot_id, bot_user, args.window)
caught = [r for r in rows if r[2] == "reprocessed"]
if caught:
findings += len(caught)
for edited_ts, original_ts, _, detail in caught:
log.warning("edits reprocessed edit at %.6f of a message "
"posted at %.6f: %s", edited_ts, original_ts, detail)
else:
log.info("edits clean %d of %d edit(s) were followed by "
"a reply", 0, len(rows))
if findings:
log.warning(" repair: switch on event.subtype with the absent case as the "
"plain new message, and name every other branch explicitly")
log.warning(" repair: for message_changed read event.message.text and "
"compare it against event.previous_message.text; event.text "
"is not there")
log.warning(" repair: ignore channel_join, channel_leave and "
"message_deleted unless you specifically want them, and treat "
"file_share and thread_broadcast as real content")
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
/**
* Measure how much of a channel's traffic is not a plain new message.
*
* Read only. auth.test for identity and conversations.history per channel.
* Nothing is posted, edited or deleted: this reports which subtypes your
* handler has been receiving, where the real content lives in each of them,
* and whether your app's replies line up with edit timestamps rather than
* original ones.
*/
const API = 'https://slack.com/api/';
// subtype -> [where the real content is, what to do with it, why].
// "elsewhere" appears once, for bot_message, because the loop that subtype
// causes is a different note and two audits reporting one row is how a team
// fixes it twice.
const SUBTYPES = new Map([
['message_changed', ['message.text', 'branch',
'the current text is at event.message.text and the previous version at ' +
'event.previous_message.text; event.text is absent']],
['message_deleted', ['previous_message.text', 'ignore',
'the message was withdrawn; an archiver that stores this stores the thing ' +
'that was just deleted']],
['message_replied', ['message.text', 'ignore',
'a metadata update on a thread parent, not a new message']],
['channel_join', ['text', 'ignore', 'joins carry text a keyword matcher will match']],
['channel_leave', ['text', 'ignore', 'the same, for leaving']],
['channel_topic', ['text', 'ignore', 'a topic change written as a message']],
['channel_purpose', ['text', 'ignore', 'a purpose change written as a message']],
['channel_name', ['text', 'ignore', 'a rename written as a message']],
['channel_archive', ['text', 'ignore', 'an archive written as a message']],
['channel_unarchive', ['text', 'ignore', 'an unarchive written as a message']],
['tombstone', ['text', 'ignore', 'a placeholder left where a message was removed']],
['bot_message', ['text', 'elsewhere',
'an app authored message; the loop this causes has its own note and this ' +
'script does not claim the row']],
['file_share', ['files', 'content',
'real content, and event.text is often empty because the point of the ' +
'message is in event.files']],
['thread_broadcast', ['text', 'content',
'a real threaded message also sent to the channel; expect to see it twice ' +
'if you read replies as well']],
['me_message', ['text', 'content', 'a real message typed with /me']],
]);
/**
* What kind of message event is this, and where is its content? Pure.
* edited-in-place is the kind that matters most: history shows an edit as a
* plain message with an edited block, while the Events API delivered that same
* edit to your handler as a message_changed with a different shape.
*/
export function classify(message) {
const m = message ?? {};
const subtype = String(m.subtype ?? '').trim();
if (!subtype) {
if (m.edited) {
return ['edited-in-place', 'message.text', 'branch',
'history shows this as a plain message with an edited block; over the ' +
'Events API the same edit arrived as message_changed, which is the payload ' +
'your handler actually saw'];
}
return ['plain', 'text', 'new-message',
'a genuine new message from a person, and the only kind event.text is the ' +
'right field for'];
}
const known = SUBTYPES.get(subtype);
if (!known) {
return [subtype, 'text', 'unknown-subtype',
"not in this script's table; read the message event reference before letting " +
'it fall through to the new message path'];
}
const [path, disposition, note] = known;
return [subtype, path, disposition, note];
}
/**
* Reduce a page of history to the share a subtype-blind handler gets wrong. Pure.
* The percentage is the argument.
*/
export function profile(messages) {
const kinds = new Map();
let plain = 0;
let subtyped = 0;
let edited = 0;
const rows = [...(messages ?? [])];
for (const m of rows) {
const [kind, , disposition, note] = classify(m);
if (!kinds.has(kind)) kinds.set(kind, { count: 0, disposition, note });
kinds.get(kind).count += 1;
if (kind === 'plain') plain += 1;
else if (kind === 'edited-in-place') edited += 1;
else subtyped += 1;
}
const total = rows.length;
const off = total - plain;
return {
total,
plain,
subtyped,
edited,
sharePercent: total ? Math.round((off * 1000.0) / total) / 10 : 0.0,
kinds,
};
}
/**
* Did the app reply at the edit time rather than at the original time? Pure.
* Written to return inconclusive readily: a reply after both timestamps proves
* nothing, and an edit made seconds after the original cannot be separated.
*/
export function editEcho(messages, botId, botUser, window = 90.0) {
const replies = [];
const edits = [];
for (const m of messages ?? []) {
const ts = Number(m.ts ?? 0);
const ours = (botId && m.bot_id === botId) || (botUser && m.user === botUser);
if (ours) { replies.push(ts); continue; }
const edited = m.edited ?? {};
if (edited.ts) edits.push([Number(edited.ts), ts]);
}
replies.sort((a, b) => a - b);
const w = Number(window);
const out = [];
for (const [editedTs, originalTs] of edits.sort((a, b) => a[0] - b[0])) {
if (editedTs - originalTs <= w) {
out.push([editedTs, originalTs, 'too-close',
'the edit landed within the correlation window of the original, so a reply ' +
'cannot be attributed to either']);
continue;
}
const afterEdit = replies.filter((r) => r >= editedTs && r <= editedTs + w);
const afterOriginal = replies.filter((r) => r >= originalTs && r <= originalTs + w);
if (afterEdit.length && !afterOriginal.length) {
out.push([editedTs, originalTs, 'reprocessed',
`${afterEdit.length} reply(ies) within ${w.toFixed(0)}s of the edit and none ` +
`within ${w.toFixed(0)}s of the original message: the edit was handled as a ` +
'new message']);
} else if (afterEdit.length) {
out.push([editedTs, originalTs, 'inconclusive',
'replies near both timestamps; a long thread can produce this without any ' +
'subtype bug']);
} else {
out.push([editedTs, originalTs, 'clean', 'no reply followed this edit']);
}
}
return out;
}
async function get(token, method, params, label) {
const qs = new URLSearchParams(params ?? {});
const res = await fetch(`${API}${method}?${qs}`, {
headers: { Authorization: `Bearer ${token}` },
});
const body = await res.json();
if (body.ok !== true) {
console.warn(`${label.padEnd(10)} ${'unavailable'.padEnd(14)} ${body.error}`);
return null;
}
return body;
}
function arg(args, name, fallback = null) {
const i = args.indexOf(name);
return i === -1 ? fallback : args[i + 1];
}
function argAll(args, name) {
const out = [];
for (let i = 0; i < args.length; i += 1) {
if (args[i] === name && args[i + 1] !== undefined) out.push(args[i + 1]);
}
return out;
}
async function main() {
const args = process.argv.slice(2);
const tokenEnv = arg(args, '--token-env', 'SLACK_BOT_TOKEN');
const token = process.env[tokenEnv];
if (!token) {
console.error(`set ${tokenEnv} (channels:history is enough)`);
process.exitCode = 2;
return;
}
const channels = argAll(args, '--channel');
if (!channels.length) {
console.error('pass at least one --channel; a test channel is all plain messages ' +
'and will show you nothing');
process.exitCode = 2;
return;
}
const limit = arg(args, '--limit', '200');
const window = Number(arg(args, '--window', 90));
const who = await get(token, 'auth.test', {}, 'auth.test');
if (!who) { process.exitCode = 2; return; }
const botUser = who.user_id ?? '';
const botId = who.bot_id ?? '';
console.log(`identity ${botUser} in ${who.team}`);
let findings = 0;
for (const channel of channels) {
const body = await get(token, 'conversations.history',
{ channel, limit: String(limit) }, 'history');
if (!body) continue;
const messages = body.messages ?? [];
const p = profile(messages);
(p.sharePercent >= 10.0 ? console.warn : console.log)(
`profile ${p.total} message(s): ${p.plain} plain, ${p.subtyped} subtyped, ` +
`${p.edited} edited, ${p.sharePercent}% not a new message`);
const kinds = [...p.kinds.entries()]
.sort((a, b) => b[1].count - a[1].count || a[0].localeCompare(b[0]));
for (const [kind, info] of kinds) {
if (kind === 'plain') continue;
const loud = info.disposition === 'branch' || info.disposition === 'unknown-subtype';
(loud ? console.warn : console.log)(
`kind ${kind.padEnd(16)} ${String(info.count).padStart(3)} ` +
`${info.disposition.padEnd(10)} ${info.note}`);
if (loud) findings += 1;
}
const rows = editEcho(messages, botId, botUser, window);
const caught = rows.filter((r) => r[2] === 'reprocessed');
if (caught.length) {
findings += caught.length;
for (const [editedTs, originalTs, , detail] of caught) {
console.warn(`edits reprocessed edit at ${editedTs} of a message ` +
`posted at ${originalTs}: ${detail}`);
}
} else {
console.log(`edits clean 0 of ${rows.length} edit(s) were ` +
'followed by a reply');
}
}
if (findings) {
console.warn(' repair: switch on event.subtype with the absent case as the plain ' +
'new message, and name every other branch explicitly');
console.warn(' repair: for message_changed read event.message.text and compare it ' +
'against event.previous_message.text; event.text is not there');
console.warn(' repair: ignore channel_join, channel_leave and message_deleted ' +
'unless you specifically want them, and treat file_share and thread_broadcast ' +
'as real content');
process.exitCode = 1;
}
}
// Only run when invoked directly, so importing this module in the tests does
// not execute main() and fail the file on a missing token.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The assertions worth arguing about are the two that decline to fire. bot_message has to come back as elsewhere, because the echo loop is a different note and a row reported twice gets fixed twice or not at all. And an edit made seconds after the original message has to come back as too-close rather than as a finding: the correlation this script rests on needs the two timestamps to be separable, and pretending otherwise would turn a typo fixed on the spot into a bug report.
from slack_message_subtypes import classify, edit_echo, profile
def msg(ts, subtype=None, text="", user="U9", bot_id="", edited_ts=None):
m = {"ts": str(ts), "text": text, "user": user}
if subtype:
m["subtype"] = subtype
if bot_id:
m["bot_id"] = bot_id
if edited_ts is not None:
m["edited"] = {"user": user, "ts": str(edited_ts)}
return m
def test_a_plain_message_is_the_only_kind_event_text_is_right_for():
kind, path, disposition, note = classify(msg(10, text="deploy please"))
assert (kind, path, disposition) == ("plain", "text", "new-message")
assert "event.text" in note
def test_message_changed_hides_the_text_one_level_down():
kind, path, disposition, note = classify({"subtype": "message_changed"})
assert (kind, path, disposition) == ("message_changed", "message.text", "branch")
assert "previous_message" in note
def test_a_history_message_with_an_edited_block_is_the_same_bug_seen_from_history():
kind, path, disposition, note = classify(msg(10, text="typo", edited_ts=99))
assert kind == "edited-in-place"
assert path == "message.text"
assert disposition == "branch"
assert "message_changed" in note
def test_joins_and_leaves_carry_text_and_must_be_ignored_deliberately():
assert classify(msg(10, "channel_join", "Ana has joined the channel"))[2] == "ignore"
assert classify(msg(11, "channel_leave"))[2] == "ignore"
assert classify(msg(12, "message_deleted"))[2] == "ignore"
def test_a_deletion_points_at_the_previous_message_for_its_content():
assert classify(msg(10, "message_deleted"))[1] == "previous_message.text"
def test_bot_message_is_handed_to_the_echo_loop_note_not_claimed():
kind, _, disposition, note = classify(msg(10, "bot_message", bot_id="B1"))
assert kind == "bot_message"
assert disposition == "elsewhere"
assert "own note" in note
def test_file_shares_and_broadcasts_are_real_content():
assert classify(msg(10, "file_share"))[2] == "content"
assert classify(msg(10, "file_share"))[1] == "files"
assert classify(msg(11, "thread_broadcast"))[2] == "content"
def test_an_unrecognised_subtype_is_reported_rather_than_treated_as_new():
kind, _, disposition, note = classify(msg(10, "some_future_subtype"))
assert kind == "some_future_subtype"
assert disposition == "unknown-subtype"
assert "reference" in note
def test_the_profile_counts_the_share_a_blind_handler_gets_wrong():
p = profile([msg(1), msg(2), msg(3, "channel_join"), msg(4, text="x", edited_ts=9)])
assert p["total"] == 4
assert p["plain"] == 2
assert p["subtyped"] == 1
assert p["edited"] == 1
assert p["share_percent"] == 50.0
def test_an_empty_sample_does_not_divide_by_zero():
p = profile([])
assert p["total"] == 0 and p["share_percent"] == 0.0 and p["kinds"] == {}
def test_the_profile_keeps_the_disposition_with_each_kind():
p = profile([msg(1, "channel_join"), msg(2, "channel_join")])
assert p["kinds"]["channel_join"]["count"] == 2
assert p["kinds"]["channel_join"]["disposition"] == "ignore"
def test_a_reply_after_the_edit_and_not_the_original_is_the_proof():
rows = edit_echo([msg(1000, text="typo", edited_ts=5000),
msg(5010, bot_id="B1", user="U1")], "B1", "U1")
assert rows[0][2] == "reprocessed"
assert "handled as a new message" in rows[0][3]
def test_a_reply_to_the_original_and_to_the_edit_is_inconclusive():
rows = edit_echo([msg(1000, text="typo", edited_ts=5000),
msg(1010, bot_id="B1", user="U1"),
msg(5010, bot_id="B1", user="U1")], "B1", "U1")
assert rows[0][2] == "inconclusive"
def test_an_edit_with_no_reply_after_it_is_clean():
rows = edit_echo([msg(1000, text="typo", edited_ts=5000)], "B1", "U1")
assert rows[0][2] == "clean"
def test_an_edit_made_seconds_later_cannot_be_separated_from_the_original():
rows = edit_echo([msg(1000, text="typo", edited_ts=1020),
msg(1030, bot_id="B1", user="U1")], "B1", "U1")
assert rows[0][2] == "too-close"
def test_another_apps_reply_is_not_counted_as_ours():
rows = edit_echo([msg(1000, text="typo", edited_ts=5000),
msg(5010, bot_id="B_OTHER", user="U_OTHER")], "B1", "U1")
assert rows[0][2] == "clean"
def test_history_with_no_edits_produces_no_rows():
assert edit_echo([msg(1), msg(2, "channel_join")], "B1", "U1") == []
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { classify, editEcho, profile } from './slack-message-subtypes.mjs';
function msg(ts, subtype = null, text = '', user = 'U9', botId = '', editedTs = null) {
const m = { ts: String(ts), text, user };
if (subtype) m.subtype = subtype;
if (botId) m.bot_id = botId;
if (editedTs !== null) m.edited = { user, ts: String(editedTs) };
return m;
}
test('a plain message is the only kind event.text is right for', () => {
const [kind, path, disposition, note] = classify(msg(10, null, 'deploy please'));
assert.deepEqual([kind, path, disposition], ['plain', 'text', 'new-message']);
assert.match(note, /event\.text/);
});
test('message_changed hides the text one level down', () => {
const [kind, path, disposition, note] = classify({ subtype: 'message_changed' });
assert.deepEqual([kind, path, disposition],
['message_changed', 'message.text', 'branch']);
assert.match(note, /previous_message/);
});
test('a history message with an edited block is the same bug seen from history', () => {
const [kind, path, disposition, note] = classify(msg(10, null, 'typo', 'U9', '', 99));
assert.equal(kind, 'edited-in-place');
assert.equal(path, 'message.text');
assert.equal(disposition, 'branch');
assert.match(note, /message_changed/);
});
test('joins and leaves carry text and must be ignored deliberately', () => {
assert.equal(classify(msg(10, 'channel_join', 'Ana has joined the channel'))[2],
'ignore');
assert.equal(classify(msg(11, 'channel_leave'))[2], 'ignore');
assert.equal(classify(msg(12, 'message_deleted'))[2], 'ignore');
});
test('a deletion points at the previous message for its content', () => {
assert.equal(classify(msg(10, 'message_deleted'))[1], 'previous_message.text');
});
test('bot_message is handed to the echo loop note not claimed', () => {
const [kind, , disposition, note] = classify(msg(10, 'bot_message', '', 'U9', 'B1'));
assert.equal(kind, 'bot_message');
assert.equal(disposition, 'elsewhere');
assert.match(note, /own note/);
});
test('file shares and broadcasts are real content', () => {
assert.equal(classify(msg(10, 'file_share'))[2], 'content');
assert.equal(classify(msg(10, 'file_share'))[1], 'files');
assert.equal(classify(msg(11, 'thread_broadcast'))[2], 'content');
});
test('an unrecognised subtype is reported rather than treated as new', () => {
const [kind, , disposition, note] = classify(msg(10, 'some_future_subtype'));
assert.equal(kind, 'some_future_subtype');
assert.equal(disposition, 'unknown-subtype');
assert.match(note, /reference/);
});
test('the profile counts the share a blind handler gets wrong', () => {
const p = profile([msg(1), msg(2), msg(3, 'channel_join'),
msg(4, null, 'x', 'U9', '', 9)]);
assert.equal(p.total, 4);
assert.equal(p.plain, 2);
assert.equal(p.subtyped, 1);
assert.equal(p.edited, 1);
assert.equal(p.sharePercent, 50.0);
});
test('an empty sample does not divide by zero', () => {
const p = profile([]);
assert.equal(p.total, 0);
assert.equal(p.sharePercent, 0.0);
assert.equal(p.kinds.size, 0);
});
test('the profile keeps the disposition with each kind', () => {
const p = profile([msg(1, 'channel_join'), msg(2, 'channel_join')]);
assert.equal(p.kinds.get('channel_join').count, 2);
assert.equal(p.kinds.get('channel_join').disposition, 'ignore');
});
test('a reply after the edit and not the original is the proof', () => {
const rows = editEcho([msg(1000, null, 'typo', 'U9', '', 5000),
msg(5010, null, '', 'U1', 'B1')], 'B1', 'U1');
assert.equal(rows[0][2], 'reprocessed');
assert.match(rows[0][3], /handled as a new message/);
});
test('a reply to the original and to the edit is inconclusive', () => {
const rows = editEcho([msg(1000, null, 'typo', 'U9', '', 5000),
msg(1010, null, '', 'U1', 'B1'), msg(5010, null, '', 'U1', 'B1')], 'B1', 'U1');
assert.equal(rows[0][2], 'inconclusive');
});
test('an edit with no reply after it is clean', () => {
const rows = editEcho([msg(1000, null, 'typo', 'U9', '', 5000)], 'B1', 'U1');
assert.equal(rows[0][2], 'clean');
});
test('an edit made seconds later cannot be separated from the original', () => {
const rows = editEcho([msg(1000, null, 'typo', 'U9', '', 1020),
msg(1030, null, '', 'U1', 'B1')], 'B1', 'U1');
assert.equal(rows[0][2], 'too-close');
});
test('another apps reply is not counted as ours', () => {
const rows = editEcho([msg(1000, null, 'typo', 'U9', '', 5000),
msg(5010, null, '', 'U_OTHER', 'B_OTHER')], 'B1', 'U1');
assert.equal(rows[0][2], 'clean');
});
test('history with no edits produces no rows', () => {
assert.deepEqual(editEcho([msg(1), msg(2, 'channel_join')], 'B1', 'U1'), []);
});
FAQ
How is this different from the bot replying to its own messages?
Direction. The echo loop is the app reacting to its own output and it is loud: a run of self-authored messages seconds apart, usually noticed within minutes. This is the app mis-reading traffic from people, one message at a time, and it can run for months. They meet at exactly one subtype, bot_message, and this script classifies that row as elsewhere so the two audits do not both claim it.
Why does event.text come back empty on an edit instead of raising?
Because message_changed is a differently shaped payload rather than the same payload with a flag. The current text moved to event.message.text and the previous version sits at event.previous_message.text, so event.text simply is not a key. Reading an absent key gives you None in Python and undefined in JavaScript, both of which flow onward and get stored rather than throwing.
Does Bolt handle subtypes for me?
Partly, and the part it handles is the one that makes people think the rest is handled too. app.message() filters bot_message, which stops the echo loop. Joins, leaves, edits, deletions, file shares and thread broadcasts all still reach your listener as message events, and you still have to branch on subtype yourself.
Can this script tell me for certain that my handler is broken?
No, and it says so in the output. Nothing in the Web API reports what your handler did with an event. What the script can show is the share of traffic that is not a plain new message, which is how often a subtype-blind handler is wrong, and one correlation: replies clustered after an edit timestamp and not after the original. That correlation is evidence, and the script reports inconclusive whenever it is not clean evidence.
Should I subscribe to message.channels at all?
Only if you genuinely want every message in every channel the app is in. If what you actually want is to be addressed, app_mention is a much smaller and much better-behaved firehose: it fires when somebody names the app, it does not fire on joins and leaves, and it does not fire on your own posts. Most handlers that struggle with subtypes are subscribed one level too broadly.
Related field notes
- the one subtype this note hands over
- duplicates that come from delivery, not from subtypes
- the fifteen message clamp that shrinks this sample
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- message event reference - Slack Docs
- conversations.history method reference - Slack Docs
- Events API - Slack Docs
- Retrieving messages - Slack Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.