Diagnostic Slack
msg_blocks_too_long: the digest that grew past 50
The nightly digest has run for eight months. This morning it returned {"ok": false, "error": "msg_blocks_too_long"} and nobody deployed anything. The incident yesterday produced sixty failing checks instead of the usual twenty, the generator emitted one block per check, and the message crossed a ceiling that has been sitting there since the day it was written.
Nothing in the payload is malformed. Every block would pass validation on its own. This is a size rule that only exists at the message level, and it is the one payload failure that is a function of your traffic rather than of your code.
A message takes at most 50 blocks. Modals and Home tabs take 100. Past that, chat.postMessage answers msg_blocks_too_long and nothing is delivered.
There is a second ceiling that catches people harder, because it is on the encoded size of the payload rather than on the count. A message can sit comfortably at 20 blocks and still be refused, and reports of rejections cluster around 13,000 characters of block JSON. Twenty blocks each carrying 700 characters of log output crosses that long before it comes anywhere near block 50.
So the useful question is not "is this message legal" - today's is, or you would not be reading this. It is how much bigger can it get before it is not, and which of the two ceilings it hits first. This script measures both on a payload you hand it, or on every message your app has already posted, and reports the growth multiple that breaks each one.
The problem in plain words
What makes this expensive is that it fires on exactly the wrong day. The generator maps a collection to blocks, one per item, so the payload is proportional to how much went wrong. On a quiet Tuesday there are twelve items and the message is 15 blocks. During the incident there are sixty, and the message that would have told everybody what was failing is the one message that does not arrive. The alerting goes silent at the moment the alerting matters, and it goes silent with an error most people have never seen.
Then the investigation goes the wrong way twice. The first wrong turn is treating it as a validation bug and staring at the blocks, all of which are fine. The second is fixing it at the call site: wrapping the send in a try, catching the error, and retrying with the first 40 blocks. That works, and it means that from then on the digest silently drops content on precisely the busiest days, with no record of what was cut.
The size ceiling makes it worse by being invisible. A team that learns about the 50-block limit adds an assertion on len(blocks), watches it pass, and is rejected anyway because the blocks got fatter rather than more numerous. Nothing in the error string distinguishes the two, so the assertion that was supposed to prevent the problem becomes evidence that the problem is something else.
And the growth is invisible in the ordinary case. A message at 34 blocks looks perfectly healthy in a channel. Nobody counts blocks by eye. The only way to know you are one busy day from the ceiling is to measure the tail of what you have already sent, which nobody does until after the first failure.
Why it happens
This is a capacity finding, not a correctness one. Every block passes. The message passes today. The output that matters is a number with a unit: at 1.6 times today's volume, this breaks. That is a sentence a team can act on before the incident, which is the only useful time to act on it.
Two ceilings, and the smaller one is not always the obvious one. The script computes the growth multiple against the block count and against the encoded size, and names whichever comes first. A digest of many short lines is block-bound; a digest carrying stack traces is size-bound. Those have different repairs, and guessing which one you have is how a cap on block count fails to fix anything.
The size ceiling is observed, not documented, and the script says so. Slack publishes the block counts. The payload size at which rejections start is a figure the community converged on rather than one you can cite, so it is a configurable default that is labelled as observed. An audit that presents a folk number as a specification is worse than one that presents it honestly.
The median tells you nothing; the tail is the whole story. Reporting the average message size of a generator whose output is proportional to failures is close to useless. The script reports the maximum, the 90th percentile and the single tightest message in the sample, because the message that breaks is by definition the biggest one.
The cap belongs in the generator, not at the call site. A cap where the blocks are built can decide what to keep, count what it dropped, and say so in the message. A cap at the send is a truncation with no knowledge of what it removed, which is how a digest quietly stops mentioning the worst failures.
Measuring the size means encoding it the way Slack receives it. The number that counts is the length of the JSON on the wire, with no pretty-printing, not the length of your Python objects. The script encodes with compact separators for exactly that reason, and a whitespace-heavy encoder is itself a finding worth knowing about.
The fix, as a flow
Nothing in this chain is a mistake and nothing was deployed. The generator is correct, every block is valid, and the input got bigger, so the red box is the last one rather than the first. The fix branch is a capacity report rather than a diagnosis: two ceilings, and the useful output is which of them your payload reaches first, because they want opposite repairs.
How to fix it
Measure the payload you are about to send, with no token
--payload takes the JSON your generator produced and reports the count, the encoded size and the growth multiple. No Slack call, no message, no channel. This is the check that belongs in a test for the generator rather than in an incident.
Encode the way the API receives it
budget serialises with compact separators, because that is what goes on the wire. If your client sends indented JSON, the number it should be measuring is larger than the one your objects suggest, and that gap is worth knowing before it costs you a message.
Ask which ceiling arrives first
break_multiple returns a growth factor and the name of the ceiling that trips at it. blocks and size want different repairs: one wants fewer items, the other wants shorter ones. A message that is size-bound will not be saved by capping the block count.
Read the tail of what you have already sent
With a token and a channel, the script measures every message your app posted and reports the maximum, the 90th percentile and the tightest single message. A distribution whose tail sits at 40 blocks is not healthy because the median is 12.
State the busy day as a number
--growth is your multiple for a bad day: three times the usual volume, say. at_growth reports how many of the messages you have already sent would be refused at that multiple. That converts "we might have a problem" into "eleven of last week's forty messages fail at 3x".
Cap where the blocks are built, and say what was dropped
Emit at most about 45 blocks, append a footer saying how many items are not shown and where to find them, and thread or upload the remainder. The number in that footer is the thing a truncation at the call site can never produce.
How to check it worked
Add the cap to the generator, then run the measurement over the same fixture that broke. The growth multiple should be the thing that changed, not the block count alone.
python3 slack_block_budget.py --payload nightly-digest.json --growth 3
# payload 62 block(s), 18944 encoded characters
# headroom over 62 blocks is past the 50 a message takes
# ceiling blocks over already: the block count went first, at 62 of 50
# growth would-break at 3.0x this payload is refused; it already is
# repair: cap the generator at about 45 blocks and append a showing N of M footerThe full code
Four pure functions and no cleverness. budget is two measurements taken the way the API takes them. headroom is the state today. break_multiple is the one worth having: a growth factor and the name of the ceiling that arrives first, which is the difference between a repair that shortens the list and one that shortens the lines. at_growth turns a fleet of past messages into how many of them a bad day would have destroyed.
"""Measure how much bigger your messages can get before Slack refuses them.
Read only, and with --payload it holds no token and calls nothing. Nothing is
sent from here: finding the ceiling by walking a generator up to it is how a
channel fills with half-rendered test digests. This measures a payload you hand
it, or the payloads conversations.history says your app already sent, and
reports the growth multiple at which each ceiling arrives.
"""
import argparse
import json
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("slack_block_budget")
API = "https://slack.com/api/"
# Documented block counts per surface.
SURFACES = {"message": 50, "modal": 100, "home": 100}
# Not documented. Rejections are widely reported to begin around this much
# encoded block JSON, and the script labels it observed everywhere it prints
# it: presenting a folk number as a specification is worse than presenting it
# honestly, and this one is a --size-ceiling away from being yours instead.
OBSERVED_SIZE_CEILING = 13200
# Above this share of either ceiling a payload is one bad day from failing.
AT_RISK = 0.8
def budget(blocks):
"""Count the two things that are actually limited. Pure.
Returns (count, size). size is the length of the JSON as the API receives
it, encoded with compact separators, because indentation in your client is
payload too and the ceiling does not care that it is only whitespace.
"""
if not isinstance(blocks, list):
return (0, 0)
encoded = json.dumps(blocks, separators=(",", ":"), ensure_ascii=False)
return (len(blocks), len(encoded))
def headroom(count, size, surface="message", size_ceiling=OBSERVED_SIZE_CEILING):
"""Where this payload stands today. Pure.
Returns (verdict, detail): over, at-risk, roomy, empty or unknown-surface.
"""
limit = SURFACES.get(surface)
if limit is None:
return ("unknown-surface", "%r is not one of %s" % (
surface, ", ".join(sorted(SURFACES))))
if count <= 0:
return ("empty", "no blocks, so there is nothing to measure")
if count > limit:
return ("over", "%d blocks is past the %d a %s takes" % (count, limit, surface))
if size > size_ceiling:
return ("over", "%d encoded characters is past the observed ceiling of %d, "
"with only %d blocks: this one is bound by size and not by "
"count" % (size, size_ceiling, count))
if count >= limit * AT_RISK or size >= size_ceiling * AT_RISK:
return ("at-risk", "%d of %d blocks and %d of an observed %d characters; this "
"is inside the last fifth of both budgets" % (
count, limit, size, size_ceiling))
return ("roomy", "%d of %d blocks and %d of an observed %d characters" % (
count, limit, size, size_ceiling))
def break_multiple(count, size, surface="message", size_ceiling=OBSERVED_SIZE_CEILING):
"""How much bigger can this get, and which ceiling arrives first. Pure.
Returns (multiple, ceiling, detail). The ceiling name is the useful half:
a block-bound message wants fewer items and a size-bound one wants shorter
ones, and a cap on the block count does nothing at all for the second.
"""
limit = SURFACES.get(surface)
if limit is None:
return (0.0, "unknown-surface", "%r is not a surface this script knows" % surface)
if count <= 0:
return (0.0, "empty", "no blocks, so there is no ceiling to grow into")
if count > limit:
return (0.0, "blocks", "over already: the block count went first, at %d of %d"
% (count, limit))
if size > size_ceiling:
return (0.0, "size", "over already: the encoded size went first, at %d of an "
"observed %d, on only %d blocks"
% (size, size_ceiling, count))
by_blocks = limit / float(count)
by_size = (size_ceiling / float(size)) if size > 0 else by_blocks
if by_blocks <= by_size:
return (round(by_blocks, 2), "blocks",
"%.2fx more items and the block count reaches %d; fewer rows is the "
"repair, not shorter ones" % (by_blocks, limit))
return (round(by_size, 2), "size",
"%.2fx more content and the encoded payload reaches an observed %d, while "
"the block count is still only %d of %d; shorter rows is the repair"
% (by_size, size_ceiling, count, limit))
def at_growth(samples, growth, surface="message", size_ceiling=OBSERVED_SIZE_CEILING):
"""How many of these payloads a busy day of this size would destroy. Pure.
samples is [(count, size)]. Returns (breaking, total, detail). The point of
the function is to answer with a count of real messages rather than with a
hypothetical, because "eleven of last week's forty" is an argument and "we
might have a problem" is not.
"""
rows = [s for s in samples or [] if s and s[0] > 0]
total = len(rows)
if not total:
return (0, 0, "no payloads with blocks in the sample")
factor = float(growth)
if factor <= 0:
return (0, total, "a growth factor of %s cannot break anything" % growth)
breaking = 0
ceilings = {"blocks": 0, "size": 0}
for count, size in rows:
multiple, ceiling, _ = break_multiple(count, size, surface, size_ceiling)
if multiple <= factor:
breaking += 1
if ceiling in ceilings:
ceilings[ceiling] += 1
if not breaking:
return (0, total, "none of %d payload(s) break at %.1fx today's volume" % (
total, factor))
return (breaking, total, "%d of %d payload(s) are refused at %.1fx today's volume: "
"%d on the block count, %d on the encoded size" % (
breaking, total, factor, ceilings["blocks"],
ceilings["size"]))
def distribution(samples):
"""Reduce a fleet of payloads to the tail that decides. Pure.
The median of a generator whose output is proportional to failures is close
to meaningless, so this reports the maximum, the 90th percentile and the
single tightest payload in the sample.
"""
rows = [s for s in samples or [] if s and s[0] > 0]
if not rows:
return {"n": 0, "max_count": 0, "max_size": 0, "p90_count": 0, "p90_size": 0}
counts = sorted(r[0] for r in rows)
sizes = sorted(r[1] for r in rows)
idx = int(round(0.9 * (len(rows) - 1)))
return {"n": len(rows), "max_count": counts[-1], "max_size": sizes[-1],
"p90_count": counts[idx], "p90_size": sizes[idx]}
def tightest(samples, surface="message", size_ceiling=OBSERVED_SIZE_CEILING):
"""The payload with the least room left, and how much that is. Pure."""
best = None
for count, size in [s for s in samples or [] if s and s[0] > 0]:
multiple, ceiling, detail = break_multiple(count, size, surface, size_ceiling)
if best is None or multiple < best[0]:
best = (multiple, ceiling, detail, count, size)
return best
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--payload", default="",
help="a JSON file holding a blocks array or a whole message; "
"with this the script needs no Slack token at all")
ap.add_argument("--token-env", default="SLACK_BOT_TOKEN",
help="environment variable holding the bot token")
ap.add_argument("--channel", action="append", default=[],
help="a channel your app posts into; repeatable")
ap.add_argument("--limit", type=int, default=200, help="messages read per channel")
ap.add_argument("--surface", default="message", choices=sorted(SURFACES),
help="message takes 50 blocks; a modal or Home tab takes 100")
ap.add_argument("--size-ceiling", type=int, default=OBSERVED_SIZE_CEILING,
help="encoded characters at which rejections are observed to begin")
ap.add_argument("--growth", type=float, default=3.0,
help="how much bigger a bad day is than an ordinary one")
args = ap.parse_args()
if args.payload:
raw = json.loads(open(args.payload, encoding="utf-8").read())
blocks = raw if isinstance(raw, list) else (raw or {}).get("blocks")
count, size = budget(blocks)
log.info("payload %d block(s), %d encoded characters", count, size)
verdict, detail = headroom(count, size, args.surface, args.size_ceiling)
(log.warning if verdict in ("over", "at-risk") else log.info)(
"headroom %-14s %s", verdict, detail)
multiple, ceiling, why = break_multiple(count, size, args.surface,
args.size_ceiling)
(log.warning if multiple and multiple <= args.growth else log.info)(
"ceiling %-14s %s", ceiling, why)
breaking, total, growth_detail = at_growth([(count, size)], args.growth,
args.surface, args.size_ceiling)
(log.warning if breaking else log.info)(
"growth %-14s %s", "would-break" if breaking else "survives",
growth_detail)
if verdict == "over" or breaking:
log.warning(" repair: cap the generator at about 45 blocks and append a "
"showing N of M footer, so the truncation is visible")
log.warning(" repair: if the ceiling is size, shorten the rows; upload the "
"long content as a file snippet and reference it from a "
"short block")
return 1
return 0
token = os.environ.get(args.token_env)
if not token:
log.error("set %s, or pass --payload to measure a file with no token at all",
args.token_env)
return 2
if not args.channel:
log.error("pass at least one --channel, or --payload")
return 2
s = requests.Session()
s.headers.update({"Authorization": "Bearer " + token})
who = s.get(API + "auth.test", timeout=30).json()
if who.get("ok") is not True:
log.error("auth.test unavailable %s", who.get("error"))
return 2
bot_id = who.get("bot_id") or ""
log.info("identity %s in %s", who.get("user_id"), who.get("team"))
samples = []
for channel in args.channel:
body = s.get(API + "conversations.history", timeout=30,
params={"channel": channel, "limit": str(args.limit)}).json()
if body.get("ok") is not True:
log.warning("history unavailable %s: %s", channel, body.get("error"))
continue
for m in body.get("messages") or []:
if not m.get("blocks"):
continue
if bot_id and m.get("bot_id") != bot_id:
continue
samples.append(budget(m["blocks"]))
log.info("history %s: %d message(s) of ours carrying blocks", channel,
len(samples))
d = distribution(samples)
if not d["n"]:
log.info("sample empty nothing of ours in these channels used blocks")
return 0
log.info("sample %d payload(s), max %d blocks / %d chars, p90 %d blocks / %d chars",
d["n"], d["max_count"], d["max_size"], d["p90_count"], d["p90_size"])
worst = tightest(samples, args.surface, args.size_ceiling)
multiple, ceiling, why, count, size = worst
(log.warning if multiple <= args.growth else log.info)(
"tightest %-14s %d blocks / %d chars: %s", ceiling, count, size, why)
breaking, total, growth_detail = at_growth(samples, args.growth, args.surface,
args.size_ceiling)
(log.warning if breaking else log.info)(
"growth %-14s %s", "would-break" if breaking else "survives", growth_detail)
if breaking or multiple <= args.growth:
log.warning(" repair: cap the block count in the generator at about 45 and "
"append a showing N of M footer with a link to the rest")
log.warning(" repair: for a size-bound payload, thread the remainder with "
"thread_ts or upload it as a file snippet and post a summary")
log.warning(" repair: for a modal, paginate the view and move the rest behind "
"a Next button that calls views.update")
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
/**
* Measure how much bigger your messages can get before Slack refuses them.
*
* Read only, and with --payload it holds no token and calls nothing. Nothing is
* sent from here: finding the ceiling by walking a generator up to it is how a
* channel fills with half-rendered test digests. This measures a payload you
* hand it, or the payloads conversations.history says your app already sent.
*/
import { readFile } from 'node:fs/promises';
const API = 'https://slack.com/api/';
// Documented block counts per surface.
const SURFACES = new Map([['message', 50], ['modal', 100], ['home', 100]]);
// Not documented. Rejections are widely reported to begin around this much
// encoded block JSON, and it is labelled observed everywhere it is printed:
// presenting a folk number as a specification is worse than being honest.
const OBSERVED_SIZE_CEILING = 13200;
const AT_RISK = 0.8;
/**
* Count the two things that are actually limited. Pure.
* Returns [count, size], where size is the JSON the API receives.
*/
export function budget(blocks) {
if (!Array.isArray(blocks)) return [0, 0];
return [blocks.length, JSON.stringify(blocks).length];
}
/**
* Where this payload stands today. Pure.
* Returns [verdict, detail]: over, at-risk, roomy, empty or unknown-surface.
*/
export function headroom(count, size, surface = 'message',
sizeCeiling = OBSERVED_SIZE_CEILING) {
const limit = SURFACES.get(surface);
if (limit === undefined) {
return ['unknown-surface',
`${surface} is not one of ${[...SURFACES.keys()].sort().join(', ')}`];
}
if (count <= 0) return ['empty', 'no blocks, so there is nothing to measure'];
if (count > limit) {
return ['over', `${count} blocks is past the ${limit} a ${surface} takes`];
}
if (size > sizeCeiling) {
return ['over', `${size} encoded characters is past the observed ceiling of ` +
`${sizeCeiling}, with only ${count} blocks: this one is bound by size and not ` +
'by count'];
}
if (count >= limit * AT_RISK || size >= sizeCeiling * AT_RISK) {
return ['at-risk', `${count} of ${limit} blocks and ${size} of an observed ` +
`${sizeCeiling} characters; this is inside the last fifth of both budgets`];
}
return ['roomy',
`${count} of ${limit} blocks and ${size} of an observed ${sizeCeiling} characters`];
}
/**
* How much bigger can this get, and which ceiling arrives first. Pure.
* Returns [multiple, ceiling, detail]. The ceiling name is the useful half: a
* block-bound message wants fewer items and a size-bound one wants shorter
* ones, and a cap on the block count does nothing at all for the second.
*/
export function breakMultiple(count, size, surface = 'message',
sizeCeiling = OBSERVED_SIZE_CEILING) {
const limit = SURFACES.get(surface);
if (limit === undefined) {
return [0, 'unknown-surface', `${surface} is not a surface this script knows`];
}
if (count <= 0) return [0, 'empty', 'no blocks, so there is no ceiling to grow into'];
if (count > limit) {
return [0, 'blocks',
`over already: the block count went first, at ${count} of ${limit}`];
}
if (size > sizeCeiling) {
return [0, 'size', `over already: the encoded size went first, at ${size} of an ` +
`observed ${sizeCeiling}, on only ${count} blocks`];
}
const byBlocks = limit / count;
const bySize = size > 0 ? sizeCeiling / size : byBlocks;
if (byBlocks <= bySize) {
return [Math.round(byBlocks * 100) / 100, 'blocks',
`${byBlocks.toFixed(2)}x more items and the block count reaches ${limit}; ` +
'fewer rows is the repair, not shorter ones'];
}
return [Math.round(bySize * 100) / 100, 'size',
`${bySize.toFixed(2)}x more content and the encoded payload reaches an observed ` +
`${sizeCeiling}, while the block count is still only ${count} of ${limit}; ` +
'shorter rows is the repair'];
}
/**
* How many of these payloads a busy day of this size would destroy. Pure.
* Answers with a count of real messages rather than a hypothetical.
*/
export function atGrowth(samples, growth, surface = 'message',
sizeCeiling = OBSERVED_SIZE_CEILING) {
const rows = (samples ?? []).filter((s) => s && s[0] > 0);
const total = rows.length;
if (!total) return [0, 0, 'no payloads with blocks in the sample'];
const factor = Number(growth);
if (!(factor > 0)) return [0, total, `a growth factor of ${growth} cannot break anything`];
let breaking = 0;
const ceilings = { blocks: 0, size: 0 };
for (const [count, size] of rows) {
const [multiple, ceiling] = breakMultiple(count, size, surface, sizeCeiling);
if (multiple <= factor) {
breaking += 1;
if (ceiling in ceilings) ceilings[ceiling] += 1;
}
}
if (!breaking) {
return [0, total,
`none of ${total} payload(s) break at ${factor.toFixed(1)}x today's volume`];
}
return [breaking, total, `${breaking} of ${total} payload(s) are refused at ` +
`${factor.toFixed(1)}x today's volume: ${ceilings.blocks} on the block count, ` +
`${ceilings.size} on the encoded size`];
}
/**
* Reduce a fleet of payloads to the tail that decides. Pure.
* The median of a generator whose output is proportional to failures is close
* to meaningless.
*/
export function distribution(samples) {
const rows = (samples ?? []).filter((s) => s && s[0] > 0);
if (!rows.length) {
return { n: 0, maxCount: 0, maxSize: 0, p90Count: 0, p90Size: 0 };
}
const counts = rows.map((r) => r[0]).sort((a, b) => a - b);
const sizes = rows.map((r) => r[1]).sort((a, b) => a - b);
const idx = Math.round(0.9 * (rows.length - 1));
return { n: rows.length,
maxCount: counts[counts.length - 1],
maxSize: sizes[sizes.length - 1],
p90Count: counts[idx],
p90Size: sizes[idx] };
}
/** The payload with the least room left, and how much that is. Pure. */
export function tightest(samples, surface = 'message',
sizeCeiling = OBSERVED_SIZE_CEILING) {
let best = null;
for (const [count, size] of (samples ?? []).filter((s) => s && s[0] > 0)) {
const [multiple, ceiling, detail] = breakMultiple(count, size, surface, sizeCeiling);
if (best === null || multiple < best[0]) best = [multiple, ceiling, detail, count, size];
}
return best;
}
function arg(args, name, fallback = null) {
const i = args.indexOf(name);
return i === -1 ? fallback : args[i + 1];
}
function argAll(args, name) {
const out = [];
for (let i = 0; i < args.length; i += 1) {
if (args[i] === name && args[i + 1] !== undefined) out.push(args[i + 1]);
}
return out;
}
async function main() {
const args = process.argv.slice(2);
const surface = arg(args, '--surface', 'message');
const sizeCeiling = Number(arg(args, '--size-ceiling', OBSERVED_SIZE_CEILING));
const growth = Number(arg(args, '--growth', '3'));
const payload = arg(args, '--payload', '');
if (payload) {
const raw = JSON.parse(await readFile(payload, 'utf8'));
const blocks = Array.isArray(raw) ? raw : (raw ?? {}).blocks;
const [count, size] = budget(blocks);
console.log(`payload ${count} block(s), ${size} encoded characters`);
const [verdict, detail] = headroom(count, size, surface, sizeCeiling);
(verdict === 'over' || verdict === 'at-risk' ? console.warn : console.log)(
`headroom ${verdict.padEnd(14)} ${detail}`);
const [multiple, ceiling, why] = breakMultiple(count, size, surface, sizeCeiling);
(multiple <= growth ? console.warn : console.log)(
`ceiling ${ceiling.padEnd(14)} ${why}`);
const [breaking, , growthDetail] = atGrowth([[count, size]], growth, surface,
sizeCeiling);
(breaking ? console.warn : console.log)(
`growth ${(breaking ? 'would-break' : 'survives').padEnd(14)} ${growthDetail}`);
if (verdict === 'over' || breaking) {
console.warn(' repair: cap the generator at about 45 blocks and append a ' +
'showing N of M footer, so the truncation is visible');
console.warn(' repair: if the ceiling is size, shorten the rows; upload the long ' +
'content as a file snippet and reference it from a short block');
process.exitCode = 1;
}
return;
}
const tokenEnv = arg(args, '--token-env', 'SLACK_BOT_TOKEN');
const token = process.env[tokenEnv];
if (!token) {
console.error(`set ${tokenEnv}, or pass --payload to measure a file with no token`);
process.exitCode = 2;
return;
}
const channels = argAll(args, '--channel');
if (!channels.length) {
console.error('pass at least one --channel, or --payload');
process.exitCode = 2;
return;
}
const limit = arg(args, '--limit', '200');
const headers = { Authorization: `Bearer ${token}` };
const who = await (await fetch(`${API}auth.test`, { headers })).json();
if (who.ok !== true) {
console.error(`auth.test unavailable ${who.error}`);
process.exitCode = 2;
return;
}
const botId = who.bot_id ?? '';
console.log(`identity ${who.user_id} in ${who.team}`);
const samples = [];
for (const channel of channels) {
const url = `${API}conversations.history?channel=${encodeURIComponent(channel)}` +
`&limit=${encodeURIComponent(limit)}`;
const body = await (await fetch(url, { headers })).json();
if (body.ok !== true) {
console.warn(`history unavailable ${channel}: ${body.error}`);
continue;
}
for (const m of body.messages ?? []) {
if (!m.blocks) continue;
if (botId && m.bot_id !== botId) continue;
samples.push(budget(m.blocks));
}
console.log(`history ${channel}: ${samples.length} message(s) of ours with blocks`);
}
const d = distribution(samples);
if (!d.n) {
console.log('sample empty nothing of ours in these channels used blocks');
return;
}
console.log(`sample ${d.n} payload(s), max ${d.maxCount} blocks / ${d.maxSize} ` +
`chars, p90 ${d.p90Count} blocks / ${d.p90Size} chars`);
const [multiple, ceiling, why, count, size] = tightest(samples, surface, sizeCeiling);
(multiple <= growth ? console.warn : console.log)(
`tightest ${ceiling.padEnd(14)} ${count} blocks / ${size} chars: ${why}`);
const [breaking, , growthDetail] = atGrowth(samples, growth, surface, sizeCeiling);
(breaking ? console.warn : console.log)(
`growth ${(breaking ? 'would-break' : 'survives').padEnd(14)} ${growthDetail}`);
if (breaking || multiple <= growth) {
console.warn(' repair: cap the block count in the generator at about 45 and append ' +
'a showing N of M footer with a link to the rest');
console.warn(' repair: for a size-bound payload, thread the remainder with ' +
'thread_ts or upload it as a file snippet and post a summary');
console.warn(' repair: for a modal, paginate the view and move the rest behind a ' +
'Next button that calls views.update');
process.exitCode = 1;
}
}
// Only run when invoked directly, so importing this module in the tests does
// not execute main() and fail the file on a missing token.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The assertion that matters most is that a payload can be size-bound while sitting at a quarter of the block count, because that is the case an assertion on len(blocks) was written to catch and does not. The rest keep the arithmetic honest: a message already over reports a growth multiple of zero rather than a negative one, an empty payload is not treated as infinitely safe, and the growth calculation reports how many real messages break rather than a probability.
import json
from slack_block_budget import (at_growth, break_multiple, budget, distribution,
headroom, tightest)
def blocks(n, text="x"):
return [{"type": "section", "text": {"type": "mrkdwn", "text": text}}
for _ in range(n)]
def test_budget_measures_the_encoding_the_api_receives():
count, size = budget(blocks(3))
assert count == 3
assert size == len(json.dumps(blocks(3), separators=(",", ":")))
def test_budget_of_something_that_is_not_a_list_is_zero():
assert budget(None) == (0, 0)
assert budget({"blocks": []}) == (0, 0)
def test_fifty_one_blocks_is_over_and_fifty_is_not():
assert headroom(51, 100)[0] == "over"
assert headroom(50, 100)[0] == "at-risk"
def test_a_modal_gets_a_hundred_blocks():
assert headroom(60, 100, "modal")[0] == "roomy"
assert headroom(60, 100, "message")[0] == "over"
assert headroom(1, 1, "carousel")[0] == "unknown-surface"
def test_a_payload_can_be_over_on_size_with_room_on_count():
verdict, detail = headroom(20, 18000)
assert verdict == "over"
assert "bound by size and not by count" in detail
def test_the_block_ceiling_arrives_first_for_many_short_rows():
multiple, ceiling, detail = break_multiple(25, 3000)
assert ceiling == "blocks"
assert multiple == 2.0
assert "fewer rows" in detail
def test_the_size_ceiling_arrives_first_for_few_long_rows():
multiple, ceiling, detail = break_multiple(10, 6600)
assert ceiling == "size"
assert multiple == 2.0
assert "shorter rows" in detail
def test_a_payload_already_over_reports_zero_not_a_negative_multiple():
assert break_multiple(60, 100) == (0.0, "blocks",
"over already: the block count went first, "
"at 60 of 50")
assert break_multiple(10, 20000)[1] == "size"
def test_an_empty_payload_is_not_treated_as_infinitely_safe():
assert break_multiple(0, 0)[1] == "empty"
assert headroom(0, 0)[0] == "empty"
def test_growth_counts_real_messages_rather_than_guessing():
samples = [(10, 1000), (20, 2000), (45, 4500)]
breaking, total, detail = at_growth(samples, 2.0)
assert (breaking, total) == (1, 3)
assert "1 of 3" in detail and "on the block count" in detail
def test_growth_names_which_ceiling_each_breakage_hit():
breaking, _, detail = at_growth([(45, 1000), (5, 9000)], 2.0)
assert breaking == 2
assert "1 on the block count, 1 on the encoded size" in detail
def test_a_zero_or_negative_growth_factor_breaks_nothing():
assert at_growth([(45, 4500)], 0)[0] == 0
assert at_growth([], 3.0)[1] == 0
def test_distribution_reports_the_tail_and_not_the_middle():
d = distribution([(2, 200)] * 8 + [(40, 4000), (48, 4800)])
assert d["n"] == 10
assert d["max_count"] == 48
assert d["p90_count"] == 40
assert d["max_size"] == 4800
def test_distribution_of_an_empty_sample_is_empty_rather_than_an_error():
assert distribution([])["n"] == 0
assert distribution(None)["n"] == 0
def test_tightest_finds_the_one_message_that_decides():
worst = tightest([(10, 1000), (40, 2000), (12, 12000)])
assert worst[1] == "size"
assert worst[3] == 12
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
atGrowth, breakMultiple, budget, distribution, headroom, tightest,
} from './slack-block-budget.mjs';
const blocks = (n, text = 'x') => Array.from({ length: n },
() => ({ type: 'section', text: { type: 'mrkdwn', text } }));
test('budget measures the encoding the api receives', () => {
const [count, size] = budget(blocks(3));
assert.equal(count, 3);
assert.equal(size, JSON.stringify(blocks(3)).length);
});
test('budget of something that is not a list is zero', () => {
assert.deepEqual(budget(null), [0, 0]);
assert.deepEqual(budget({ blocks: [] }), [0, 0]);
});
test('fifty one blocks is over and fifty is not', () => {
assert.equal(headroom(51, 100)[0], 'over');
assert.equal(headroom(50, 100)[0], 'at-risk');
});
test('a modal gets a hundred blocks', () => {
assert.equal(headroom(60, 100, 'modal')[0], 'roomy');
assert.equal(headroom(60, 100, 'message')[0], 'over');
assert.equal(headroom(1, 1, 'carousel')[0], 'unknown-surface');
});
test('a payload can be over on size with room on count', () => {
const [verdict, detail] = headroom(20, 18000);
assert.equal(verdict, 'over');
assert.match(detail, /bound by size and not by count/);
});
test('the block ceiling arrives first for many short rows', () => {
const [multiple, ceiling, detail] = breakMultiple(25, 3000);
assert.equal(ceiling, 'blocks');
assert.equal(multiple, 2);
assert.match(detail, /fewer rows/);
});
test('the size ceiling arrives first for few long rows', () => {
const [multiple, ceiling, detail] = breakMultiple(10, 6600);
assert.equal(ceiling, 'size');
assert.equal(multiple, 2);
assert.match(detail, /shorter rows/);
});
test('a payload already over reports zero not a negative multiple', () => {
const [multiple, ceiling, detail] = breakMultiple(60, 100);
assert.equal(multiple, 0);
assert.equal(ceiling, 'blocks');
assert.match(detail, /over already/);
assert.equal(breakMultiple(10, 20000)[1], 'size');
});
test('an empty payload is not treated as infinitely safe', () => {
assert.equal(breakMultiple(0, 0)[1], 'empty');
assert.equal(headroom(0, 0)[0], 'empty');
});
test('growth counts real messages rather than guessing', () => {
const [breaking, total, detail] = atGrowth([[10, 1000], [20, 2000], [45, 4500]], 2);
assert.equal(breaking, 1);
assert.equal(total, 3);
assert.match(detail, /1 of 3/);
});
test('growth names which ceiling each breakage hit', () => {
const [breaking, , detail] = atGrowth([[45, 1000], [5, 9000]], 2);
assert.equal(breaking, 2);
assert.match(detail, /1 on the block count, 1 on the encoded size/);
});
test('a zero or negative growth factor breaks nothing', () => {
assert.equal(atGrowth([[45, 4500]], 0)[0], 0);
assert.equal(atGrowth([], 3)[1], 0);
});
test('distribution reports the tail and not the middle', () => {
const samples = Array.from({ length: 8 }, () => [2, 200]);
samples.push([40, 4000], [48, 4800]);
const d = distribution(samples);
assert.equal(d.n, 10);
assert.equal(d.maxCount, 48);
assert.equal(d.p90Count, 40);
assert.equal(d.maxSize, 4800);
});
test('distribution of an empty sample is empty rather than an error', () => {
assert.equal(distribution([]).n, 0);
assert.equal(distribution(null).n, 0);
});
test('tightest finds the one message that decides', () => {
const worst = tightest([[10, 1000], [40, 2000], [12, 12000]]);
assert.equal(worst[1], 'size');
assert.equal(worst[3], 12);
});
FAQ
Is the 50-block limit really 50, or is it fewer in practice?
The count is 50 for a message and 100 for a modal or a Home tab, and those are documented. What is not documented is that a payload can be refused well under 50 blocks because the encoded JSON is too large; rejections are widely reported to start around 13,000 characters. The script treats the count as a specification and the size as an observed figure you can override, because that is what each of them actually is.
Why does the script measure characters instead of bytes?
Because the reported threshold is expressed in characters of encoded JSON and that is what people compare against. For ASCII content the two are the same number. For content with a lot of emoji or non-Latin text the byte count is larger, so if you are close to the ceiling with such content, treat the character measurement as the optimistic one and leave more room.
Can I just catch msg_blocks_too_long and retry with fewer blocks?
You can, and as a last-resort safety net in the send path it is better than losing the message. It is a bad fix, though, because the truncation happens where nothing knows what was removed, so the digest silently drops content on precisely the days the content matters most. Cap in the generator, where you can count what you dropped and say so.
Should I thread the overflow or upload it as a file?
Thread it when the extra content is more of the same and a reader might want to scroll it. Upload it when the extra content is bulk that nobody reads in a chat client, such as a full log or a diff. The deciding question is whether the overflow is more message or more artefact, and the two answers produce very different channels to live in.
Our messages are nowhere near 50 blocks. Do we still need this?
Run it once. The number you want is not the block count, it is the growth multiple, and a generator that emits one block per item has a multiple that is entirely determined by how bad a day can get. If the tightest message in a week of history breaks at 1.4x, you have a problem that has not happened yet, and that is the only time it is cheap to fix.
Related field notes
- when a single block is the thing that is wrong
- the per-field ceilings underneath this one
- before you make uploading the overflow your repair
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Block Kit - Slack Docs
- chat.postMessage method reference - Slack Docs
- views.open method reference - Slack Docs
- files.getUploadURLExternal method reference - Slack Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.