Diagnostic Slack
The Slack app configuration token died twelve hours ago
The manifest step in CI passed yesterday and fails today. Someone regenerates the token by hand, the build goes green, and the same thing happens tomorrow morning. Meanwhile the audit that runs beside it reports every manifest-derived check as clean, because a check that could not run returned nothing to complain about.
App configuration tokens are a separate credential class from the tokens your app runs on. They are used only by the apps.manifest.* and tooling.tokens.* families, they are issued as an access and refresh pair from the app management page, and the access half expires after twelve hours. The refresh half is redeemed through tooling.tokens.rotate, not through oauth.v2.access — a different endpoint from the one that rotates workspace tokens.
Paste the access token into a CI secret and you get one working day out of it. Store the refresh token and rotate at the start of every run and it keeps working indefinitely, provided you write both returned values back, because the new refresh token is single use too.
The second half of this note is the part that matters more. When the configuration token is dead, every check in your audit that reads the manifest cannot run — and a check that cannot run must be reported as not assessed, never as clean. The script below establishes the token's state, then splits the audit's own check list into what was actually assessed and what was silently skipped.
The problem in plain words
Nothing about this failure touches the running app. The bot keeps posting, events keep arriving, customers notice nothing. What breaks is the tooling around the app: manifest sync, drift detection, the audit that reads settings.socket_mode_enabled to decide whether Socket Mode is even the transport. All of it goes quiet, and quiet reads as fine.
The twelve-hour life is the trap because it is just long enough. A developer generates a token, wires the pipeline, watches three builds pass, and ships. The token dies overnight. The first failure looks like a flake, the second like an infrastructure problem, and the manual regeneration that fixes it also resets the clock, so the pattern takes several days to become visible as a pattern.
The rotation endpoint being different from the workspace one compounds it. Teams that already built a refresh loop for rotating bot tokens reach for oauth.v2.access, get an error that does not obviously mean "wrong endpoint", and conclude that configuration tokens cannot be refreshed programmatically. They can; the call is tooling.tokens.rotate and it takes the refresh token.
And underneath all of it sits the reporting problem. An audit that runs twenty checks, six of which need the manifest, will happily print fourteen results and a summary line saying no problems found. That summary is false, and it is false in the direction that makes people stop looking.
Why it happens
This is not the app's token and not the app's clock. A rotating workspace token lives twelve hours too, but it is a different credential, refreshed at a different endpoint, held by a different process. Conflating them sends someone to fix the app's refresh loop when the app was never involved.
Store the refresh half, derive the access half. The access token is a twelve-hour artefact; the refresh token is the durable secret. A pipeline that holds the access token is holding the wrong string, and no amount of care about expiry makes it the right one.
Both halves come back, and both must be written. tooling.tokens.rotate returns a new access token and a new refresh token, and the old refresh token is spent. Persisting only the access token leaves the next run holding a refresh token that has already been redeemed, which fails identically to expiry and is usually diagnosed as expiry.
The error tells you which of four things went wrong. token_expired and invalid_auth mean the pair needs rotating; missing_scope means the token was created without app_configurations:read; app_not_found or invalid_app_id mean the token belongs to a different app account. Those are four different repairs and it is worth naming which one applies.
A blind check is not a passing check. This is the honest-reporting rule that the whole note is built around. When the credential that powers a branch of the audit is dead, that branch is unknown, and the summary has to say how many checks were skipped. Otherwise the audit's most useful property — that a clean run means something — is gone.
The fix, as a flow
The finding is not that a credential died. It is what the death does to the report: six checks that could not run are not six checks that passed, so the script splits its own list into assessed and deferred and refuses to summarise the second as clean.
How to fix it
Ask the manifest API one question
A single read of apps.manifest.export for the app id you care about. You are not looking at the manifest yet; you are looking at whether the credential works, and the error name distinguishes expiry from a missing scope from a token issued against a different app account.
Check which half of the pair the environment holds
If only the access token is present, the pipeline has a one-day credential and will fail tomorrow regardless of what today's call returned. If only the refresh token is present, that is the correct shape: the access token should be derived per run and never stored at all.
Rotate at the start of the run, not on failure
Call tooling.tokens.rotate with the refresh token as the first step of the job, before anything can half-succeed. Rotating in an error handler means the first call of every run is a call you expect to fail, which makes real failures indistinguishable from routine ones.
Write both returned values back
The rotate call returns a new access token and a new refresh token, and the refresh token you sent is spent. Persist both into the secret store in one operation. A pipeline that saves only the access token works exactly once more and then fails in a way that looks like expiry.
Mark the manifest checks not assessed
Take the audit's own check list, tagged with which entries need the manifest, and split it. Print the deferred names individually rather than as a count, and make sure the summary line distinguishes zero problems from zero problems observed. This is the step that keeps the rest of the audit trustworthy.
Split read from write across two tokens
Give the audit a token scoped app_configurations:read and give only the deployment pipeline one that can write. They rotate independently, and a leaked audit credential cannot rewrite your app's manifest.
How to check it worked
With the refresh token in the secret store and the rotate step at the top of the job, the run should report the manifest as readable and defer nothing.
python3 slack_config_token_state.py --app-id A0123456789
# readable the configuration token works; 0 check(s) deferredThe full code
One GET, and then the part that is really the point. manifest_access turns the answer into one of five states with a repair attached to each, stored_pair says which half of the credential pair the environment is holding, and coverage takes the audit's own list of checks and splits it into what was assessed and what has to be reported as unknown. That last function is why the script exists.
"""Report the state of a Slack app configuration token, and what its death hides.
Read only. One GET of apps.manifest.export, purely to establish whether the
credential works. The point of the script is the second half: when that call
fails, every check in the audit that depends on the manifest has to be reported
as not assessed rather than quietly as clean.
"""
import argparse
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("slack_config_token_state")
API = "https://slack.com/api/"
# The audit's own checks, tagged with whether they need the manifest. Replace
# this with your real list; the shape is what matters.
DEFAULT_CHECKS = [
("bot token identity", False),
("granted scope list", False),
("bot channel membership", False),
("request URL configured", True),
("socket mode setting", True),
("token rotation enabled", True),
("scopes declared in the manifest", True),
]
def manifest_access(body):
"""Name the state of the configuration token. Pure. -> (state, detail)"""
if body.get("ok") is True:
return ("readable",
"the configuration token works and the manifest came back.")
error = body.get("error") or "<no error field>"
if error in ("token_expired", "expired_token"):
return ("expired",
"token_expired: configuration tokens live twelve hours. Rotate "
"the pair with tooling.tokens.rotate rather than regenerating by "
"hand every morning.")
if error == "invalid_auth":
return ("rejected",
"invalid_auth: the stored value is not a usable configuration "
"token. It is often the access half of a pair that has already "
"expired, or a workspace token in the wrong slot.")
if error == "missing_scope":
return ("missing-app-configurations-read",
"missing_scope: the configuration token was created without "
"app_configurations:read, so it cannot read a manifest at all.")
if error in ("app_not_found", "invalid_app_id"):
return ("wrong-app-account",
"%s: the token is valid but was issued under a different app "
"account, or the app id does not belong to it." % error)
if error == "ratelimited":
return ("unknown", "ratelimited: no conclusion is available this run.")
return ("unknown",
"error=%s, which is not one of the answers this check knows. Treat "
"the manifest as unavailable." % error)
def stored_pair(present):
"""Which half of the credential pair the environment holds. Pure.
`present` is {"access": bool, "refresh": bool}. -> (state, detail)
"""
access, refresh = bool(present.get("access")), bool(present.get("refresh"))
if not access and not refresh:
return ("no-credential",
"neither half of the pair is in the environment, so no manifest "
"check can run at all.")
if access and not refresh:
return ("access-only",
"only the access token is stored. That is the half that dies in "
"twelve hours, so this pipeline has a one-day credential no "
"matter what today's call returned.")
if refresh and not access:
return ("refresh-only",
"only the refresh token is stored, which is the right shape: "
"rotate at the start of the run and let the access token exist "
"for the length of the job and no longer.")
return ("both-stored",
"both halves are stored. Fine, as long as the rotate step writes "
"both returned values back; writing only one breaks the next run.")
def coverage(access_state, checks):
"""Split the audit's checks into assessed and deferred. Pure.
`checks` is [(name, needs_manifest), ...]. Returns (assessed, deferred).
A check that could not run is not a check that passed.
"""
blind = access_state != "readable"
assessed, deferred = [], []
for name, needs_manifest in checks:
target = deferred if (blind and needs_manifest) else assessed
target.append(name)
return (assessed, deferred)
def export_manifest(session, token, app_id):
"""One GET. We want the error, not the manifest."""
r = session.get(API + "apps.manifest.export",
headers={"Authorization": "Bearer " + token},
params={"app_id": app_id}, timeout=30)
try:
return r.json()
except ValueError:
return {"ok": False, "error": "unparseable_body"}
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--app-id", required=True, help="the app id to export, A...")
ap.add_argument("--access-env", default="SLACK_CONFIG_ACCESS_TOKEN")
ap.add_argument("--refresh-env", default="SLACK_CONFIG_REFRESH_TOKEN")
args = ap.parse_args()
access = os.environ.get(args.access_env)
present = {"access": bool(access),
"refresh": bool(os.environ.get(args.refresh_env))}
pair_state, pair_detail = stored_pair(present)
if not access:
state, detail = ("unknown",
"%s is not set, so the manifest was never asked for."
% args.access_env)
else:
state, detail = manifest_access(
export_manifest(requests.Session(), access, args.app_id))
assessed, deferred = coverage(state, DEFAULT_CHECKS)
log.info("%-32s %s", state, detail)
log.info(" credential: %s -- %s", pair_state, pair_detail)
log.info(" %d check(s) assessed, %d deferred", len(assessed), len(deferred))
for name in deferred:
log.warning(" not assessed: %s", name)
if state == "expired" or pair_state == "access-only":
log.warning(" repair: store the refresh token in CI, not the access token, "
"and call tooling.tokens.rotate with it at the start of every run")
log.warning(" repair: persist BOTH values the rotate call returns; the new "
"refresh token is single use too, so writing one of them breaks "
"the next run")
if state == "missing-app-configurations-read":
log.warning(" repair: reissue the configuration token with "
"app_configurations:read for audit use, and keep "
"app_configurations:write to the pipeline that deploys manifests")
if state == "wrong-app-account":
log.warning(" repair: configuration tokens are issued per app account. "
"Reissue from the account that owns %s", args.app_id)
if deferred:
log.warning(" note: those checks did not run. Do not read their silence as "
"a pass in whatever consumes this report.")
return 0 if state == "readable" and pair_state != "access-only" else 1
if __name__ == "__main__":
sys.exit(main())
/**
* Report the state of a Slack app configuration token, and what its death hides.
*
* Read only. One GET of apps.manifest.export, purely to establish whether the
* credential works. The point of the script is the second half: when that call
* fails, every check in the audit that depends on the manifest has to be
* reported as not assessed rather than quietly as clean.
*/
const API = 'https://slack.com/api/';
// The audit's own checks, tagged with whether they need the manifest. Replace
// this with your real list; the shape is what matters.
export const DEFAULT_CHECKS = [
['bot token identity', false],
['granted scope list', false],
['bot channel membership', false],
['request URL configured', true],
['socket mode setting', true],
['token rotation enabled', true],
['scopes declared in the manifest', true],
];
/**
* Name the state of the configuration token. Pure.
*/
export function manifestAccess(body) {
if (body?.ok === true) {
return ['readable', 'the configuration token works and the manifest came back.'];
}
const error = body?.error ?? '<no error field>';
if (error === 'token_expired' || error === 'expired_token') {
return ['expired',
'token_expired: configuration tokens live twelve hours. Rotate the pair with ' +
'tooling.tokens.rotate rather than regenerating by hand every morning.'];
}
if (error === 'invalid_auth') {
return ['rejected',
'invalid_auth: the stored value is not a usable configuration token. It is ' +
'often the access half of a pair that has already expired, or a workspace ' +
'token in the wrong slot.'];
}
if (error === 'missing_scope') {
return ['missing-app-configurations-read',
'missing_scope: the configuration token was created without ' +
'app_configurations:read, so it cannot read a manifest at all.'];
}
if (error === 'app_not_found' || error === 'invalid_app_id') {
return ['wrong-app-account',
`${error}: the token is valid but was issued under a different app account, ` +
'or the app id does not belong to it.'];
}
if (error === 'ratelimited') {
return ['unknown', 'ratelimited: no conclusion is available this run.'];
}
return ['unknown',
`error=${error}, which is not one of the answers this check knows. Treat the ` +
'manifest as unavailable.'];
}
/**
* Which half of the credential pair the environment holds. Pure.
* `present` is { access: bool, refresh: bool }.
*/
export function storedPair(present) {
const access = Boolean(present?.access);
const refresh = Boolean(present?.refresh);
if (!access && !refresh) {
return ['no-credential',
'neither half of the pair is in the environment, so no manifest check can ' +
'run at all.'];
}
if (access && !refresh) {
return ['access-only',
'only the access token is stored. That is the half that dies in twelve ' +
'hours, so this pipeline has a one-day credential no matter what the call ' +
'in this run returned.'];
}
if (refresh && !access) {
return ['refresh-only',
'only the refresh token is stored, which is the right shape: rotate at the ' +
'start of the run and let the access token exist for the length of the job ' +
'and no longer.'];
}
return ['both-stored',
'both halves are stored. Fine, as long as the rotate step writes both returned ' +
'values back; writing only one breaks the next run.'];
}
/**
* Split the audit's checks into assessed and deferred. Pure.
* `checks` is [[name, needsManifest], ...]. A check that could not run is not a
* check that passed.
*/
export function coverage(accessState, checks) {
const blind = accessState !== 'readable';
const assessed = [];
const deferred = [];
for (const [name, needsManifest] of checks) {
(blind && needsManifest ? deferred : assessed).push(name);
}
return [assessed, deferred];
}
async function exportManifest(token, appId) {
const url = `${API}apps.manifest.export?app_id=${encodeURIComponent(appId)}`;
const res = await fetch(url, { headers: { Authorization: `Bearer ${token}` } });
try {
return await res.json();
} catch {
return { ok: false, error: 'unparseable_body' };
}
}
function arg(args, name, fallback = null) {
const i = args.indexOf(name);
return i === -1 ? fallback : args[i + 1];
}
async function main() {
const args = process.argv.slice(2);
const appId = arg(args, '--app-id');
if (!appId) {
console.error('usage: --app-id A0123456789 [--access-env NAME] [--refresh-env NAME]');
process.exitCode = 2;
return;
}
const accessEnv = arg(args, '--access-env', 'SLACK_CONFIG_ACCESS_TOKEN');
const refreshEnv = arg(args, '--refresh-env', 'SLACK_CONFIG_REFRESH_TOKEN');
const access = process.env[accessEnv];
const [pairState, pairDetail] = storedPair({
access: Boolean(access), refresh: Boolean(process.env[refreshEnv]),
});
const [state, detail] = access
? manifestAccess(await exportManifest(access, appId))
: ['unknown', `${accessEnv} is not set, so the manifest was never asked for.`];
const [assessed, deferred] = coverage(state, DEFAULT_CHECKS);
console.log(`${state.padEnd(32)} ${detail}`);
console.log(` credential: ${pairState} -- ${pairDetail}`);
console.log(` ${assessed.length} check(s) assessed, ${deferred.length} deferred`);
for (const name of deferred) console.warn(` not assessed: ${name}`);
if (state === 'expired' || pairState === 'access-only') {
console.warn(' repair: store the refresh token in CI, not the access token, and ' +
'call tooling.tokens.rotate with it at the start of every run');
console.warn(' repair: persist BOTH values the rotate call returns; the new ' +
'refresh token is single use too, so writing one of them breaks the ' +
'next run');
}
if (state === 'missing-app-configurations-read') {
console.warn(' repair: reissue the configuration token with ' +
'app_configurations:read for audit use, and keep ' +
'app_configurations:write to the pipeline that deploys manifests');
}
if (state === 'wrong-app-account') {
console.warn(' repair: configuration tokens are issued per app account. Reissue ' +
`from the account that owns ${appId}`);
}
if (deferred.length) {
console.warn(' note: those checks did not run. Do not read their silence as a ' +
'pass in whatever consumes this report.');
}
process.exitCode = (state === 'readable' && pairState !== 'access-only') ? 0 : 1;
}
// Only run when invoked directly, so importing this module in the tests does not
// require an app id on the command line.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The suite spends most of its assertions on coverage, because that is the function whose failure mode is a false clean bill of health rather than a wrong label. The others pin the four repairs apart — expiry, missing scope, wrong app account, wrong credential entirely — and pin the credential shape that guarantees tomorrow's failure even when today's call succeeded.
from slack_config_token_state import coverage, manifest_access, stored_pair
CHECKS = [("bot identity", False), ("socket mode setting", True),
("request URL configured", True)]
def test_a_readable_manifest_assesses_everything():
assessed, deferred = coverage("readable", CHECKS)
assert deferred == []
assert len(assessed) == 3
def test_a_dead_token_defers_the_manifest_checks_rather_than_passing_them():
assessed, deferred = coverage("expired", CHECKS)
assert assessed == ["bot identity"]
assert deferred == ["socket mode setting", "request URL configured"]
def test_every_unreadable_state_defers_the_same_way():
for state in ("rejected", "missing-app-configurations-read",
"wrong-app-account", "unknown"):
assert coverage(state, CHECKS)[1] == ["socket mode setting",
"request URL configured"]
def test_expiry_names_the_rotate_endpoint_not_the_oauth_one():
state, detail = manifest_access({"ok": False, "error": "token_expired"})
assert state == "expired"
assert "tooling.tokens.rotate" in detail
def test_a_missing_scope_is_not_an_expiry():
state, _ = manifest_access({"ok": False, "error": "missing_scope"})
assert state == "missing-app-configurations-read"
def test_the_token_belonging_to_another_app_account_is_its_own_state():
assert manifest_access({"ok": False,
"error": "app_not_found"})[0] == "wrong-app-account"
assert manifest_access({"ok": False,
"error": "invalid_app_id"})[0] == "wrong-app-account"
def test_an_unrecognised_error_leaves_the_manifest_unavailable():
assert manifest_access({"ok": False, "error": "fatal_error"})[0] == "unknown"
assert manifest_access({"ok": True})[0] == "readable"
def test_holding_only_the_access_token_is_a_one_day_credential():
state, detail = stored_pair({"access": True, "refresh": False})
assert state == "access-only"
assert "one-day credential" in detail
def test_holding_only_the_refresh_token_is_the_right_shape():
state, detail = stored_pair({"access": False, "refresh": True})
assert state == "refresh-only"
assert "right shape" in detail
def test_holding_both_is_conditional_on_writing_both_back():
state, detail = stored_pair({"access": True, "refresh": True})
assert state == "both-stored"
assert "writing only one" in detail
def test_holding_neither_is_reported_before_any_call_is_made():
assert stored_pair({})[0] == "no-credential"
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { coverage, manifestAccess, storedPair } from './slack-config-token-state.mjs';
const CHECKS = [['bot identity', false], ['socket mode setting', true],
['request URL configured', true]];
test('a readable manifest assesses everything', () => {
const [assessed, deferred] = coverage('readable', CHECKS);
assert.deepEqual(deferred, []);
assert.equal(assessed.length, 3);
});
test('a dead token defers the manifest checks rather than passing them', () => {
const [assessed, deferred] = coverage('expired', CHECKS);
assert.deepEqual(assessed, ['bot identity']);
assert.deepEqual(deferred, ['socket mode setting', 'request URL configured']);
});
test('every unreadable state defers the same way', () => {
for (const state of ['rejected', 'missing-app-configurations-read',
'wrong-app-account', 'unknown']) {
assert.deepEqual(coverage(state, CHECKS)[1],
['socket mode setting', 'request URL configured']);
}
});
test('expiry names the rotate endpoint not the oauth one', () => {
const [state, detail] = manifestAccess({ ok: false, error: 'token_expired' });
assert.equal(state, 'expired');
assert.match(detail, /tooling\.tokens\.rotate/);
});
test('a missing scope is not an expiry', () => {
assert.equal(manifestAccess({ ok: false, error: 'missing_scope' })[0],
'missing-app-configurations-read');
});
test('the token belonging to another app account is its own state', () => {
assert.equal(manifestAccess({ ok: false, error: 'app_not_found' })[0],
'wrong-app-account');
assert.equal(manifestAccess({ ok: false, error: 'invalid_app_id' })[0],
'wrong-app-account');
});
test('an unrecognised error leaves the manifest unavailable', () => {
assert.equal(manifestAccess({ ok: false, error: 'fatal_error' })[0], 'unknown');
assert.equal(manifestAccess({ ok: true })[0], 'readable');
});
test('holding only the access token is a one-day credential', () => {
const [state, detail] = storedPair({ access: true, refresh: false });
assert.equal(state, 'access-only');
assert.match(detail, /one-day credential/);
});
test('holding only the refresh token is the right shape', () => {
const [state, detail] = storedPair({ access: false, refresh: true });
assert.equal(state, 'refresh-only');
assert.match(detail, /right shape/);
});
test('holding both is conditional on writing both back', () => {
const [state, detail] = storedPair({ access: true, refresh: true });
assert.equal(state, 'both-stored');
assert.match(detail, /writing only one/);
});
test('holding neither is reported before any call is made', () => {
assert.equal(storedPair({})[0], 'no-credential');
});
FAQ
How is this different from the twelve-hour token rotation on my bot token?
Different credential, different endpoint, different holder. A rotating workspace token belongs to the running app and refreshes through oauth.v2.access; a configuration token belongs to whoever manages the app and refreshes through tooling.tokens.rotate. The lifetimes happen to match, which is precisely why the two get confused and why someone ends up debugging the app's refresh loop for a failure the app was never part of.
Can I make the configuration token last longer?
No. Twelve hours is fixed and there is no long-lived variant, which is deliberate: the token can rewrite your app's configuration, so it is meant to be minted per session rather than parked in a secret store. Build the rotate step instead of looking for a setting.
What exactly does tooling.tokens.rotate give back?
A new access token and a new refresh token. The refresh token you sent is spent, so the next run needs the one that came back. This is the detail that breaks pipelines: a job that writes only the access token to the secret store works one more time and then fails with an error indistinguishable from ordinary expiry.
Why should the audit care that the manifest was unreadable?
Because the alternative is a report that says nothing is wrong when in fact six checks never ran. Every finding that depends on the manifest, such as whether Socket Mode is enabled or whether token rotation was switched on, is unknown rather than clean, and a summary that does not distinguish those two states destroys the value of a clean run.
Do I need a write-scoped configuration token for an audit?
No, and you should not have one. Issue the audit a token with app_configurations:read only, and keep app_configurations:write to the pipeline that actually deploys manifests. They rotate independently, so a leak of the read credential cannot rewrite the app, and the split costs nothing beyond one extra secret.
Related field notes
- the app's own twelve-hour clock
- when a refresh token is spent twice
- reading the manifest to explain a socket
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- apps.manifest.export method reference — Slack Docs
- tooling.tokens.rotate method reference — Slack Docs
- Token types — Slack Docs
- Using token rotation — Slack Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.