Skip to content

Diagnostic Slack

Slack refresh tokens are single use: a replay kills the pair

Rotation was working. Then one afternoon every call returns token_expired, the refresh call answers invalid_refresh_token, and the only recovery is to send a customer through the install flow again. Nothing was deployed. What changed is that the service went from one replica to two, and both of them woke up on the same cron minute.

Read-only token Python and Node.js Tests included
Camera studio set up
Photo by Alexander Dummer on Unsplash
The short answer

A Slack refresh token is single use. Redeeming it returns a new refresh token and begins retiring the old one, and Slack keeps at most two active tokens per installation. Two workers refreshing at the same moment, or one worker retrying after a request that actually succeeded but whose response was lost, both spend the same token twice.

The Web API cannot tell you this happened. All it will say is token_expired, which is what an un-refreshed token says too. The evidence lives in the refresh attempts your own app recorded: two attempts seconds apart from different workers, or more than two successful refreshes inside one twelve-hour window. The script below reads that ledger, calls auth.test once per installation to prove the app is still installed, and names the refresh that burned the pair.

The problem in plain words

Token rotation is opt-in and cannot be switched back off. Once it is on, an installation carries two secrets rather than one: an access token prefixed xoxe.xoxb- that expires after exactly 43200 seconds, and a refresh token prefixed xoxe-1- that exists to mint the next pair. The app is expected to redeem the second before the first expires.

The trap is that redeeming is not idempotent. Each redemption issues a fresh pair and starts revoking the one it replaced, so the operation is only safe if exactly one actor performs it at a time. Nothing in the deployment enforces that. A refresh scheduled on a fixed cron runs on every replica. A refresh triggered lazily by "the token expires in under an hour" fires on whichever requests arrive first, which on a busy service is several at once. And an HTTP client that retries on timeout will happily redeem a token whose response was lost in transit rather than never sent.

What makes it hard to see is that the failure is delayed and total. The first replay usually succeeds; both workers get a valid pair and both write one, and whichever wrote last wins. The loser's pair is still live, because Slack allows two. The breakage arrives on the third redemption inside the window, when the two-token limit retires something still in use, or on the next cycle when the stored refresh token turns out to be the one that was superseded. By then the logs that would explain it have rolled.

Cron fires onbothone installationBoth redeem atoncethe token issingle useTwo pairsissuedlast write winsThirdredemptionoldest retiredEvery callrefusedonly OAuthrecovers
The replay usually succeeds. Both workers get a valid pair, both write one, and the breakage arrives a cycle later when the stored half turns out to be the superseded one.

Why it happens

Single use is the whole mechanism. Rotation exists so that a leaked token has a short life. That guarantee is only worth anything if redeeming a refresh token invalidates it, so Slack does exactly that. A design that treats the refresh token as a long-lived credential to be reused is not slightly wrong; it is using the feature backwards.

Two active tokens, not unlimited. Slack keeps a small number of tokens live per installation so that a redemption whose response was lost does not immediately lock you out. Refresh more than that inside one window and the oldest is retired — which, if the oldest is the one a still-running worker holds, looks exactly like a random logout.

The store write has to be atomic with the read. Reading the refresh token, calling out, and writing the result back is a read-modify-write across a network call. Without a per-installation lock held for the whole sequence, two workers interleave and the later write can be the older pair.

A retry is a second redemption. A gateway timeout does not mean the request failed; it means you do not know. Retrying it spends the token a second time. Refresh calls should not be inside a generic retrying HTTP client, or the client should retry only on a connection error that provably never reached Slack.

Nothing distinguishes this from plain expiry at the API. auth.test answers token_expired either way. The only thing that separates "we never built the refresh loop" from "we built it and two copies of it fought" is the record of attempts, which is why this note's detection is a ledger read and not a probe.

The fix, as a flow

The script reads timestamps before it reads Slack. Two redemptions inside the lock window is the finding, and the live call exists only to say whether it has landed yet: expired means it has, and ok means the same bug is still waiting.

Ledger plus one auth.testtimestamps and worker idsTwo workers, one minuteno lock anywhereSame worker, after a timeouta retry spent it twiceThree in twelve hoursover the active limitExpired, ledger cleanthe loop never ranOnce per windowserialised and healthy
A missing lock and a retried timeout produce the identical error and need opposite repairs, so the split has to happen before anyone is sent to fix something.

How to fix it

Write down the refresh attempts, if you are not already

The script consumes a JSON array of what your app recorded: for each attempt, the installation it was for, an ISO-8601 timestamp, the worker or pod that made it, and whether it came back ok, timed out, or errored. If you have never logged this, that is the first repair — four fields at the point of redemption, and the next incident explains itself.

Look for two attempts inside the lock window

Two redemptions for one installation seconds apart from different workers is a concurrency bug and needs a lock. Two seconds apart from the same worker, where the first did not return cleanly, is a retry spending the token twice and needs the retry removed. They have the same symptom and opposite fixes, so the script keeps them apart.

Count successful refreshes per twelve-hour window

A rotated access token lives 43200 seconds, so a healthy app refreshes once or twice per window. More than two successes inside one sliding window means older tokens are being retired by the active-token limit while something may still be holding them.

Ask the live token what state it is actually in

One auth.test per installation, which needs no scopes. token_revoked means the app was uninstalled and the ledger is a red herring. token_expired alongside a ledger finding is the replay. ok: true alongside a ledger finding is the worst of the three: the bug is live and simply has not landed yet.

Serialise the refresh and write both halves together

Take a per-installation lock — a row lock in the database that holds the installation is enough — re-read the stored pair inside it, skip the redemption if another worker already wrote something fresher, and persist the new access token and the new refresh token in one transaction. Schedule at roughly half the lifetime rather than at expiry.

Accept that a dead refresh token is dead

There is no recovery call. If both halves of the pair are gone, the installation has to go back through OAuth. Say so in the report rather than scheduling retries against a credential that will never answer again.

How to check it worked

After the lock is in place, re-run over a fresh ledger. Every installation should refresh once or twice per window, always from a worker that held the lock, and no two attempts should sit inside the lock window.

python3 slack_refresh_ledger_audit.py --ledger refreshes.json
# 6 installation(s) checked, 0 with a refresh that can burn the pair

The full code

One GET per installation, and it is the smaller half of the job — the finding is computed from timestamps before Slack is contacted at all. Two pure functions: ledger_verdict reads one installation's attempt history and names the shape of the misuse, and install_state combines that with the live auth.test answer, because a burned pair and an uninstalled app produce the same silence.

Get this script on GitHub Follow @allanninal Python and Node.js, with tests. Read only, it never writes. One of 20 Slack fixes, free and open source.
slack_refresh_ledger_audit.py
"""Find the refresh that burned a Slack rotating token, and the worker that did it.

Read only. One GET per installation and nothing else; the finding itself is
computed from timestamps the app already wrote down. The repair is a lock and a
transaction, and it is printed for a human to implement.
"""
import argparse
import json
import logging
import os
import sys
from datetime import datetime, timedelta, timezone

import requests

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("slack_refresh_ledger_audit")

API = "https://slack.com/api/"

WINDOW_HOURS = 12    # a rotated access token lives 43200 seconds
ACTIVE_LIMIT = 2     # Slack keeps at most this many live tokens per installation
LOCK_SECONDS = 60    # two redemptions closer than this were not serialised

BURNED = ("concurrent-refresh", "retry-after-timeout", "over-active-limit")


def _at(value):
    """ISO-8601 to an aware datetime. Pure, and tolerant of a trailing Z."""
    text = str(value).replace("Z", "+00:00")
    stamp = datetime.fromisoformat(text)
    return stamp if stamp.tzinfo else stamp.replace(tzinfo=timezone.utc)


def ledger_verdict(events, window_hours=WINDOW_HOURS, lock_seconds=LOCK_SECONDS):
    """Classify one installation's recorded refresh attempts. Pure.

    `events` is what the app logged at the point of redemption:
    [{"at": "2026-08-30T09:00:00Z", "worker": "web-2", "outcome": "ok"}, ...]
    in any order. Returns (state, detail).
    """
    rows = sorted(({"at": _at(e.get("at")),
                    "worker": str(e.get("worker") or "?"),
                    "outcome": str(e.get("outcome") or "ok")} for e in events),
                  key=lambda r: r["at"])
    if not rows:
        return ("no-refresh-recorded",
                "no redemption was ever logged for this installation. Either "
                "rotation is off, or the refresh loop does not exist yet, which "
                "is a different problem from spending the token twice.")

    for older, newer in zip(rows, rows[1:]):
        gap = (newer["at"] - older["at"]).total_seconds()
        if gap > lock_seconds:
            continue
        if newer["worker"] != older["worker"]:
            return ("concurrent-refresh",
                    "%s and %s both redeemed within %.0fs at %s. The token is "
                    "single use, so one of those two pairs was already dying "
                    "when it was written."
                    % (older["worker"], newer["worker"], gap,
                       newer["at"].isoformat()))
        if older["outcome"] != "ok":
            return ("retry-after-timeout",
                    "%s redeemed, saw outcome=%s, and redeemed again %.0fs "
                    "later. A timeout is not a failure: the first call may have "
                    "spent the token and lost the answer."
                    % (older["worker"], older["outcome"], gap))

    ok_rows = [r for r in rows if r["outcome"] == "ok"]
    span = timedelta(hours=window_hours)
    for i, first in enumerate(ok_rows):
        inside = [r for r in ok_rows[i:] if r["at"] - first["at"] < span]
        if len(inside) > ACTIVE_LIMIT:
            return ("over-active-limit",
                    "%d successful redemptions in the %dh window starting %s. "
                    "Slack keeps %d tokens live, so the oldest were retired "
                    "while something may still have been holding them."
                    % (len(inside), window_hours, first["at"].isoformat(),
                       ACTIVE_LIMIT))

    return ("serialised",
            "%d redemption(s), none inside the %ds lock window and at most %d "
            "per %dh window" % (len(rows), lock_seconds, ACTIVE_LIMIT, window_hours))


def install_state(identity, ledger_state):
    """Combine the live auth.test answer with the ledger classification. Pure.

    The Web API cannot distinguish a replayed refresh token from one that was
    never refreshed, so neither half is conclusive alone.
    """
    burned = ledger_state in BURNED
    if identity.get("ok") is True:
        if burned:
            return ("at-risk",
                    "the token works right now and the ledger shows a redemption "
                    "that can spend the pair twice. This is the cheap moment to "
                    "fix it: after the next collision the only repair is OAuth.")
        return ("healthy", "auth.test answers ok and the refresh history is clean")

    error = identity.get("error") or "<no error field>"
    if error == "token_revoked":
        return ("uninstalled",
                "token_revoked. The app was removed from the workspace, which is "
                "not a rotation problem at all; tombstone the row instead.")
    if error in ("token_expired", "invalid_auth"):
        if burned:
            return ("refresh-token-burned",
                    "error=%s with a redemption that spent the pair twice. The "
                    "stored refresh token is the superseded one and will not be "
                    "redeemable. Only a fresh install recovers this." % error)
        return ("expired-not-refreshed",
                "error=%s and the ledger shows no misuse. This looks like a "
                "refresh that never ran rather than one that ran twice." % error)
    return ("inconclusive",
            "error=%s, which is neither expiry nor revocation. Resolve that "
            "before reading anything into the ledger." % error)


def auth_test(session, token):
    r = session.get(API + "auth.test", headers={"Authorization": "Bearer " + token},
                    timeout=30)
    try:
        return r.json()
    except ValueError:
        return {"ok": False, "error": "unparseable_body"}


def load_ledger(path):
    """Group the recorded attempts by installation."""
    rows = json.loads(open(path, encoding="utf-8").read())
    grouped = {}
    for row in rows:
        grouped.setdefault(str(row.get("install") or "<unkeyed>"), []).append(row)
    return grouped


def main():
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("--ledger", required=True,
                    help="JSON array of refresh attempts: install, at, worker, outcome")
    ap.add_argument("--token-env", default="SLACK_BOT_TOKEN",
                    help="environment variable holding the token for the single-install case")
    ap.add_argument("--tokens",
                    help="JSON object mapping install id to the env var holding its token")
    args = ap.parse_args()

    grouped = load_ledger(args.ledger)
    token_envs = json.loads(open(args.tokens, encoding="utf-8").read()) if args.tokens else {}
    s = requests.Session()

    bad = 0
    for install, events in sorted(grouped.items()):
        state, detail = ledger_verdict(events)
        env_name = token_envs.get(install, args.token_env)
        token = os.environ.get(env_name)
        if not token:
            log.warning("%-24s %-14s ledger says %s; %s is unset so the live "
                        "state is unknown", "no-token", install, state, env_name)
            bad += 1
            continue

        combined, live_detail = install_state(auth_test(s, token), state)
        line = "%-24s %-14s %s" % (combined, install, live_detail)
        if combined == "healthy":
            log.info(line)
            continue
        bad += 1
        log.warning(line)
        log.warning("  ledger: %s -- %s", state, detail)
        if state in BURNED:
            log.warning("  repair: hold a per-installation lock across read, redeem "
                        "and write; persist both new values in one transaction")
            log.warning("  repair: do not retry a redemption on timeout, and do not "
                        "schedule one on a fixed cron across replicas")

    log.info("%d installation(s) checked, %d with a refresh that can burn the pair",
             len(grouped), bad)
    return 1 if bad else 0


if __name__ == "__main__":
    sys.exit(main())
slack-refresh-ledger-audit.mjs
/**
 * Find the refresh that burned a Slack rotating token, and the worker that did it.
 *
 * Read only. One GET per installation and nothing else; the finding itself is
 * computed from timestamps the app already wrote down. The repair is a lock and
 * a transaction, and it is printed for a human to implement.
 */
import { readFile } from 'node:fs/promises';

const API = 'https://slack.com/api/';

const WINDOW_HOURS = 12;   // a rotated access token lives 43200 seconds
const ACTIVE_LIMIT = 2;    // Slack keeps at most this many live tokens per install
const LOCK_SECONDS = 60;   // two redemptions closer than this were not serialised

const BURNED = new Set(['concurrent-refresh', 'retry-after-timeout', 'over-active-limit']);

/**
 * Classify one installation's recorded refresh attempts. Pure.
 * `events` is [{ at, worker, outcome }, ...] in any order.
 */
export function ledgerVerdict(events, windowHours = WINDOW_HOURS, lockSeconds = LOCK_SECONDS) {
  const rows = events
    .map((e) => ({
      at: new Date(e.at),
      worker: String(e.worker ?? '?'),
      outcome: String(e.outcome ?? 'ok'),
    }))
    .sort((a, b) => a.at - b.at);

  if (rows.length === 0) {
    return ['no-refresh-recorded',
      'no redemption was ever logged for this installation. Either rotation is ' +
      'off, or the refresh loop does not exist yet, which is a different problem ' +
      'from spending the token twice.'];
  }

  for (let i = 1; i < rows.length; i += 1) {
    const older = rows[i - 1];
    const newer = rows[i];
    const gap = (newer.at - older.at) / 1000;
    if (gap > lockSeconds) continue;
    if (newer.worker !== older.worker) {
      return ['concurrent-refresh',
        `${older.worker} and ${newer.worker} both redeemed within ${gap.toFixed(0)}s ` +
        `at ${newer.at.toISOString()}. The token is single use, so one of those two ` +
        'pairs was already dying when it was written.'];
    }
    if (older.outcome !== 'ok') {
      return ['retry-after-timeout',
        `${older.worker} redeemed, saw outcome=${older.outcome}, and redeemed again ` +
        `${gap.toFixed(0)}s later. A timeout is not a failure: the first call may ` +
        'have spent the token and lost the answer.'];
    }
  }

  const okRows = rows.filter((r) => r.outcome === 'ok');
  const span = windowHours * 3600 * 1000;
  for (let i = 0; i < okRows.length; i += 1) {
    const inside = okRows.slice(i).filter((r) => r.at - okRows[i].at < span);
    if (inside.length > ACTIVE_LIMIT) {
      return ['over-active-limit',
        `${inside.length} successful redemptions in the ${windowHours}h window ` +
        `starting ${okRows[i].at.toISOString()}. Slack keeps ${ACTIVE_LIMIT} tokens ` +
        'live, so the oldest were retired while something may still have been ' +
        'holding them.'];
    }
  }

  return ['serialised',
    `${rows.length} redemption(s), none inside the ${lockSeconds}s lock window and ` +
    `at most ${ACTIVE_LIMIT} per ${windowHours}h window`];
}

/**
 * Combine the live auth.test answer with the ledger classification. Pure.
 * The Web API cannot distinguish a replayed refresh token from one that was
 * never refreshed, so neither half is conclusive alone.
 */
export function installState(identity, ledgerState) {
  const burned = BURNED.has(ledgerState);
  if (identity?.ok === true) {
    if (burned) {
      return ['at-risk',
        'the token works right now and the ledger shows a redemption that can ' +
        'spend the pair twice. This is the cheap moment to fix it: after the next ' +
        'collision the only repair is OAuth.'];
    }
    return ['healthy', 'auth.test answers ok and the refresh history is clean'];
  }

  const error = identity?.error ?? '<no error field>';
  if (error === 'token_revoked') {
    return ['uninstalled',
      'token_revoked. The app was removed from the workspace, which is not a ' +
      'rotation problem at all; tombstone the row instead.'];
  }
  if (error === 'token_expired' || error === 'invalid_auth') {
    if (burned) {
      return ['refresh-token-burned',
        `error=${error} with a redemption that spent the pair twice. The stored ` +
        'refresh token is the superseded one and will not be redeemable. Only a ' +
        'fresh install recovers this.'];
    }
    return ['expired-not-refreshed',
      `error=${error} and the ledger shows no misuse. This looks like a refresh ` +
      'that never ran rather than one that ran twice.'];
  }
  return ['inconclusive',
    `error=${error}, which is neither expiry nor revocation. Resolve that before ` +
    'reading anything into the ledger.'];
}

async function authTest(token) {
  const res = await fetch(API + 'auth.test', {
    headers: { Authorization: `Bearer ${token}` },
  });
  try {
    return await res.json();
  } catch {
    return { ok: false, error: 'unparseable_body' };
  }
}

function arg(args, name, fallback = null) {
  const i = args.indexOf(name);
  return i === -1 ? fallback : args[i + 1];
}

async function main() {
  const args = process.argv.slice(2);
  const ledgerPath = arg(args, '--ledger');
  if (!ledgerPath) {
    console.error('usage: --ledger refreshes.json [--tokens tokens.json] ' +
                  '[--token-env SLACK_BOT_TOKEN]');
    process.exitCode = 2;
    return;
  }
  const tokenEnv = arg(args, '--token-env', 'SLACK_BOT_TOKEN');
  const tokensPath = arg(args, '--tokens');
  const tokenEnvs = tokensPath ? JSON.parse(await readFile(tokensPath, 'utf8')) : {};

  const grouped = new Map();
  for (const row of JSON.parse(await readFile(ledgerPath, 'utf8'))) {
    const key = String(row.install ?? '<unkeyed>');
    if (!grouped.has(key)) grouped.set(key, []);
    grouped.get(key).push(row);
  }

  let bad = 0;
  for (const [install, events] of [...grouped.entries()].sort()) {
    const [state, detail] = ledgerVerdict(events);
    const envName = tokenEnvs[install] ?? tokenEnv;
    const token = process.env[envName];
    if (!token) {
      console.warn(`${'no-token'.padEnd(24)} ${install.padEnd(14)} ledger says ` +
                   `${state}; ${envName} is unset so the live state is unknown`);
      bad += 1;
      continue;
    }

    const [combined, liveDetail] = installState(await authTest(token), state);
    const line = `${combined.padEnd(24)} ${install.padEnd(14)} ${liveDetail}`;
    if (combined === 'healthy') {
      console.log(line);
      continue;
    }
    bad += 1;
    console.warn(line);
    console.warn(`  ledger: ${state} -- ${detail}`);
    if (BURNED.has(state)) {
      console.warn('  repair: hold a per-installation lock across read, redeem and ' +
                   'write; persist both new values in one transaction');
      console.warn('  repair: do not retry a redemption on timeout, and do not ' +
                   'schedule one on a fixed cron across replicas');
    }
  }

  console.log(`${grouped.size} installation(s) checked, ${bad} with a refresh that ` +
              'can burn the pair');
  process.exitCode = bad ? 1 : 0;
}

// Only run when invoked directly, so importing this module in the tests does not
// execute main() and fail the file on a missing ledger.
if (import.meta.url === `file://${process.argv[1]}`) {
  main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}

Add a test

The pair worth pinning is the two redemptions sixty seconds apart. From different workers it is a missing lock; from the same worker after a timeout it is a retry that spent the token twice. Identical timestamps, identical symptom at the API, opposite repairs — so the tests assert on which one the classifier picked, not merely that it complained.

test_slack_refresh_ledger_audit.py
from slack_refresh_ledger_audit import install_state, ledger_verdict


def test_two_workers_inside_the_lock_window_is_a_concurrency_finding():
    state, detail = ledger_verdict([
        {"at": "2026-08-30T09:00:00Z", "worker": "web-1", "outcome": "ok"},
        {"at": "2026-08-30T09:00:12Z", "worker": "web-2", "outcome": "ok"},
    ])
    assert state == "concurrent-refresh"
    assert "web-1" in detail and "web-2" in detail


def test_same_worker_retrying_a_timeout_is_a_different_finding():
    state, detail = ledger_verdict([
        {"at": "2026-08-30T09:00:00Z", "worker": "web-1", "outcome": "timeout"},
        {"at": "2026-08-30T09:00:20Z", "worker": "web-1", "outcome": "ok"},
    ])
    assert state == "retry-after-timeout"
    assert "timeout is not a failure" in detail


def test_three_successes_in_one_window_exceed_the_active_token_limit():
    state, _ = ledger_verdict([
        {"at": "2026-08-30T00:00:00Z", "worker": "w", "outcome": "ok"},
        {"at": "2026-08-30T04:00:00Z", "worker": "w", "outcome": "ok"},
        {"at": "2026-08-30T08:00:00Z", "worker": "w", "outcome": "ok"},
    ])
    assert state == "over-active-limit"


def test_two_refreshes_a_window_apart_are_normal():
    state, _ = ledger_verdict([
        {"at": "2026-08-30T00:00:00Z", "worker": "w", "outcome": "ok"},
        {"at": "2026-08-30T06:00:00Z", "worker": "w", "outcome": "ok"},
        {"at": "2026-08-30T18:00:00Z", "worker": "w", "outcome": "ok"},
    ])
    assert state == "serialised"


def test_an_empty_ledger_is_not_reported_as_misuse():
    assert ledger_verdict([])[0] == "no-refresh-recorded"


def test_expired_token_plus_a_burned_ledger_names_the_replay():
    state, detail = install_state({"ok": False, "error": "token_expired"},
                                  "concurrent-refresh")
    assert state == "refresh-token-burned"
    assert "fresh install" in detail


def test_expired_token_with_a_clean_ledger_is_a_missing_loop_not_a_replay():
    state, _ = install_state({"ok": False, "error": "token_expired"}, "serialised")
    assert state == "expired-not-refreshed"


def test_uninstall_is_not_read_as_a_rotation_problem():
    state, _ = install_state({"ok": False, "error": "token_revoked"},
                             "concurrent-refresh")
    assert state == "uninstalled"


def test_a_working_token_with_a_burned_ledger_is_still_a_finding():
    state, detail = install_state({"ok": True, "team_id": "T1"}, "over-active-limit")
    assert state == "at-risk"
    assert "cheap moment" in detail


def test_a_working_token_with_a_clean_ledger_is_reported_as_healthy():
    assert install_state({"ok": True, "team_id": "T1"}, "serialised")[0] == "healthy"
slack-refresh-ledger-audit.test.mjs
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { installState, ledgerVerdict } from './slack-refresh-ledger-audit.mjs';

test('two workers inside the lock window is a concurrency finding', () => {
  const [state, detail] = ledgerVerdict([
    { at: '2026-08-30T09:00:00Z', worker: 'web-1', outcome: 'ok' },
    { at: '2026-08-30T09:00:12Z', worker: 'web-2', outcome: 'ok' },
  ]);
  assert.equal(state, 'concurrent-refresh');
  assert.match(detail, /web-1/);
  assert.match(detail, /web-2/);
});

test('same worker retrying a timeout is a different finding', () => {
  const [state, detail] = ledgerVerdict([
    { at: '2026-08-30T09:00:00Z', worker: 'web-1', outcome: 'timeout' },
    { at: '2026-08-30T09:00:20Z', worker: 'web-1', outcome: 'ok' },
  ]);
  assert.equal(state, 'retry-after-timeout');
  assert.match(detail, /timeout is not a failure/);
});

test('three successes in one window exceed the active token limit', () => {
  const [state] = ledgerVerdict([
    { at: '2026-08-30T00:00:00Z', worker: 'w', outcome: 'ok' },
    { at: '2026-08-30T04:00:00Z', worker: 'w', outcome: 'ok' },
    { at: '2026-08-30T08:00:00Z', worker: 'w', outcome: 'ok' },
  ]);
  assert.equal(state, 'over-active-limit');
});

test('two refreshes a window apart are normal', () => {
  const [state] = ledgerVerdict([
    { at: '2026-08-30T00:00:00Z', worker: 'w', outcome: 'ok' },
    { at: '2026-08-30T06:00:00Z', worker: 'w', outcome: 'ok' },
    { at: '2026-08-30T18:00:00Z', worker: 'w', outcome: 'ok' },
  ]);
  assert.equal(state, 'serialised');
});

test('an empty ledger is not reported as misuse', () => {
  assert.equal(ledgerVerdict([])[0], 'no-refresh-recorded');
});

test('expired token plus a burned ledger names the replay', () => {
  const [state, detail] = installState(
    { ok: false, error: 'token_expired' }, 'concurrent-refresh');
  assert.equal(state, 'refresh-token-burned');
  assert.match(detail, /fresh install/);
});

test('expired token with a clean ledger is a missing loop not a replay', () => {
  const [state] = installState({ ok: false, error: 'token_expired' }, 'serialised');
  assert.equal(state, 'expired-not-refreshed');
});

test('uninstall is not read as a rotation problem', () => {
  const [state] = installState({ ok: false, error: 'token_revoked' }, 'concurrent-refresh');
  assert.equal(state, 'uninstalled');
});

test('a working token with a burned ledger is still a finding', () => {
  const [state, detail] = installState({ ok: true, team_id: 'T1' }, 'over-active-limit');
  assert.equal(state, 'at-risk');
  assert.match(detail, /cheap moment/);
});

test('a working token with a clean ledger is reported as healthy', () => {
  assert.equal(installState({ ok: true, team_id: 'T1' }, 'serialised')[0], 'healthy');
});

FAQ

Can I detect the replay from the Slack API alone?

No, and that is the honest answer rather than a limitation of this script. auth.test returns token_expired for a token that was never refreshed and for one whose pair was spent twice, and there is no read method that reports how many times a refresh token has been redeemed. The distinguishing evidence is the record of your own redemption attempts, which is why the script asks for it.

Is a timeout on the refresh call safe to retry?

Not blindly. A timeout means the answer was lost, not that the request never arrived, so the token may already have been spent. Retry only on an error that provably never reached Slack, such as a DNS failure or a refused connection, and on anything else re-read the store first to see whether a new pair was written.

Why does Slack allow two active tokens instead of one?

So that a redemption whose response was lost does not lock the installation out immediately: the previous token stays usable for a short grace period. It is a safety margin for a single well-behaved refresher, not a budget for several. Three redemptions inside one window will retire something that is still in use.

Can I turn rotation off once I discover the refresh loop is broken?

No. Rotation is a one-way switch on the app configuration, so the only path forward is to implement the refresh loop correctly. If an app adopted a copied manifest with rotation enabled and never noticed, that is worth knowing before the twelve hours are up rather than after.

What recovers an installation whose refresh token is already dead?

Only a fresh OAuth install. There is no method that reissues a pair from a revoked refresh token, and retrying the dead one produces the same error indefinitely. Report it as needing re-authorisation and stop scheduling work against it, rather than burning rate limit on a credential that cannot come back.

Related field notes

Sources

Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.

Stuck on a tricky one?

If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.