Skip to content

Diagnostic Twilio

signature validation rejects Twilio with 403 behind a proxy

It works on the laptop with the tunnel running. It works in staging. Then it goes behind the load balancer and every Twilio request comes back 403 from your own middleware — the code you added to keep other people out. Twilio logs it as 11200, exactly like a 404 or a crash, and the Debugger row looks the same as every other retrieval failure on the account.

Read-only key Python and Node.js Tests included
Man in white button up shirt smiling
Photo by LumenSoft Technologies on Unsplash
The short answer

Sweep GET https://monitor.twilio.com/v1/Alerts?LogLevel=error&StartDate=YYYY-MM-DD for error_code == 11200, then fetch a sample of those alerts individually with GET https://monitor.twilio.com/v1/Alerts/{Sid} and read response_body. A body that mentions a signature, or is a bare 403, separates this from an ordinary 5xx.

The cause is almost always the URL. X-Twilio-Signature is an HMAC-SHA1 over the full URL Twilio called — scheme, host, port and query string — plus the sorted POST parameters. A TLS-terminating proxy hands your app http:// and often an internal hostname, so the app signs a different string and rejects a legitimate request. The alert's request_url is the exact string the validator has to be given.

The problem in plain words

Every other webhook failure in this section is something breaking. This one is something working: the validator is doing precisely what it was written to do, on an input that is wrong by one field. The request really did come from Twilio, the signature header really is valid, and your app really cannot verify it, because it is verifying against a URL that no longer matches the one Twilio signed.

What makes it expensive is where it appears. It cannot be reproduced locally, because locally there is no proxy in front and the app sees the same URL Twilio called. It survives every test that mocks the request. It shows up the hour the service moves behind a load balancer, a CDN, an ingress controller or an API gateway, and it presents as 11200 — the same code as a 404 and the same code as a crash. Teams spend a day on routing and deploys before anyone opens a single alert body and reads the word signature.

Twilio signsHMAC over the fullURLRequest senthttps, public hostProxy forwardshttp, internalhostApp rebuildsURLdifferent string,different HMAC403 to Twiliologged as 11200
Nothing along the way is broken. The proxy is doing its job and the validator is doing its job, on two different URLs.

Why it happens

The signature covers the URL, and the proxy changes the URL. Twilio computes the HMAC over the exact URL it requested. Your app reconstructs that URL from what the proxy gave it: http instead of https, an internal service name instead of the public host, sometimes port 8080 instead of no port. Change one character and the HMAC changes completely, which is the property that makes signatures useful and this bug invisible.

11200 is a bucket, not a diagnosis. Twilio treats anything outside 2xx as a retrieval failure. A 403 from your validator, a 404 from a moved route and a 500 from a crash all arrive with the same error_code, the same shape and the same request_url field. The only thing that separates them is what came back, and that is not in the list.

The response body is only on the single-alert fetch. Every row of GET /v1/Alerts has response_body and response_headers blank. They are populated when you fetch one alert by SID and nowhere else. So the sweep is cheap and tells you nothing about the cause, and the cause costs one request per alert.

The obvious normalisation is the wrong move here. Grouping webhook alerts by lowercase host and path is the right instinct for finding which endpoint is failing, and it is exactly wrong for fixing this one. The scheme, the port and the query string are all inside the HMAC. The string you need is request_url untouched, and a script that tidies it before printing it has thrown away the answer.

The fix, as a flow

The script fetches alerts one at a time to read the response body, because the list blanks that field and the error code alone cannot tell a signature rejection from a 404 or a crash.

GET Alerts, then GETAlerts/{Sid}error_code plus response_bodyBody names a signaturethe URL, rebuild itBare 403 pagea WAF in front, other ownerStack tracethe app threw, not validationEmpty bodyread the status and the logs
The error code is the same for all four. Only the response body, which lives on the single alert fetch, tells them apart.

How to fix it

Sweep 11200 over a bounded window and group by host

GET https://monitor.twilio.com/v1/Alerts?LogLevel=error&StartDate=YYYY-MM-DD&PageSize=1000, following meta.next_page_url. Keep alerts where error_code is 11200 — read it as an integer, because the Monitor API returns error codes as strings. Group by hostname so a burst on one service is one line rather than four hundred.

Fetch a few of them by SID to see what came back

GET https://monitor.twilio.com/v1/Alerts/{Sid} is the only place response_body and response_headers exist. Two or three per host is enough to characterise the failure, and each one is a request, so cap it. If the sample comes back with empty bodies, the endpoint returned nothing at all and this is not your validator.

Read the body before you believe the error code

A body naming X-Twilio-Signature, saying invalid signature, or coming from a framework's request-validator middleware is this problem. A bare 403 Forbidden page from nginx or a WAF is a different one with a different owner: something in front of the app is refusing Twilio before your code runs. A stack trace is an application error, and none of the three share a repair.

Take request_url exactly as logged and change nothing

That field is the string the HMAC was computed over. Do not lowercase it, do not drop the query string, do not add or remove a trailing slash. Feed it to the validator alongside the POST parameters and the header, and you will reproduce the failure locally in one line — which is the fastest confirmation available and needs no traffic.

Rebuild the public URL in the app, then re-run the sweep

Reconstruct the URL from X-Forwarded-Proto and X-Forwarded-Host, or hardcode the public base URL and append the request path, before calling RequestValidator.validate(). Trust those headers only from your own proxy. Re-run this script over the window afterwards: the signature bucket should empty while the other buckets stay exactly as they were.

How to check it worked

Re-run over a window that starts after the deploy. The signature bucket goes to zero; anything left is a different failure and is reported as one.

python3 twilio_signature_403_audit.py --days 1
# 0 endpoint(s) rejecting Twilio's signature, 1 with other 11200 failures

The full code

One cheap sweep, then a capped number of single-alert fetches, because that is the only place the response body lives. The classifier is pure and takes the fetched alert as an argument rather than doing the fetching, so every branch — a signature rejection, a proxy 403, a stack trace, an empty body — is exercised offline. signed_url() is one line and has a test anyway, because the temptation to normalise that string is the whole trap.

Get this script on GitHub Follow @allanninal Python and Node.js, with tests. Read only, it never writes. One of 117 Twilio fixes, free and open source.
twilio_signature_403_audit.py
"""Separate signature-validation rejections from ordinary 11200 webhook failures.

Read only. GET requests and nothing else: give this an API Key with read access
rather than the account auth token. The repair is printed, never performed,
because this script holds a credential to an account that can send messages and
spend money.
"""
import argparse
import datetime as dt
import logging
import os
import sys
from urllib.parse import urlsplit

import requests

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("twilio_signature_403_audit")

MONITOR = "https://monitor.twilio.com/v1"

RETRIEVAL_FAILURE = 11200

# Alerts are retained 30 days. A longer window is not more history, it is the
# same history under a label that makes the report look more thorough.
MAX_DAYS = 30

# Phrases that mean the endpoint refused Twilio's own request. Framework
# middleware and hand-rolled checks both tend to name the header or the word.
SIGNATURE_MARKERS = (
    "x-twilio-signature",
    "invalid signature",
    "signature validation",
    "signature mismatch",
    "signature verification",
    "twilio signature",
    "requestvalidator",
)

# A refusal with no mention of a signature. Something in front of the app said
# no before the app ran, which is a different owner and a different repair.
FORBIDDEN_MARKERS = (
    "403 forbidden",
    "forbidden",
    "access denied",
    "not authorized",
    "unauthorized",
)

# The application ran and blew up. Nothing to do with request validation.
APP_ERROR_MARKERS = (
    "traceback (most recent call last)",
    "internal server error",
    "stack trace",
    "exception",
)


def code_of(alert):
    """Read error_code off an alert as an integer, or None.

    The Monitor API hands this back as a string while the Messages list hands
    back a number for the same concept. A comparison written against one and
    pointed at the other matches nothing and reports a healthy account.
    """
    raw = alert.get("error_code")
    if raw is None or raw == "":
        return None
    try:
        return int(raw)
    except (TypeError, ValueError):
        return None


def host_of(url):
    """Lowercase hostname, for grouping only.

    Grouping is the one place it is safe to throw information away. The repair
    needs the whole URL, which is why signed_url() exists separately.
    """
    if not url:
        return ""
    parts = urlsplit(str(url).strip())
    return (parts.hostname or str(url).strip()).lower()


def signed_url(alert):
    """The exact string the signature was computed over.

    Returned untouched on purpose. Scheme, host, port and query string are all
    inside the HMAC, so lowercasing the host or dropping the parameters gives
    you a string that will never validate and looks like it should.
    """
    return str(alert.get("request_url") or "").strip()


def header_text(headers):
    """Flatten response_headers into searchable text.

    Twilio returns this field in more than one shape depending on the product
    that logged the alert: a mapping, a list of lines, or one blob. Handling
    only the shape you happened to see first is how a working check quietly
    stops matching six months later.
    """
    if headers is None:
        return ""
    if isinstance(headers, dict):
        return "\n".join("%s: %s" % (k, v) for k, v in headers.items())
    if isinstance(headers, (list, tuple)):
        return "\n".join(str(h) for h in headers)
    return str(headers)


def found(text, needles):
    """Which of these phrases appear, case-insensitively. Pure and boring."""
    low = str(text or "").lower()
    return [n for n in needles if n in low]


def classify(alert, detail):
    """Decide what one 11200 alert actually was.

    Pure: `detail` is the single-alert fetch, GET /v1/Alerts/{Sid}, or None when
    it was not fetched. The list response blanks response_body and
    response_headers on every row, so without that second request there is no
    honest verdict to give and this says so rather than guessing.

    Returns (state, detail_text).
    """
    if code_of(alert) != RETRIEVAL_FAILURE:
        return ("not-11200", "some other error code; this script only reads 11200")

    if detail is None:
        return ("unfetched",
                "the alerts list blanks response_body, so what the endpoint "
                "returned is unknown until this alert is fetched by SID")

    body = str(detail.get("response_body") or "")
    text = body + "\n" + header_text(detail.get("response_headers"))

    hits = found(text, SIGNATURE_MARKERS)
    if hits:
        return ("signature",
                "the endpoint rejected Twilio's own request (%s). The signature "
                "covers the full URL Twilio called, and behind a TLS-terminating "
                "proxy the app rebuilds a different one." % ", ".join(hits))

    if found(text, FORBIDDEN_MARKERS):
        return ("forbidden",
                "refused with nothing about signatures: a WAF, an ingress rule "
                "or auth middleware in front of the app said no before your code "
                "ran. Different owner, different repair.")

    if found(text, APP_ERROR_MARKERS):
        return ("app-error",
                "the handler ran and threw. This is an application failure "
                "wearing the same error code, not a validation problem.")

    if not body.strip():
        return ("no-body",
                "non-2xx with an empty body. Nothing here points at validation; "
                "look at the status the endpoint returned and at its own logs.")

    return ("other",
            "non-2xx with a body that names neither a signature nor an error. "
            "Read the first line of it before deciding.")


def group(alerts):
    """Bucket 11200 alerts by hostname. Pure.

    date_generated is ISO 8601 in UTC on every alert, so a string comparison
    orders them and finding the ends needs no date parsing.
    """
    out = {}
    for a in alerts:
        if code_of(a) != RETRIEVAL_FAILURE:
            continue
        host = host_of(a.get("request_url"))
        row = out.setdefault(host, {"alerts": 0, "sids": [], "urls": [],
                                    "methods": set(), "first": None, "last": None})
        row["alerts"] += 1
        if len(row["sids"]) < 5:
            row["sids"].append(a.get("sid"))
            row["urls"].append(signed_url(a))
        method = str(a.get("request_method") or "").upper()
        if method:
            row["methods"].add(method)
        when = a.get("date_generated") or ""
        if when:
            row["first"] = when if row["first"] is None else min(row["first"], when)
            row["last"] = when if row["last"] is None else max(row["last"], when)
    return out


def get(session, url, **params):
    r = session.get(url, params=params, timeout=30)
    if r.status_code in (401, 403):
        raise SystemExit("%d from Twilio: check TWILIO_ACCOUNT_SID and that the "
                         "API key belongs to that account with read access"
                         % r.status_code)
    r.raise_for_status()
    return r.json()


def list_alerts(session, since, limit, log_level="error"):
    """Page the Monitor alerts. next_page_url is absolute on this API."""
    url = MONITOR + "/Alerts"
    params = {"LogLevel": log_level, "StartDate": since, "PageSize": 1000}
    out = []
    while url and len(out) < limit:
        page = get(session, url, **params)
        out.extend(page.get("alerts", []))
        url = (page.get("meta") or {}).get("next_page_url")
        params = {}
    return out[:limit]


def fetch_alert(session, sid):
    """One alert by SID: the only place response_body is populated."""
    return get(session, "%s/Alerts/%s" % (MONITOR, sid))


def main():
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("--days", type=int, default=1,
                    help="how far back to read alerts (Twilio keeps 30 days)")
    ap.add_argument("--max-alerts", type=int, default=10000,
                    help="stop paging alerts after this many")
    ap.add_argument("--sample", type=int, default=2,
                    help="alerts to fetch individually per host for the response "
                         "body (0 to skip; each one is a request)")
    args = ap.parse_args()

    account = os.environ.get("TWILIO_ACCOUNT_SID")
    key = os.environ.get("TWILIO_API_KEY")
    secret = os.environ.get("TWILIO_API_SECRET")
    if not (account and key and secret):
        log.error("set TWILIO_ACCOUNT_SID, TWILIO_API_KEY and TWILIO_API_SECRET "
                  "(an API Key with read access, not the auth token)")
        return 2

    days = args.days
    if days > MAX_DAYS:
        log.warning("alerts are retained %d days; reading %d instead of %d",
                    MAX_DAYS, MAX_DAYS, days)
        days = MAX_DAYS

    session = requests.Session()
    session.auth = (key, secret)

    since = (dt.date.today() - dt.timedelta(days=days)).isoformat()
    alerts = list_alerts(session, since, args.max_alerts)
    rows = group(alerts)
    log.info("%d alert(s) since %s, %d host(s) with 11200", len(alerts), since,
             len(rows))

    signature = other = 0
    for host, row in sorted(rows.items()):
        states = []
        for sid in row["sids"][:max(0, args.sample)]:
            states.append(classify({"error_code": RETRIEVAL_FAILURE},
                                   fetch_alert(session, sid)))
        if not states:
            states = [classify({"error_code": RETRIEVAL_FAILURE}, None)]

        state, detail = states[0]
        for s, d in states:
            if s == "signature":
                state, detail = s, d
                break

        log.warning("%-10s %s  %d x 11200 (%s)  %s", state, host, row["alerts"],
                    ", ".join(sorted(row["methods"])) or "?", detail)
        log.warning("  first %s, last %s", row["first"], row["last"])
        if state == "signature":
            signature += 1
            log.warning("  validate against this exact string, unmodified: %s",
                        row["urls"][0])
            log.warning("  repair: rebuild the URL from X-Forwarded-Proto and "
                        "X-Forwarded-Host (or hardcode the public base URL) "
                        "before calling RequestValidator.validate, and trust "
                        "those headers only from your own proxy")
        else:
            other += 1

    log.info("%d endpoint(s) rejecting Twilio's signature, %d with other 11200 "
             "failures", signature, other)
    return 1 if signature else 0


if __name__ == "__main__":
    sys.exit(main())
twilio-signature-403-audit.mjs
/**
 * Separate signature-validation rejections from ordinary 11200 webhook failures.
 *
 * Read only. GET requests and nothing else: give this an API Key with read
 * access rather than the account auth token. The repair is printed, never
 * performed.
 */
const MONITOR = 'https://monitor.twilio.com/v1';

const RETRIEVAL_FAILURE = 11200;

// Alerts are retained 30 days. A longer window is the same history mislabelled.
const MAX_DAYS = 30;

// Phrases that mean the endpoint refused Twilio's own request.
const SIGNATURE_MARKERS = [
  'x-twilio-signature', 'invalid signature', 'signature validation',
  'signature mismatch', 'signature verification', 'twilio signature',
  'requestvalidator',
];

// A refusal with no mention of a signature: something in front of the app.
const FORBIDDEN_MARKERS = [
  '403 forbidden', 'forbidden', 'access denied', 'not authorized', 'unauthorized',
];

// The application ran and blew up. Nothing to do with request validation.
const APP_ERROR_MARKERS = [
  'traceback (most recent call last)', 'internal server error', 'stack trace',
  'exception',
];

/**
 * Read error_code off an alert as a number, or null. The Monitor API returns it
 * as a string while the Messages list returns a number.
 */
export function codeOf(alert) {
  const raw = alert.error_code;
  if (raw === null || raw === undefined || raw === '') return null;
  const n = Number(raw);
  return Number.isFinite(n) ? n : null;
}

/** Lowercase hostname, for grouping only. */
export function hostOf(url) {
  if (!url) return '';
  const raw = String(url).trim();
  try {
    return new URL(raw).hostname.toLowerCase();
  } catch {
    return raw.toLowerCase();
  }
}

/**
 * The exact string the signature was computed over, returned untouched. Scheme,
 * host, port and query string are all inside the HMAC.
 */
export function signedUrl(alert) {
  return String(alert.request_url ?? '').trim();
}

/**
 * Flatten response_headers into searchable text. Twilio returns this field as a
 * mapping, a list of lines or one blob depending on the product.
 */
export function headerText(headers) {
  if (headers === null || headers === undefined) return '';
  if (Array.isArray(headers)) return headers.map(String).join('\n');
  if (typeof headers === 'object') {
    return Object.entries(headers).map(([k, v]) => `${k}: ${v}`).join('\n');
  }
  return String(headers);
}

/** Which of these phrases appear, case-insensitively. */
export function found(text, needles) {
  const low = String(text ?? '').toLowerCase();
  return needles.filter((n) => low.includes(n));
}

/**
 * Decide what one 11200 alert actually was. Pure: `detail` is the single-alert
 * fetch, or null when it was not fetched. Returns [state, detailText].
 */
export function classify(alert, detail) {
  if (codeOf(alert) !== RETRIEVAL_FAILURE) {
    return ['not-11200', 'some other error code; this script only reads 11200'];
  }
  if (detail === null || detail === undefined) {
    return ['unfetched',
      'the alerts list blanks response_body, so what the endpoint returned is ' +
      'unknown until this alert is fetched by SID'];
  }

  const body = String(detail.response_body ?? '');
  const text = `${body}\n${headerText(detail.response_headers)}`;

  const hits = found(text, SIGNATURE_MARKERS);
  if (hits.length) {
    return ['signature',
      `the endpoint rejected Twilio's own request (${hits.join(', ')}). The ` +
      'signature covers the full URL Twilio called, and behind a ' +
      'TLS-terminating proxy the app rebuilds a different one.'];
  }

  if (found(text, FORBIDDEN_MARKERS).length) {
    return ['forbidden',
      'refused with nothing about signatures: a WAF, an ingress rule or auth ' +
      'middleware in front of the app said no before your code ran. Different ' +
      'owner, different repair.'];
  }

  if (found(text, APP_ERROR_MARKERS).length) {
    return ['app-error',
      'the handler ran and threw. This is an application failure wearing the ' +
      'same error code, not a validation problem.'];
  }

  if (!body.trim()) {
    return ['no-body',
      'non-2xx with an empty body. Nothing here points at validation; look at ' +
      'the status the endpoint returned and at its own logs.'];
  }

  return ['other',
    'non-2xx with a body that names neither a signature nor an error. Read the ' +
    'first line of it before deciding.'];
}

/** Bucket 11200 alerts by hostname. Pure. */
export function group(alerts) {
  const out = new Map();
  for (const a of alerts) {
    if (codeOf(a) !== RETRIEVAL_FAILURE) continue;
    const host = hostOf(a.request_url);
    if (!out.has(host)) {
      out.set(host, {
        alerts: 0, sids: [], urls: [], methods: new Set(), first: null, last: null,
      });
    }
    const row = out.get(host);
    row.alerts += 1;
    if (row.sids.length < 5) {
      row.sids.push(a.sid);
      row.urls.push(signedUrl(a));
    }
    const method = String(a.request_method ?? '').toUpperCase();
    if (method) row.methods.add(method);
    const when = a.date_generated ?? '';
    if (when) {
      row.first = row.first === null || when < row.first ? when : row.first;
      row.last = row.last === null || when > row.last ? when : row.last;
    }
  }
  return out;
}

function authHeader(key, secret) {
  return `Basic ${Buffer.from(`${key}:${secret}`).toString('base64')}`;
}

async function get(auth, url, params = {}) {
  const u = new URL(url);
  for (const [k, v] of Object.entries(params)) u.searchParams.set(k, v);
  const res = await fetch(u, { headers: { Authorization: auth } });
  if (res.status === 401 || res.status === 403) {
    throw new Error(`${res.status} from Twilio: check TWILIO_ACCOUNT_SID and ` +
                    'that the API key belongs to that account with read access');
  }
  if (!res.ok) throw new Error(`${res.status} from ${u.pathname}`);
  return res.json();
}

export async function listAlerts(auth, since, limit = 10000, logLevel = 'error') {
  let url = `${MONITOR}/Alerts`;
  let params = { LogLevel: logLevel, StartDate: since, PageSize: 1000 };
  const out = [];
  while (url && out.length < limit) {
    const page = await get(auth, url, params);
    out.push(...(page.alerts ?? []));
    url = page.meta?.next_page_url ?? null;
    params = {};
  }
  return out.slice(0, limit);
}

/** One alert by SID: the only place response_body is populated. */
export async function fetchAlert(auth, sid) {
  return get(auth, `${MONITOR}/Alerts/${sid}`);
}

async function main() {
  const account = process.env.TWILIO_ACCOUNT_SID;
  const key = process.env.TWILIO_API_KEY;
  const secret = process.env.TWILIO_API_SECRET;
  if (!account || !key || !secret) {
    console.error('set TWILIO_ACCOUNT_SID, TWILIO_API_KEY and TWILIO_API_SECRET ' +
                  '(an API Key with read access, not the auth token)');
    process.exitCode = 2;
    return;
  }
  const auth = authHeader(key, secret);

  let days = Number(process.argv.includes('--days')
    ? process.argv[process.argv.indexOf('--days') + 1] : 1) || 1;
  if (days > MAX_DAYS) {
    console.warn(`alerts are retained ${MAX_DAYS} days; reading ${MAX_DAYS} instead`);
    days = MAX_DAYS;
  }
  const sample = Number(process.argv.includes('--sample')
    ? process.argv[process.argv.indexOf('--sample') + 1] : 2);
  const since = new Date(Date.now() - days * 86400000).toISOString().slice(0, 10);

  const alerts = await listAlerts(auth, since);
  const rows = group(alerts);
  console.log(`${alerts.length} alert(s) since ${since}, ${rows.size} host(s) with 11200`);

  let signature = 0;
  let other = 0;
  for (const [host, row] of [...rows.entries()].sort()) {
    const states = [];
    for (const sid of row.sids.slice(0, Math.max(0, sample))) {
      states.push(classify({ error_code: RETRIEVAL_FAILURE }, await fetchAlert(auth, sid)));
    }
    if (!states.length) states.push(classify({ error_code: RETRIEVAL_FAILURE }, null));

    let [state, detail] = states[0];
    for (const [s, d] of states) {
      if (s === 'signature') { state = s; detail = d; break; }
    }

    const methods = [...row.methods].sort().join(', ') || '?';
    console.warn(`${state.padEnd(10)} ${host}  ${row.alerts} x 11200 (${methods})  ${detail}`);
    console.warn(`  first ${row.first}, last ${row.last}`);
    if (state === 'signature') {
      signature += 1;
      console.warn(`  validate against this exact string, unmodified: ${row.urls[0]}`);
      console.warn('  repair: rebuild the URL from X-Forwarded-Proto and ' +
                   'X-Forwarded-Host (or hardcode the public base URL) before ' +
                   'calling RequestValidator.validate, and trust those headers ' +
                   'only from your own proxy');
    } else {
      other += 1;
    }
  }

  console.log(`${signature} endpoint(s) rejecting Twilio's signature, ${other} ` +
              'with other 11200 failures');
  process.exitCode = signature ? 1 : 0;
}

// Only run when invoked directly. The test file imports this module, and without
// the guard main() would run there too, fail on the missing credentials and set a
// non-zero exit code that fails the whole test file even as every test passes.
if (import.meta.url === `file://${process.argv[1]}`) {
  main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}

Add a test

The tests pin the four bodies that arrive at a 403 by four different routes, because filing any one of them under the wrong heading sends somebody to the wrong team. They also pin the thing that is easy to get right and easier to undo: signed_url() returns request_url with its scheme, port and query string intact, while host_of() throws all three away — one is for the repair, the other is only for grouping.

test_twilio_signature_403_audit.py
from twilio_signature_403_audit import (classify, code_of, found, group,
                                          header_text, host_of, signed_url)


def alert(sid, url, code="11200", when="2026-04-02T10:00:00Z", method="POST"):
    return {"sid": sid, "request_url": url, "error_code": code,
            "date_generated": when, "request_method": method, "log_level": "error"}


def detail(body="", headers=None):
    return {"response_body": body, "response_headers": headers}


def test_code_of_reads_the_string_the_monitor_api_returns():
    assert code_of({"error_code": "11200"}) == 11200
    assert code_of({"error_code": 11200}) == 11200
    assert code_of({"error_code": ""}) is None
    assert code_of({}) is None


def test_signed_url_keeps_everything_the_hmac_covers():
    # The scheme, the port and the query string are all inside the signature.
    # Tidying any of them produces a string that can never validate.
    a = alert("NO1", "https://hooks.example.com:8443/twilio/voice?From=%2B15551112222")
    assert signed_url(a) == \
        "https://hooks.example.com:8443/twilio/voice?From=%2B15551112222"


def test_host_of_throws_away_what_signed_url_keeps():
    assert host_of("https://Hooks.Example.com:8443/twilio/voice?a=b") == \
        "hooks.example.com"
    assert host_of(None) == ""


def test_a_body_naming_the_header_is_a_signature_rejection():
    state, why = classify(alert("NO1", "https://a.example.com/voice"),
                          detail("Invalid signature for X-Twilio-Signature"))
    assert state == "signature"
    assert "URL" in why


def test_a_bare_403_page_is_not_blamed_on_the_validator():
    # nginx or a WAF refused before the app ran. Same error code, other owner.
    state, why = classify(alert("NO1", "https://a.example.com/voice"),
                          detail("<html><head><title>403 Forbidden</title></head></html>"))
    assert state == "forbidden"
    assert "WAF" in why


def test_a_stack_trace_is_an_application_error():
    state, _ = classify(alert("NO1", "https://a.example.com/voice"),
                        detail("Traceback (most recent call last):\n  File ..."))
    assert state == "app-error"


def test_an_empty_body_is_reported_as_unknown_rather_than_guessed():
    state, _ = classify(alert("NO1", "https://a.example.com/voice"), detail(""))
    assert state == "no-body"


def test_without_the_single_alert_fetch_there_is_no_verdict():
    state, why = classify(alert("NO1", "https://a.example.com/voice"), None)
    assert state == "unfetched"
    assert "response_body" in why


def test_markers_are_also_read_from_the_response_headers():
    state, _ = classify(
        alert("NO1", "https://a.example.com/voice"),
        detail("", {"X-Rejected-By": "RequestValidator", "Server": "gunicorn"}))
    assert state == "signature"


def test_header_text_flattens_every_shape_the_field_arrives_in():
    assert header_text({"A": "1"}) == "A: 1"
    assert header_text(["A: 1", "B: 2"]) == "A: 1\nB: 2"
    assert header_text("A: 1") == "A: 1"
    assert header_text(None) == ""


def test_group_buckets_by_host_and_records_the_ends():
    rows = group([
        alert("NO1", "https://a.example.com/voice?x=1", when="2026-04-02T10:00:00Z"),
        alert("NO2", "https://a.example.com/sms?x=2", when="2026-04-01T09:00:00Z"),
        alert("NO3", "https://b.example.com/voice", code="11205"),
    ])
    assert set(rows) == {"a.example.com"}
    assert rows["a.example.com"]["alerts"] == 2
    assert rows["a.example.com"]["first"] == "2026-04-01T09:00:00Z"
    assert rows["a.example.com"]["urls"][0].endswith("?x=1")


def test_found_is_case_insensitive():
    assert found("INVALID SIGNATURE", ["invalid signature"]) == ["invalid signature"]
    assert found(None, ["invalid signature"]) == []
twilio-signature-403-audit.test.mjs
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
  classify, codeOf, found, group, headerText, hostOf, signedUrl,
} from './twilio-signature-403-audit.mjs';

const alert = (sid, url, code = '11200', when = '2026-04-02T10:00:00Z') => ({
  sid, request_url: url, error_code: code, date_generated: when,
  request_method: 'POST', log_level: 'error',
});

const detail = (body = '', headers = null) => ({
  response_body: body, response_headers: headers,
});

test('codeOf reads the string the Monitor API returns', () => {
  assert.equal(codeOf({ error_code: '11200' }), 11200);
  assert.equal(codeOf({ error_code: 11200 }), 11200);
  assert.equal(codeOf({ error_code: '' }), null);
  assert.equal(codeOf({}), null);
});

test('signedUrl keeps everything the HMAC covers', () => {
  const a = alert('NO1', 'https://hooks.example.com:8443/twilio/voice?From=%2B15551112222');
  assert.equal(signedUrl(a),
    'https://hooks.example.com:8443/twilio/voice?From=%2B15551112222');
});

test('hostOf throws away what signedUrl keeps', () => {
  assert.equal(hostOf('https://Hooks.Example.com:8443/twilio/voice?a=b'),
    'hooks.example.com');
  assert.equal(hostOf(null), '');
});

test('a body naming the header is a signature rejection', () => {
  const [state, why] = classify(alert('NO1', 'https://a.example.com/voice'),
    detail('Invalid signature for X-Twilio-Signature'));
  assert.equal(state, 'signature');
  assert.match(why, /URL/);
});

test('a bare 403 page is not blamed on the validator', () => {
  const [state, why] = classify(alert('NO1', 'https://a.example.com/voice'),
    detail('<html><head><title>403 Forbidden</title></head></html>'));
  assert.equal(state, 'forbidden');
  assert.match(why, /WAF/);
});

test('a stack trace is an application error', () => {
  const [state] = classify(alert('NO1', 'https://a.example.com/voice'),
    detail('Traceback (most recent call last):\n  File ...'));
  assert.equal(state, 'app-error');
});

test('an empty body is reported as unknown rather than guessed', () => {
  const [state] = classify(alert('NO1', 'https://a.example.com/voice'), detail(''));
  assert.equal(state, 'no-body');
});

test('without the single-alert fetch there is no verdict', () => {
  const [state, why] = classify(alert('NO1', 'https://a.example.com/voice'), null);
  assert.equal(state, 'unfetched');
  assert.match(why, /response_body/);
});

test('markers are also read from the response headers', () => {
  const [state] = classify(alert('NO1', 'https://a.example.com/voice'),
    detail('', { 'X-Rejected-By': 'RequestValidator', Server: 'gunicorn' }));
  assert.equal(state, 'signature');
});

test('headerText flattens every shape the field arrives in', () => {
  assert.equal(headerText({ A: '1' }), 'A: 1');
  assert.equal(headerText(['A: 1', 'B: 2']), 'A: 1\nB: 2');
  assert.equal(headerText('A: 1'), 'A: 1');
  assert.equal(headerText(null), '');
});

test('group buckets by host and records the ends', () => {
  const rows = group([
    alert('NO1', 'https://a.example.com/voice?x=1', '11200', '2026-04-02T10:00:00Z'),
    alert('NO2', 'https://a.example.com/sms?x=2', '11200', '2026-04-01T09:00:00Z'),
    alert('NO3', 'https://b.example.com/voice', '11205'),
  ]);
  assert.deepEqual([...rows.keys()], ['a.example.com']);
  assert.equal(rows.get('a.example.com').alerts, 2);
  assert.equal(rows.get('a.example.com').first, '2026-04-01T09:00:00Z');
  assert.ok(rows.get('a.example.com').urls[0].endsWith('?x=1'));
});

test('found is case-insensitive', () => {
  assert.deepEqual(found('INVALID SIGNATURE', ['invalid signature']),
    ['invalid signature']);
  assert.deepEqual(found(null, ['invalid signature']), []);
});

FAQ

Why does it work locally and fail behind the load balancer?

Because locally your app sees the same URL Twilio called, and behind a proxy it does not. The proxy terminates TLS and forwards http:// to an internal hostname, so the app reconstructs a URL that differs from the signed one in scheme, host or port. The HMAC is over the whole string, so one character is enough.

Is there a Twilio error code specific to this?

No. It arrives as 11200, the generic HTTP retrieval failure, exactly like a 404 or a 500. That is why the script fetches alerts individually: the error code cannot tell these apart, and the response body can.

Why can't the alerts list show me the response body?

Because response_body, response_headers, request_headers and request_variables are populated only on the single-alert fetch, GET /v1/Alerts/{Sid}. Every row of the list has them blank. The list is the cheap sweep; reading a body is one request per alert, which is why the sample size is an argument.

Should I just trust X-Forwarded-Proto and X-Forwarded-Host?

Only from your own proxy. Those headers are client-supplied unless something you control overwrites them, so an app that trusts them from anywhere lets a caller choose the URL its own signature is checked against. Terminate the trust at the proxy, or hardcode the public base URL instead.

Can I confirm the fix without waiting for real traffic?

Yes, and that is the fastest route. Take request_url from a failing alert exactly as logged, feed it to the validator with the POST parameters and the signature header, and you reproduce the rejection offline. Then rerun with the rebuilt URL and watch it pass.

Related field notes

Sources

Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.

Stuck on a tricky one?

If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.