Diagnostic Twilio
a Studio Flow whose definition is invalid, so widgets never run
The Flow looks finished. The canvas renders, the widgets are connected, the Console opens it and saves it without complaint. But executions end two widgets in, or skip a branch that is plainly drawn on screen, and valid on the Flow resource is false. The picture is a drawing of a definition, and the drawing does not stop being tidy when the definition stops compiling.
Read GET https://studio.twilio.com/v2/Flows/{FlowSid} and look at valid. When it is false, the diagnosis is already in the same response: errors[] holds one entry per fault, each with a message and the path of the widget that caused it.
warnings[] is the other array and is not the same finding. A warning does not make a definition invalid; it flags something that will behave oddly rather than something that stops it running. Report the two in separate columns, or one real outage arrives buried in eleven notes about naming.
The problem in plain words
An invalid definition fails in the middle of a call. The execution starts, runs the widgets it can, and stops where the definition breaks — so the caller hears the greeting and then silence, or the SMS branch that should have replied never fires. There is nothing wrong with the number, nothing wrong with the webhook, and nothing wrong with the Flow's status: it can be published, live, and taking traffic while valid is false.
The Console is no help because the canvas keeps drawing. Deleting a widget removes the box; it does not remove the transitions in other widgets that named it, and a transition with no destination simply draws no line. Nothing on screen turns red. The only place the fault is stated in words is errors[] on the Flow resource, and that is a field you have to go and ask for.
Why it happens
The canvas and the definition are not the same object. Studio renders a picture from the Flow's JSON, and rendering is forgiving: a widget whose transition points at something that no longer exists is drawn as a widget with one fewer arrow. Compiling is not forgiving. The two disagree, and only one of them is what runs.
Deleting a widget does not delete what pointed at it. This is the most common single cause. Somebody removes a step that is no longer needed, the widgets that transitioned into it keep the reference, and the definition is invalid from that save onwards even though the visible change was a deletion nobody could get wrong.
Liquid is not checked while you type it. A template referencing a variable from a widget that was renamed, or a filter with the wrong number of arguments, is accepted into the definition and reported later as an error against that widget's path. The person who renamed the widget is not the person who wrote the template.
An invalid draft and an invalid published Flow need opposite advice. A draft that does not compile cannot be published at all, so telling anyone to press Publish is wrong: the widget has to be fixed first. A published Flow that is invalid is failing executions right now. The same valid: false means both, which is why the script reads status alongside it rather than reporting one line for every case.
The fix, as a flow
The diagnosis is read from the single flow rather than the list, because errors[] and warnings[] are carried there. status only decides who is affected: a published Flow is failing now, a draft cannot be published.
How to fix it
List the Flows, then fetch each one
GET https://studio.twilio.com/v2/Flows?PageSize=50, following meta.next_page_url, gives you the SIDs. The diagnosis comes from GET https://studio.twilio.com/v2/Flows/{FlowSid}: that is the fetch that carries errors[] and warnings[], and without them all you have is a boolean saying something is wrong somewhere.
Read valid before you read anything else
valid is the finding. status only tells you who is affected: published means executions are hitting the fault now, draft means the last published revision is still serving and the publish is blocked until the widget is fixed.
Take the path, not just the message
Each entry in errors[] carries a message and a path. The message is what broke; the path is where. A report that prints only messages sends somebody hunting through a canvas with forty widgets on it for a transition that names a widget that is not there.
Keep warnings in their own column
warnings[] can be populated on a Flow whose valid is true. Those are worth reading and they are not this outage. Fold them into the error list and the report grows a long tail that gets skimmed, which is how the one entry that mattered stops being read.
Fix the widget, validate, then publish
Repair the widget named by path, then check the definition against https://studio.twilio.com/v2/Flows/Validate before you publish it — that endpoint answers the compile question without touching the live Flow. Leave this audit on a schedule: the fault arrives with an ordinary edit, not with a deploy, so nothing in your release process is watching for it.
How to check it worked
Re-run after the fix. Every flow should report valid, and the count of definitions that do not compile should be zero.
python3 twilio_studio_flow_validity_audit.py
# 7 flow(s), 0 with a definition that does not compileThe full code
One paginated GET for the Flows and one GET per flow for the detail, on an API Key with read access and nothing else. Two pure functions carry the judgement: one normalises errors[] and warnings[] into deduplicated path-and-message pairs, and one decides which of the five situations a flow is in. Both are worth reading on their own, because the whole note is the difference between an invalid draft and an invalid published Flow.
"""Report Twilio Studio Flows whose definition does not compile.
Read only. GET requests and nothing else: give this an API Key with read access
rather than the account auth token. The repair is printed, never performed,
because this script holds a credential to an account that can send messages and
spend money.
"""
import argparse
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("twilio_studio_flow_validity_audit")
STUDIO = "https://studio.twilio.com/v2"
def normalise(entries):
"""Reduce errors[] or warnings[] to deduplicated (path, message) pairs. Pure.
Each entry names the widget in `path` and what broke in `message`. Entries
that arrive as a bare string are kept with an empty path rather than
dropped: a finding with no location is still a finding, and silently losing
it is worse than printing it without one.
The same fault is reported once per referencing transition, so a single
deleted widget can produce four identical entries. Deduplicating here keeps
the report a list of problems rather than a list of mentions.
"""
out = []
for e in entries or []:
if isinstance(e, dict):
path = str(e.get("path") or "").strip()
message = str(e.get("message") or "").strip()
else:
path, message = "", str(e or "").strip()
if not (path or message):
continue
pair = (path, message)
if pair not in out:
out.append(pair)
return out
def verdict(flow):
"""Classify one Studio Flow by whether its definition compiles. Pure, so the
five cases sit together instead of being spread through a request loop.
`status` does not change the finding, only who is affected by it: a
published Flow is failing executions now, a draft cannot be published until
the widget is fixed. Returns (state, detail).
"""
valid = flow.get("valid")
status = str(flow.get("status") or "").lower()
errors = normalise(flow.get("errors"))
warnings = normalise(flow.get("warnings"))
if valid is None:
return ("unknown",
"no valid field on this response: read the single flow at "
"/v2/Flows/{FlowSid}, which is where errors[] and warnings[] "
"are carried.")
if valid is False:
where = errors[0][0] if errors and errors[0][0] else "an unnamed widget"
what = errors[0][1] if errors else "no message returned with the error"
if not errors:
detail = ("definition does not compile but errors[] came back empty. "
"Fetch the flow on its own; the list view is not where the "
"detail lives.")
elif status == "published":
detail = ("published and does not compile: executions stop at the "
"fault. %d error(s), first at %s: %s"
% (len(errors), where, what))
else:
detail = ("draft and does not compile, so it cannot be published at "
"all. %d error(s), first at %s: %s"
% (len(errors), where, what))
return ("invalid-published" if status == "published" else "invalid-draft",
detail)
if warnings:
return ("warnings",
"compiles, with %d warning(s), first at %s: %s"
% (len(warnings), warnings[0][0] or "an unnamed widget",
warnings[0][1] or "no message"))
return ("valid", "definition compiles with no errors or warnings")
def get(session, url, **params):
r = session.get(url, params=params, timeout=30)
if r.status_code in (401, 403):
raise SystemExit("%d from Twilio: check TWILIO_ACCOUNT_SID and that the "
"API key belongs to that account with read access"
% r.status_code)
r.raise_for_status()
return r.json()
def paged(session, url, key, limit, **first):
"""Page a studio.twilio.com list. meta.next_page_url is absolute."""
params = dict(first)
params.setdefault("PageSize", 50)
out = []
while url and len(out) < limit:
page = get(session, url, **params)
out.extend(page.get(key, []))
url = (page.get("meta") or {}).get("next_page_url")
params = {}
return out[:limit]
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--max-flows", type=int, default=200,
help="stop after this many flows")
ap.add_argument("--warnings", action="store_true",
help="also report flows that compile but carry warnings")
args = ap.parse_args()
account = os.environ.get("TWILIO_ACCOUNT_SID")
key = os.environ.get("TWILIO_API_KEY")
secret = os.environ.get("TWILIO_API_SECRET")
if not (account and key and secret):
log.error("set TWILIO_ACCOUNT_SID, TWILIO_API_KEY and TWILIO_API_SECRET "
"(an API Key with read access, not the auth token)")
return 2
session = requests.Session()
session.auth = (key, secret)
flows = paged(session, STUDIO + "/Flows", "flows", args.max_flows)
if not flows:
log.info("no Studio Flows on this account")
return 0
bad = 0
for listed in flows:
# The list gives the SIDs; errors[] and warnings[] come from the
# single-flow fetch, and without them there is nothing to report but a
# boolean saying something somewhere is wrong.
flow = get(session, "%s/Flows/%s" % (STUDIO, listed.get("sid")))
state, detail = verdict(flow)
line = "%-18s %s %s" % (state, flow.get("friendly_name") or flow.get("sid"),
detail)
if state == "valid":
log.info(line)
continue
if state == "warnings" and not args.warnings:
log.info("%-18s %s %d warning(s); re-run with --warnings to see them",
state, flow.get("friendly_name") or flow.get("sid"),
len(normalise(flow.get("warnings"))))
continue
bad += 1
log.warning(line)
for path, message in normalise(flow.get("errors")):
log.warning(" error at %s: %s", path or "(no path)", message)
if state.startswith("invalid"):
log.warning(" repair: fix the widget at that path in %s, check the "
"definition against %s/Flows/Validate, then republish.",
flow.get("sid"), STUDIO)
log.info("%d flow(s), %d with a definition that does not compile",
len(flows), bad)
return 1 if bad else 0
if __name__ == "__main__":
sys.exit(main())
/**
* Report Twilio Studio Flows whose definition does not compile.
*
* Read only. GET requests and nothing else: give this an API Key with read
* access rather than the account auth token. The repair is printed, never
* performed.
*/
const STUDIO = 'https://studio.twilio.com/v2';
/**
* Reduce errors[] or warnings[] to deduplicated [path, message] pairs. Pure.
*
* Entries that arrive as a bare string keep an empty path rather than being
* dropped. The same fault is reported once per referencing transition, so one
* deleted widget can produce four identical entries; deduplicating here keeps
* the report a list of problems rather than a list of mentions.
*/
export function normalise(entries) {
const out = [];
const seen = new Set();
for (const e of entries ?? []) {
let path = '';
let message = '';
if (e && typeof e === 'object') {
path = String(e.path ?? '').trim();
message = String(e.message ?? '').trim();
} else {
message = String(e ?? '').trim();
}
if (!path && !message) continue;
const key = `${path}\u0000${message}`;
if (seen.has(key)) continue;
seen.add(key);
out.push([path, message]);
}
return out;
}
/**
* Classify one Studio Flow by whether its definition compiles. Pure.
*
* `status` does not change the finding, only who is affected by it: a published
* Flow is failing executions now, a draft cannot be published until the widget
* is fixed. Returns [state, detail].
*/
export function verdict(flow) {
const valid = flow.valid;
const status = String(flow.status ?? '').toLowerCase();
const errors = normalise(flow.errors);
const warnings = normalise(flow.warnings);
if (valid === null || valid === undefined) {
return ['unknown',
'no valid field on this response: read the single flow at ' +
'/v2/Flows/{FlowSid}, which is where errors[] and warnings[] are carried.'];
}
if (valid === false) {
const where = errors.length && errors[0][0] ? errors[0][0] : 'an unnamed widget';
const what = errors.length ? errors[0][1] : 'no message returned with the error';
let detail;
if (!errors.length) {
detail = 'definition does not compile but errors[] came back empty. Fetch ' +
'the flow on its own; the list view is not where the detail lives.';
} else if (status === 'published') {
detail = `published and does not compile: executions stop at the fault. ` +
`${errors.length} error(s), first at ${where}: ${what}`;
} else {
detail = 'draft and does not compile, so it cannot be published at all. ' +
`${errors.length} error(s), first at ${where}: ${what}`;
}
return [status === 'published' ? 'invalid-published' : 'invalid-draft', detail];
}
if (warnings.length) {
return ['warnings',
`compiles, with ${warnings.length} warning(s), first at ` +
`${warnings[0][0] || 'an unnamed widget'}: ${warnings[0][1] || 'no message'}`];
}
return ['valid', 'definition compiles with no errors or warnings'];
}
function authHeader(key, secret) {
return `Basic ${Buffer.from(`${key}:${secret}`).toString('base64')}`;
}
async function get(auth, url, params = {}) {
const u = new URL(url);
for (const [k, v] of Object.entries(params)) u.searchParams.set(k, v);
const res = await fetch(u, { headers: { Authorization: auth } });
if (res.status === 401 || res.status === 403) {
throw new Error(`${res.status} from Twilio: check TWILIO_ACCOUNT_SID and ` +
'that the API key belongs to that account with read access');
}
if (!res.ok) throw new Error(`${res.status} from ${u.pathname}`);
return res.json();
}
async function paged(auth, url, key, limit, first = {}) {
let params = { PageSize: 50, ...first };
const out = [];
while (url && out.length < limit) {
const page = await get(auth, url, params);
out.push(...(page[key] ?? []));
url = (page.meta ?? {}).next_page_url ?? null;
params = {};
}
return out.slice(0, limit);
}
async function main() {
const account = process.env.TWILIO_ACCOUNT_SID;
const key = process.env.TWILIO_API_KEY;
const secret = process.env.TWILIO_API_SECRET;
if (!account || !key || !secret) {
console.error('set TWILIO_ACCOUNT_SID, TWILIO_API_KEY and TWILIO_API_SECRET ' +
'(an API Key with read access, not the auth token)');
process.exitCode = 2;
return;
}
const auth = authHeader(key, secret);
const showWarnings = process.argv.includes('--warnings');
const flows = await paged(auth, `${STUDIO}/Flows`, 'flows', 200);
if (!flows.length) {
console.log('no Studio Flows on this account');
return;
}
let bad = 0;
for (const listed of flows) {
const flow = await get(auth, `${STUDIO}/Flows/${listed.sid}`);
const [state, detail] = verdict(flow);
const name = flow.friendly_name || flow.sid;
const line = `${state.padEnd(18)} ${name} ${detail}`;
if (state === 'valid') { console.log(line); continue; }
if (state === 'warnings' && !showWarnings) {
console.log(`${state.padEnd(18)} ${name} ` +
`${normalise(flow.warnings).length} warning(s); re-run with ` +
'--warnings to see them');
continue;
}
bad += 1;
console.warn(line);
for (const [path, message] of normalise(flow.errors)) {
console.warn(` error at ${path || '(no path)'}: ${message}`);
}
if (state.startsWith('invalid')) {
console.warn(` repair: fix the widget at that path in ${flow.sid}, check ` +
`the definition against ${STUDIO}/Flows/Validate, then republish.`);
}
}
console.log(`${flows.length} flow(s), ${bad} with a definition that does not compile`);
process.exitCode = bad ? 1 : 0;
}
// Only run when invoked directly. The test file imports this module, and without
// the guard main() would run there too, fail on the missing credentials and set a
// non-zero exit code that fails the whole test file even as every test passes.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The cases worth pinning are the ones that decide what a human is told to do: an invalid published Flow, which is an outage, against an invalid draft, which must not be told to press Publish. Then the ones that make a report unreadable — four copies of the same error from four transitions into one deleted widget, and a warning on a Flow that compiles perfectly well.
from twilio_studio_flow_validity_audit import normalise, verdict
def flow(**kw):
base = {"sid": "FW1", "friendly_name": "support", "status": "published",
"valid": True, "errors": [], "warnings": []}
base.update(kw)
return base
def test_published_and_invalid_is_an_outage_now():
state, detail = verdict(flow(valid=False, errors=[
{"path": "states[3].transitions[0]", "message": "unknown next widget"}]))
assert state == "invalid-published"
assert "executions stop" in detail
assert "states[3].transitions[0]" in detail
def test_draft_and_invalid_is_never_told_to_publish():
state, detail = verdict(flow(status="draft", valid=False, errors=[
{"path": "states[1]", "message": "liquid syntax error"}]))
assert state == "invalid-draft"
assert "cannot be published" in detail
def test_one_deleted_widget_reported_four_times_is_one_error():
entry = {"path": "states[2]", "message": "transition to a deleted widget"}
state, detail = verdict(flow(valid=False, errors=[entry, dict(entry),
dict(entry), dict(entry)]))
assert state == "invalid-published"
assert "1 error(s)" in detail
def test_warnings_do_not_make_a_flow_invalid():
state, detail = verdict(flow(warnings=[{"path": "states[0]",
"message": "widget name is not unique"}]))
assert state == "warnings"
assert "compiles" in detail
def test_a_clean_flow_is_valid():
assert verdict(flow())[0] == "valid"
def test_invalid_with_an_empty_errors_array_says_where_to_look():
state, detail = verdict(flow(valid=False, errors=[]))
assert state == "invalid-published"
assert "Fetch the flow on its own" in detail
def test_a_response_with_no_valid_field_is_not_assumed_healthy():
listed = flow()
del listed["valid"]
assert verdict(listed)[0] == "unknown"
def test_normalise_keeps_string_entries_and_drops_empty_ones():
assert normalise(["transition to a deleted widget", {}, None, ""]) == \
[("", "transition to a deleted widget")]
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { normalise, verdict } from './twilio-studio-flow-validity-audit.mjs';
const flow = (kw = {}) => ({
sid: 'FW1', friendly_name: 'support', status: 'published', valid: true,
errors: [], warnings: [], ...kw,
});
test('published and invalid is an outage now', () => {
const [state, detail] = verdict(flow({
valid: false,
errors: [{ path: 'states[3].transitions[0]', message: 'unknown next widget' }],
}));
assert.equal(state, 'invalid-published');
assert.match(detail, /executions stop/);
assert.match(detail, /states\[3\]/);
});
test('draft and invalid is never told to publish', () => {
const [state, detail] = verdict(flow({
status: 'draft', valid: false,
errors: [{ path: 'states[1]', message: 'liquid syntax error' }],
}));
assert.equal(state, 'invalid-draft');
assert.match(detail, /cannot be published/);
});
test('one deleted widget reported four times is one error', () => {
const entry = { path: 'states[2]', message: 'transition to a deleted widget' };
const [state, detail] = verdict(flow({
valid: false, errors: [entry, { ...entry }, { ...entry }, { ...entry }],
}));
assert.equal(state, 'invalid-published');
assert.match(detail, /1 error\(s\)/);
});
test('warnings do not make a flow invalid', () => {
const [state, detail] = verdict(flow({
warnings: [{ path: 'states[0]', message: 'widget name is not unique' }],
}));
assert.equal(state, 'warnings');
assert.match(detail, /compiles/);
});
test('a clean flow is valid', () => {
assert.equal(verdict(flow())[0], 'valid');
});
test('invalid with an empty errors array says where to look', () => {
const [state, detail] = verdict(flow({ valid: false, errors: [] }));
assert.equal(state, 'invalid-published');
assert.match(detail, /Fetch the flow on its own/);
});
test('a response with no valid field is not assumed healthy', () => {
const listed = flow();
delete listed.valid;
assert.equal(verdict(listed)[0], 'unknown');
});
test('normalise keeps string entries and drops empty ones', () => {
assert.deepEqual(normalise(['transition to a deleted widget', {}, null, '']),
[['', 'transition to a deleted widget']]);
});
FAQ
What actually makes a Studio definition invalid?
Structural faults in the JSON behind the canvas: a transition naming a widget that has been deleted, a required field on a widget left empty, or Liquid that does not parse. Each one is reported as an entry in errors[] with the path of the widget it belongs to.
Why does the Console still render a Flow that does not compile?
Because rendering and compiling are different operations on the same JSON. Drawing a widget whose transition points at nothing produces a widget with one fewer arrow, which looks like a widget that simply has fewer arrows. Nothing turns red, so the canvas is not where you find this.
Is a warning worth fixing?
Often yes, but not urgently, and not in the same list. warnings[] can be populated on a Flow whose valid is true; those Flows run. Mixing warnings into the error report is how the one line that meant an outage ends up eleventh in a list nobody finishes reading.
Can I just press Publish and see whether it takes?
Not on an invalid definition, which is why the script separates a draft from a published Flow. The safe way to ask the question is the Validate endpoint: it answers whether a definition compiles without touching what is live.
Why fetch every flow separately instead of reading the list?
Because the diagnosis is errors[] and warnings[], and those are read from the single-flow fetch at /v2/Flows/{FlowSid}. An account with two hundred Flows costs two hundred and a few GETs, which is a cheap price for a report that names the widget rather than the Flow.
Related field notes
- A Studio Flow left in draft, live nowhere
- A published Flow no number points at
- TwiML that fails to parse: 12100
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Studio Flow resource (v2) — Twilio Docs
- Studio Flow Validate resource (v2) — Twilio Docs
- Studio Execution resource (v2) — Twilio Docs
- Studio FAQ — Twilio Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.