Diagnostic Twilio
call recordings stored without encryption at rest
The question arrives in a spreadsheet from an auditor and it is one line: are call recordings encrypted at rest. Nobody knows. Recording was switched on years ago by whoever built the support queue, the files play fine in the console, and the answer turns out to be a field that is either present or absent on each recording — absent on all four years of them, because Voice Recording Encryption is opt-in and nobody opted in.
Page GET /2010-04-01/Accounts/{AccountSid}/Recordings.json and read encryption_details on each row. It is absent when the recording was stored unencrypted, and present when it was not. That is the whole test: presence, not a value.
Read it across a date-ordered sample rather than on one recording. Enabling encryption is not retroactive, so the interesting answer is where in the timeline the field starts or stops appearing: everything on the unencrypted side of that boundary stays readable with account credentials for as long as you keep it.
The problem in plain words
Unencrypted media is not an outage, a cost or a bug, so nothing in the system has any reason to mention it. The recordings record, store, download and play exactly as intended. The only thing wrong is who can read them, and Twilio has no opinion about that: with encryption off, the media is retrievable by anything holding account credentials, which includes a leaked Auth Token, a key on a laptop, and every integration you have ever handed credentials to.
The reason it surfaces during an audit rather than before one is that the answer is genuinely not knowable from the console at a glance. Nothing shows a padlock. You have to fetch a recording resource and notice a field that is not there, which is a strange thing to go looking for unless somebody has asked. And when somebody does ask, the honest answer is usually not yes or no but "since when", which is a question only a date-ordered sweep can answer.
Why it happens
It is opt-in, and the default is off. Turning recording on is one switch and encrypting it is a different one, in a different place, that also requires you to generate a key pair and upload the public half. Nobody hits that path by accident, so an account that has never deliberately done it has none of it.
Enabling it does not reach backwards. New recordings carry encryption_details from the moment the setting goes on; the ones already stored do not, and never will. So an account that fixed this last year still holds however many years of plaintext media, and a check that samples only recent recordings reports it as clean.
Absence is the signal, which makes it easy to code past. The field is simply not there rather than set to a falsy value. Code that reads it defensively and moves on will never notice, and a check that compares it against something specific will fail to match anything at all.
Turning it off is as quiet as never turning it on. If the setting is disabled, recordings from that moment stop carrying the field while everything older keeps it. That reads as normal in every direction except a date-ordered one, which is why the classifier sorts before it judges.
This is a different problem from the storage bill. Recordings that accumulate forever are a cost with its own note; these are the same files considered as a liability rather than a line item. The two findings overlap in the fix, though: deleting a recording you no longer need is the only way to un-hold plaintext that is already stored.
The fix, as a flow
The classifier sorts before it judges, because a mix of encrypted and unencrypted recordings is two opposite findings and only the ordering tells you which of them you have.
How to fix it
Sample recordings across the whole range, not the recent ones
GET /2010-04-01/Accounts/{AccountSid}/Recordings.json?PageSize=1000, following next_page_uri. Because enabling encryption is not retroactive, a sample of only the newest recordings answers a different question from the one the auditor asked.
Test for presence of encryption_details, not for a value
The field is absent on an unencrypted recording. There is no flag set to false and nothing to compare against, so the check is truthiness and the report is a count of rows that have it against rows that do not.
Parse date_created as RFC 2822 before sorting
This API returns Tue, 18 Apr 2023 09:12:00 +0000, not ISO 8601 — the newer Twilio domains differ. Python reads it with email.utils.parsedate_to_datetime; datetime.fromisoformat returns nothing, and a sweep that silently sorts nothing looks exactly like a clean account.
Find the boundary and say which side is which
The newest unencrypted recording is the moment encryption was turned on, if the newer ones carry the field. If the newest recording is the unencrypted one and older ones are not, the setting was turned off instead, and everything since is in the clear. Those are opposite findings and only the ordering separates them.
Enable it, then decide what to do with the backlog
Console → Voice → Settings → General, enable Voice Recording Encryption and upload a public key. Keep the private half somewhere you will still have it in three years, because without it the encrypted recordings are unrecoverable. Then deal with the plaintext backlog deliberately: it does not get encrypted, so the choice is keep it or delete it.
How to check it worked
Re-run after enabling it and recording a new call. The newest rows should carry the field.
python3 twilio_recording_encryption_audit.py
# encrypted all 240 sampled recording(s) carry encryption details.The full code
One paginated GET over the recordings, with read access and nothing more. The classifier is pure and works on a date-ordered list rather than on a single recording, because every interesting answer here is about a boundary: encryption that started on a date and left everything before it in the clear, or encryption that stopped on one and left everything after it exposed. Those two look identical to a count and opposite to a sort.
"""Report Twilio call recordings stored without encryption at rest.
Voice Recording Encryption is opt-in. With it off, encryption_details is simply
absent from the recording and the media is retrievable by anything holding
account credentials. Enabling it later is not retroactive, so the useful answer
is not yes or no but since when.
Read only. GET requests and nothing else: give this an API Key with read access
rather than the account auth token. The repair is printed, never performed,
because this script holds a credential to an account that can send messages and
spend money.
"""
import argparse
import email.utils
import logging
import os
import sys
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("twilio_recording_encryption_audit")
HOST = "https://api.twilio.com"
BASE = HOST + "/2010-04-01"
def is_encrypted(recording):
"""True when the recording carries encryption details.
A presence test rather than a comparison: with encryption off the field is
absent, not set to false, so there is nothing to compare it against and code
that looks for a specific value matches nothing at all.
"""
return bool(recording.get("encryption_details"))
def parse_when(value):
"""Parse date_created from the 2010-04-01 API.
This API returns RFC 2822 (Tue, 18 Apr 2023 09:12:00 +0000). The newer
Twilio domains return ISO 8601, so a parser written for one returns nothing
for the other, and a sweep that silently sorts nothing reads exactly like a
clean account.
"""
text = str(value or "").strip()
if not text:
return None
try:
return email.utils.parsedate_to_datetime(text)
except (TypeError, ValueError):
return None
def newest_first(recordings):
"""Sort a sample newest first, keeping undated rows at the end. Pure.
The ordering is the analysis. A count of encrypted against unencrypted says
nothing about which of the two opposite findings you have.
"""
rows = list(recordings or [])
dated = [(parse_when(r.get("date_created")), r) for r in rows]
have = sorted([p for p in dated if p[0]], key=lambda p: p[0], reverse=True)
return [r for _when, r in have] + [r for when, r in dated if not when]
def switch_point(recordings):
"""The date_created of the newest recording with no encryption details.
On an account where encryption was turned on, that is the moment it happened:
nothing after it is in the clear, and nothing before it will ever be
encrypted, because the setting does not reach backwards.
"""
for recording in newest_first(recordings):
if not is_encrypted(recording):
return recording.get("date_created")
return None
def verdict(recordings):
"""Classify a date-ordered sample of recordings. Pure, so the rules can be
tested without a network.
Returns (state, detail).
"""
rows = newest_first(recordings)
if not rows:
return ("none", "no recordings on this account: nothing stored, so "
"nothing stored in the clear.")
plain = [r for r in rows if not is_encrypted(r)]
if not plain:
return ("encrypted",
"all %d sampled recording(s) carry encryption details." % len(rows))
if len(plain) == len(rows):
return ("plaintext",
"none of the %d sampled recording(s) carry encryption details: "
"Voice Recording Encryption has never been on, and every one of "
"these is readable by anything holding account credentials."
% len(rows))
if is_encrypted(rows[0]):
return ("backlog",
"the newest sampled recording is encrypted and %d older one(s) "
"are not: enabling encryption does not reach backwards, so those "
"stay in the clear for as long as you keep them." % len(plain))
return ("regressed",
"the newest sampled recording has no encryption details while %d "
"older one(s) do: encryption was on and is not any more, so "
"everything recorded since it stopped is in the clear."
% (len(rows) - len(plain)))
def get(session, url, **params):
r = session.get(url, params=params, timeout=30)
if r.status_code in (401, 403):
raise SystemExit("%d from Twilio: check TWILIO_ACCOUNT_SID and that the "
"API key belongs to that account with read access"
% r.status_code)
r.raise_for_status()
return r.json()
def list_recordings(session, account, limit):
"""Page Recordings. next_page_uri is a path, not an absolute URL."""
url = "%s/Accounts/%s/Recordings.json" % (BASE, account)
params = {"PageSize": 1000}
out = []
while url and len(out) < limit:
page = get(session, url, **params)
out.extend(page.get("recordings", []))
nxt = page.get("next_page_uri")
url, params = (HOST + nxt) if nxt else None, {}
return out[:limit]
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--max-recordings", type=int, default=2000,
help="stop after this many recordings")
args = ap.parse_args()
account = os.environ.get("TWILIO_ACCOUNT_SID")
key = os.environ.get("TWILIO_API_KEY")
secret = os.environ.get("TWILIO_API_SECRET")
if not (account and key and secret):
log.error("set TWILIO_ACCOUNT_SID, TWILIO_API_KEY and TWILIO_API_SECRET "
"(an API Key with read access, not the auth token)")
return 2
session = requests.Session()
session.auth = (key, secret)
recordings = list_recordings(session, account, args.max_recordings)
state, detail = verdict(recordings)
if state in ("none", "encrypted"):
log.info("%-14s %s", state, detail)
return 0
log.warning("%-14s %s", state, detail)
boundary = switch_point(recordings)
if boundary and state != "plaintext":
log.warning(" newest unencrypted recording: %s", boundary)
log.warning(" repair: Console > Voice > Settings > General, enable Voice "
"Recording Encryption and upload a public key. Keep the private "
"half: without it the encrypted recordings are unrecoverable")
log.warning(" the recordings already stored in the clear are not re-encrypted "
"when you enable it, so decide separately whether to keep them")
return 1
if __name__ == "__main__":
sys.exit(main())
/**
* Report Twilio call recordings stored without encryption at rest.
*
* Voice Recording Encryption is opt-in. With it off, encryption_details is
* simply absent and the media is retrievable by anything holding account
* credentials. Enabling it later is not retroactive, so the useful answer is not
* yes or no but since when.
*
* Read only. GET requests and nothing else: give this an API Key with read
* access rather than the account auth token. The repair is printed, never
* performed.
*/
const HOST = 'https://api.twilio.com';
const BASE = `${HOST}/2010-04-01`;
/**
* True when the recording carries encryption details. A presence test rather
* than a comparison: with encryption off the field is absent, not false.
*/
export function isEncrypted(recording) {
return Boolean(recording.encryption_details);
}
/**
* Parse date_created from the 2010-04-01 API, which returns RFC 2822 rather than
* the ISO 8601 the newer Twilio domains use.
*/
export function parseWhen(value) {
const text = String(value ?? '').trim();
if (!text) return null;
const ms = Date.parse(text);
return Number.isNaN(ms) ? null : ms;
}
/**
* Sort a sample newest first, keeping undated rows at the end. Pure. The
* ordering is the analysis: a count says nothing about which of the two
* opposite findings you have.
*/
export function newestFirst(recordings) {
const rows = [...(recordings ?? [])];
const dated = rows.map((r) => [parseWhen(r.date_created), r]);
const have = dated.filter(([w]) => w !== null).sort((a, b) => b[0] - a[0]);
return [...have.map(([, r]) => r), ...dated.filter(([w]) => w === null).map(([, r]) => r)];
}
/**
* The date_created of the newest recording with no encryption details: on an
* account where encryption was turned on, the moment it happened.
*/
export function switchPoint(recordings) {
for (const recording of newestFirst(recordings)) {
if (!isEncrypted(recording)) return recording.date_created ?? null;
}
return null;
}
/**
* Classify a date-ordered sample of recordings. Pure. Returns [state, detail].
*/
export function verdict(recordings) {
const rows = newestFirst(recordings);
if (rows.length === 0) {
return ['none',
'no recordings on this account: nothing stored, so nothing stored in the clear.'];
}
const plain = rows.filter((r) => !isEncrypted(r));
if (plain.length === 0) {
return ['encrypted',
`all ${rows.length} sampled recording(s) carry encryption details.`];
}
if (plain.length === rows.length) {
return ['plaintext',
`none of the ${rows.length} sampled recording(s) carry encryption details: ` +
'Voice Recording Encryption has never been on, and every one of these is ' +
'readable by anything holding account credentials.'];
}
if (isEncrypted(rows[0])) {
return ['backlog',
`the newest sampled recording is encrypted and ${plain.length} older one(s) ` +
'are not: enabling encryption does not reach backwards, so those stay in ' +
'the clear for as long as you keep them.'];
}
return ['regressed',
`the newest sampled recording has no encryption details while ` +
`${rows.length - plain.length} older one(s) do: encryption was on and is not ` +
'any more, so everything recorded since it stopped is in the clear.'];
}
function authHeader(key, secret) {
return `Basic ${Buffer.from(`${key}:${secret}`).toString('base64')}`;
}
async function get(auth, url, params = {}) {
const u = new URL(url);
for (const [k, v] of Object.entries(params)) u.searchParams.set(k, v);
const res = await fetch(u, { headers: { Authorization: auth } });
if (res.status === 401 || res.status === 403) {
throw new Error(`${res.status} from Twilio: check TWILIO_ACCOUNT_SID and ` +
'that the API key belongs to that account with read access');
}
if (!res.ok) throw new Error(`${res.status} from ${u.pathname}`);
return res.json();
}
export async function listRecordings(auth, account, limit = 2000) {
let url = `${BASE}/Accounts/${account}/Recordings.json`;
let params = { PageSize: 1000 };
const out = [];
while (url && out.length < limit) {
const page = await get(auth, url, params);
out.push(...(page.recordings ?? []));
url = page.next_page_uri ? HOST + page.next_page_uri : null;
params = {};
}
return out.slice(0, limit);
}
async function main() {
const account = process.env.TWILIO_ACCOUNT_SID;
const key = process.env.TWILIO_API_KEY;
const secret = process.env.TWILIO_API_SECRET;
if (!account || !key || !secret) {
console.error('set TWILIO_ACCOUNT_SID, TWILIO_API_KEY and TWILIO_API_SECRET ' +
'(an API Key with read access, not the auth token)');
process.exitCode = 2;
return;
}
const auth = authHeader(key, secret);
const recordings = await listRecordings(auth, account);
const [state, detail] = verdict(recordings);
if (state === 'none' || state === 'encrypted') {
console.log(`${state.padEnd(14)} ${detail}`);
return;
}
console.warn(`${state.padEnd(14)} ${detail}`);
const boundary = switchPoint(recordings);
if (boundary && state !== 'plaintext') {
console.warn(` newest unencrypted recording: ${boundary}`);
}
console.warn(' repair: Console > Voice > Settings > General, enable Voice ' +
'Recording Encryption and upload a public key. Keep the private ' +
'half: without it the encrypted recordings are unrecoverable');
console.warn(' the recordings already stored in the clear are not re-encrypted ' +
'when you enable it, so decide separately whether to keep them');
process.exitCode = 1;
}
// Only run when invoked directly. The test file imports this module, and without
// the guard main() would run there too, fail on the missing credentials and set a
// non-zero exit code that fails the whole test file even as every test passes.
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((err) => { console.error(err.message); process.exitCode = 2; });
}
Add a test
The two cases that share a count and differ in every other respect are the ones to pin: encryption switched on last year, leaving an old backlog in the clear, and encryption switched off last month, leaving everything recent in the clear. Both are a mix of encrypted and unencrypted rows. Only the ordering says which, and only one of them is still getting worse.
from twilio_recording_encryption_audit import (
is_encrypted, newest_first, parse_when, switch_point, verdict)
DETAILS = {"type": "rsa-aes-cbc-gcm"}
def rec(day, encrypted=False, sid="RE01"):
row = {"sid": sid, "date_created": "Tue, %02d Apr 2024 09:12:00 +0000" % day}
if encrypted:
row["encryption_details"] = DETAILS
return row
def test_an_account_with_no_recordings_has_nothing_in_the_clear():
state, detail = verdict([])
assert state == "none"
assert "nothing stored" in detail
def test_every_recording_encrypted_is_the_clean_answer():
state, _ = verdict([rec(1, True), rec(2, True)])
assert state == "encrypted"
def test_nothing_encrypted_anywhere_means_it_was_never_switched_on():
state, detail = verdict([rec(1), rec(2)])
assert state == "plaintext"
assert "never been on" in detail
def test_newest_encrypted_and_older_not_is_a_backlog_that_stays():
state, detail = verdict([rec(1), rec(2), rec(3, True)])
assert state == "backlog"
assert "2 older one(s)" in detail
assert "does not reach backwards" in detail
def test_newest_unencrypted_while_older_ones_are_encrypted_is_a_regression():
state, detail = verdict([rec(1, True), rec(2, True), rec(3)])
assert state == "regressed"
assert "was on and is not any more" in detail
def test_the_two_mixed_cases_are_told_apart_only_by_the_ordering():
rows = [rec(1), rec(2), rec(3, True)]
assert verdict(rows)[0] == "backlog"
assert verdict(list(reversed(rows)))[0] == "backlog"
def test_the_switch_point_is_the_newest_recording_still_in_the_clear():
assert switch_point([rec(1), rec(5), rec(9, True)]) == \
"Tue, 05 Apr 2024 09:12:00 +0000"
assert switch_point([rec(1, True)]) is None
def test_presence_is_the_test_rather_than_a_value():
assert is_encrypted({"encryption_details": DETAILS}) is True
assert is_encrypted({"encryption_details": None}) is False
assert is_encrypted({}) is False
def test_an_unparseable_date_sorts_last_instead_of_raising():
rows = newest_first([{"sid": "RE99", "date_created": "whenever"}, rec(4)])
assert [r["sid"] for r in rows] == ["RE01", "RE99"]
def test_parse_when_reads_rfc_2822():
assert parse_when("Tue, 18 Apr 2023 09:12:00 +0000") is not None
assert parse_when("") is None
assert parse_when("not a date") is None
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
isEncrypted, newestFirst, parseWhen, switchPoint, verdict,
} from './twilio-recording-encryption-audit.mjs';
const DETAILS = { type: 'rsa-aes-cbc-gcm' };
const rec = (day, encrypted = false, sid = 'RE01') => {
const row = {
sid,
date_created: `Tue, ${String(day).padStart(2, '0')} Apr 2024 09:12:00 +0000`,
};
if (encrypted) row.encryption_details = DETAILS;
return row;
};
test('an account with no recordings has nothing in the clear', () => {
const [state, detail] = verdict([]);
assert.equal(state, 'none');
assert.match(detail, /nothing stored/);
});
test('every recording encrypted is the clean answer', () => {
assert.equal(verdict([rec(1, true), rec(2, true)])[0], 'encrypted');
});
test('nothing encrypted anywhere means it was never switched on', () => {
const [state, detail] = verdict([rec(1), rec(2)]);
assert.equal(state, 'plaintext');
assert.match(detail, /never been on/);
});
test('newest encrypted and older not is a backlog that stays', () => {
const [state, detail] = verdict([rec(1), rec(2), rec(3, true)]);
assert.equal(state, 'backlog');
assert.match(detail, /2 older one\(s\)/);
assert.match(detail, /does not reach backwards/);
});
test('newest unencrypted while older ones are encrypted is a regression', () => {
const [state, detail] = verdict([rec(1, true), rec(2, true), rec(3)]);
assert.equal(state, 'regressed');
assert.match(detail, /was on and is not any more/);
});
test('the two mixed cases are told apart only by the ordering', () => {
const rows = [rec(1), rec(2), rec(3, true)];
assert.equal(verdict(rows)[0], 'backlog');
assert.equal(verdict([...rows].reverse())[0], 'backlog');
});
test('the switch point is the newest recording still in the clear', () => {
assert.equal(switchPoint([rec(1), rec(5), rec(9, true)]),
'Tue, 05 Apr 2024 09:12:00 +0000');
assert.equal(switchPoint([rec(1, true)]), null);
});
test('presence is the test rather than a value', () => {
assert.equal(isEncrypted({ encryption_details: DETAILS }), true);
assert.equal(isEncrypted({ encryption_details: null }), false);
assert.equal(isEncrypted({}), false);
});
test('an unparseable date sorts last instead of throwing', () => {
const rows = newestFirst([{ sid: 'RE99', date_created: 'whenever' }, rec(4)]);
assert.deepEqual(rows.map((r) => r.sid), ['RE01', 'RE99']);
});
test('parseWhen reads RFC 2822', () => {
assert.notEqual(parseWhen('Tue, 18 Apr 2023 09:12:00 +0000'), null);
assert.equal(parseWhen(''), null);
assert.equal(parseWhen('not a date'), null);
});
FAQ
Can I encrypt the recordings that are already stored?
No. Enabling Voice Recording Encryption applies from the moment it is on, and everything recorded before that stays exactly as it is. That is why the script reports a boundary rather than a yes or no: the backlog is a separate decision, and the only ways to stop holding plaintext media are to delete it or to accept that you are holding it.
Is this the same problem as recordings piling up and billing forever?
No, but they are the same files. Storage cost is about how many you keep and for how long, and has its own note. This is about who can read them while you keep them. They meet at the fix, though: deleting a recording is the only action that resolves both, which is why a retention policy usually ends up doing more for this finding than encryption does.
What actually protects the recordings without encryption?
Account credentials, and nothing else. The media is retrievable by anything holding them, so the blast radius of a leaked Auth Token includes every call you have ever recorded. That is the sentence to bring to the review, and it is also the reason this section keeps insisting on read-scoped API keys instead.
What happens if we lose the private key?
The recordings encrypted with the matching public key become unrecoverable, permanently. Twilio does not hold your private half, which is the entire point of the design. Treat the key as a production secret with the same backup and rotation story as anything else you cannot regenerate, and check that story exists before you enable the feature rather than after.
Why sample recordings across the whole range rather than the recent ones?
Because the recent ones answer a question nobody asked. If encryption was enabled last year, the newest thousand recordings are all encrypted and the report reads clean while four years of plaintext sit underneath. The finding lives at the boundary, so the sweep has to reach far enough back to include it.
Related field notes
- The same recordings considered as a bill
- The credential that can fetch every one of them
- Who else still holds a way in
Sources
Every figure in this note is traced to one of these. Prices are list rates and change — check them for your own region before acting.
- Voice recording encryption — Twilio Docs
- Recording resource — Twilio Docs
- Manage unused resources — Twilio Docs
- API keys — Twilio Docs
If your setup is misbehaving in a way this note does not cover, message me on LinkedIn with what you are seeing.