Diagnostic Webhooks & Events
Webhook payload fields diverge from documented schema
You built your handler against the sample payload in the Saleor docs, or against the fields your subscription query asked for, and it worked. Then a field you rely on shows up as null, or a whole nested object disappears, or an extra field appears that nothing in the docs mentions. Saleor never raised an error. Nothing failed. The payload just stopped matching what you expected. Here is why that gap exists and a script that diffs a real delivery against what it should contain and tells you exactly what changed.
Saleor supports two incompatible ways to shape a webhook payload. A legacy webhook gets a fixed, hard-coded JSON shape that matches the sample payloads in the docs. A webhook created with a subscription query on the query field of Webhook gets whatever fields that specific GraphQL subscription selects, which is not a fixed documented schema at all. Drift shows up when the subscription query goes stale after a Saleor field is renamed, deprecated, or moved behind a new type, such as the 3.22 useLegacyUpdateWebhookEmission change altering whether metadata-only updates fire *_UPDATED events, or when someone compares a subscription delivery against the generic legacy sample by mistake. Saleor never validates a delivered payload against the docs or against the query at send time, so nothing errors. Run a small Python or Node.js script that pulls a webhook's subscriptionQuery, extracts the fields it selects, fetches a recent delivery, and diffs the parsed payload against those fields. Full code, tests, and a dry run guarded report are below.
The problem in plain words
Saleor webhooks come in two flavors that look similar from the outside but behave very differently. The old style is a legacy, hard-coded payload. Saleor decides what goes in it, the docs show a sample for every event, and that sample is a reliable contract you can build against.
The newer style lets you attach a GraphQL subscription query to the webhook itself, stored in its query field, with a fragment like on ProductUpdated { product { id name } }. When that event fires, Saleor executes your query and sends back exactly what it selects. There is no fixed schema here. The payload is whatever your query asks for, and Saleor has no concept of a documented shape to check it against. If your query asks for five fields, you get five fields. If it asks for fifteen, you get fifteen. Change the query and the very next delivery has a different shape, with no version bump and no warning.
Why it happens
- Saleor supports two incompatible webhook payload mechanisms side by side, a legacy hard-coded shape and a subscription-defined shape set by the
queryfield onWebhook, and nothing in the delivery process reconciles the two, a gap raised directly in saleor/saleor#8054. - A subscription's delivered fields are exactly the fields it selects, so when a Saleor upgrade renames a field, deprecates one, or moves it behind a new type, any webhook whose query still references the old shape gets a payload with that field missing or null, and no error surfaces anywhere in the pipeline.
- The 3.22
useLegacyUpdateWebhookEmissionchange altered whether metadata-only updates fire*_UPDATEDevents at all, so a subscription written before that change can start seeing fewer deliveries, or deliveries with a different trigger condition, without the query itself ever being edited. - Developers frequently diff a real subscription delivery against the generic sample payload shown on the legacy webhooks documentation pages, which was never meant to describe a subscription payload, a confusion documented across saleor/saleor#9500 and saleor/saleor#14194.
- A webhook created through the Dashboard and one created by calling the API directly can end up with subtly different queries for what looks like the same event, so two webhooks pointed at the same URL can legitimately deliver different shapes.
None of this trips an alarm. The webhook still fires, Saleor still returns a 200 to itself internally, and the delivery still lands in your app's queue. The only sign something is wrong is that a field your handler reads comes back null, missing, or renamed, usually discovered only when a downstream process breaks.
You cannot fix this by reading the docs harder, because the docs describe the legacy shape, not your subscription's shape. The only reliable check is to compare a real delivered payload against the fields your own subscriptionQuery actually asks for. Pull the query, walk its selection set, pull a recent delivery, parse the payload, and diff the two key sets. Anything the query asked for that is missing or null in the payload is drift from Saleor's side. Anything in the payload that the query never asked for is worth a second look too, since it usually means you are diffing against the wrong query version.
The fix, as a flow
The script lists every webhook and its subscriptionQuery, extracts the flat field names each event type selects, pulls a handful of recent eventDeliveries, parses each delivery's payload JSON, and diffs it against the expected fields with one pure function. Every webhook gets a report of missing and unexpected fields, keyed by event type, with a sample delivery id attached. Nothing about the subscription query is ever rewritten automatically.
Build it step by step
Get an app token with read access to webhooks
Create an app in Saleor Dashboard under Configuration, Apps, and give it permission to read apps and webhooks, or exchange staff credentials with tokenCreate. Keep the API URL and token in environment variables, never in the file.
pip install requests
export SALEOR_API_URL="https://store.saleor.cloud/graphql/"
export SALEOR_AUTH_TOKEN="your-app-or-staff-token"
export DRY_RUN="true" # start safe, this script only reports by default// Node 18+ has fetch built in, no dependencies needed
export SALEOR_API_URL="https://store.saleor.cloud/graphql/"
export SALEOR_AUTH_TOKEN="your-app-or-staff-token"
export DRY_RUN="true" // start safe, this script only reports by defaultTalk to the Saleor GraphQL API
Saleor is one GraphQL endpoint. Every call is a POST with a JSON body of {query, variables} and an Authorization: Bearer <token> header. A small helper sends a query and returns the data, raising if Saleor reports errors.
import os, requests
API_URL = os.environ["SALEOR_API_URL"]
TOKEN = os.environ["SALEOR_AUTH_TOKEN"]
def gql(query, variables=None):
r = requests.post(
API_URL,
json={"query": query, "variables": variables or {}},
headers={"Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json"},
timeout=30,
)
r.raise_for_status()
body = r.json()
if body.get("errors"):
raise RuntimeError(body["errors"])
return body["data"]const API_URL = process.env.SALEOR_API_URL;
const TOKEN = process.env.SALEOR_AUTH_TOKEN;
async function gql(query, variables = {}) {
const res = await fetch(API_URL, {
method: "POST",
headers: { Authorization: `Bearer ${TOKEN}`, "Content-Type": "application/json" },
body: JSON.stringify({ query, variables }),
});
if (!res.ok) throw new Error(`Saleor ${res.status}`);
const body = await res.json();
if (body.errors) throw new Error(JSON.stringify(body.errors));
return body.data;
}List webhooks and their subscription queries
Ask for every webhook's id, name, targetUrl, isActive, subscriptionQuery, and events. The subscriptionQuery is the raw GraphQL string stored on the webhook, the same one Saleor executes at delivery time. Webhooks with no subscription query are legacy webhooks, and their expected shape comes from the documented sample instead of a query.
WEBHOOKS_QUERY = """
query {
webhooks(first: 100) {
edges {
node { id name targetUrl isActive subscriptionQuery events }
}
}
}"""
def list_webhooks():
data = gql(WEBHOOKS_QUERY)["webhooks"]
return [edge["node"] for edge in data["edges"]]const WEBHOOKS_QUERY = `
query {
webhooks(first: 100) {
edges {
node { id name targetUrl isActive subscriptionQuery events }
}
}
}`;
async function listWebhooks() {
const data = (await gql(WEBHOOKS_QUERY)).webhooks;
return data.edges.map((edge) => edge.node);
}Extract the flat field names a subscription query selects
You do not need a full GraphQL parser to get useful drift detection. A small regex walk over the query string, collecting every bare field name token and skipping GraphQL keywords, fragment markers, and argument blocks, is enough to build the flat list of fields the query expects at the top level of its selection set. For legacy webhooks with no query, fall back to a documented sample field list you keep for each event type.
import re
GRAPHQL_KEYWORDS = {"query", "mutation", "subscription", "fragment", "on"}
FIELD_TOKEN = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)\s*(?:\([^)]*\))?\s*\{")
BARE_TOKEN = re.compile(r"^\s*([A-Za-z_][A-Za-z0-9_]*)\s*$")
def extract_selection_fields(fragment_body):
"""Return the flat field names selected at the top level of one
braces-delimited selection set, ignoring nested sub-selections."""
depth = 0
fields = []
i = 0
n = len(fragment_body)
while i < n:
ch = fragment_body[i]
if ch == "{":
depth += 1
i += 1
continue
if ch == "}":
depth -= 1
i += 1
continue
if depth == 1:
m = FIELD_TOKEN.match(fragment_body, i)
if m and m.group(1) not in GRAPHQL_KEYWORDS:
fields.append(m.group(1))
i = m.end() - 1
continue
m2 = re.match(r"[A-Za-z_][A-Za-z0-9_]*", fragment_body[i:])
if m2:
token = m2.group(0)
nxt = fragment_body[i + len(token):].lstrip()
if not nxt.startswith("{") and token not in GRAPHQL_KEYWORDS:
fields.append(token)
i += len(token)
continue
i += 1
return sorted(set(fields))const GRAPHQL_KEYWORDS = new Set(["query", "mutation", "subscription", "fragment", "on"]);
export function extractSelectionFields(fragmentBody) {
// Flat field names selected at the top level of one braces-delimited
// selection set, ignoring nested sub-selections.
let depth = 0;
const fields = new Set();
let i = 0;
const n = fragmentBody.length;
while (i < n) {
const ch = fragmentBody[i];
if (ch === "{") { depth++; i++; continue; }
if (ch === "}") { depth--; i++; continue; }
if (depth === 1) {
const m = /^([A-Za-z_][A-Za-z0-9_]*)\s*(?:\([^)]*\))?\s*\{/.exec(fragmentBody.slice(i));
if (m && !GRAPHQL_KEYWORDS.has(m[1])) {
fields.add(m[1]);
i += m[0].length - 1;
continue;
}
const m2 = /^[A-Za-z_][A-Za-z0-9_]*/.exec(fragmentBody.slice(i));
if (m2) {
const token = m2[0];
const rest = fragmentBody.slice(i + token.length).trimStart();
if (!rest.startsWith("{") && !GRAPHQL_KEYWORDS.has(token)) {
fields.add(token);
}
i += token.length;
continue;
}
}
i++;
}
return Array.from(fields).sort();
}Fetch a recent delivery and parse its payload
Pull a handful of recent eventDeliveries for the webhook, ordered newest first, and read each one's eventType and raw payload string. Parse the payload as JSON. If it fails to parse, that is its own kind of drift worth reporting, since a delivery you cannot even parse cannot be diffed at all.
DELIVERIES_QUERY = """
query($webhookId: ID!) {
webhook(id: $webhookId) {
eventDeliveries(first: 50, sortBy: { field: CREATED_AT, direction: DESC }) {
edges {
node {
id eventType payload createdAt
attempts(first: 1) { edges { node { responseStatusCode response } } }
}
}
}
}
}"""
def recent_deliveries(webhook_id):
data = gql(DELIVERIES_QUERY, {"webhookId": webhook_id})["webhook"]
return [edge["node"] for edge in data["eventDeliveries"]["edges"]]
def parse_payload(delivery):
import json
try:
return json.loads(delivery["payload"])
except (TypeError, ValueError):
return Noneconst DELIVERIES_QUERY = `
query($webhookId: ID!) {
webhook(id: $webhookId) {
eventDeliveries(first: 50, sortBy: { field: CREATED_AT, direction: DESC }) {
edges {
node {
id eventType payload createdAt
attempts(first: 1) { edges { node { responseStatusCode response } } }
}
}
}
}
}`;
async function recentDeliveries(webhookId) {
const data = (await gql(DELIVERIES_QUERY, { webhookId })).webhook;
return data.eventDeliveries.edges.map((edge) => edge.node);
}
export function parsePayload(delivery) {
try {
return JSON.parse(delivery.payload);
} catch {
return null;
}
}Diff the payload against the expected fields, with one pure function
The decision lives in a pure function that takes the parsed payload, the flat list of expected field names, and returns two lists, fields the query or schema expected but the payload is missing or has null, and fields the payload has that were never expected. No network, no file I/O, so it is trivial to test with fixture pairs for a renamed field, a deprecated field returning null, and a newer field a later Saleor version added.
def diff_payload_against_schema(payload, expected_fields, path=""):
missing_in_payload = []
unexpected_in_payload = []
if not isinstance(payload, dict):
return {
"missingInPayload": [f"{path}.{f}" if path else f for f in expected_fields],
"unexpectedInPayload": [],
}
expected_set = set(expected_fields)
payload_keys = set(payload.keys())
for field in expected_fields:
label = f"{path}.{field}" if path else field
if field not in payload or payload[field] is None:
missing_in_payload.append(label)
for key in payload_keys:
label = f"{path}.{key}" if path else key
if key not in expected_set:
unexpected_in_payload.append(label)
return {
"missingInPayload": sorted(missing_in_payload),
"unexpectedInPayload": sorted(unexpected_in_payload),
}export function diffPayloadAgainstSchema(payload, expectedFields, options = {}) {
const path = options.path || "";
const label = (name) => (path ? `${path}.${name}` : name);
if (typeof payload !== "object" || payload === null || Array.isArray(payload)) {
return {
missingInPayload: expectedFields.map(label).sort(),
unexpectedInPayload: [],
};
}
const expectedSet = new Set(expectedFields);
const payloadKeys = Object.keys(payload);
const missingInPayload = expectedFields
.filter((field) => !(field in payload) || payload[field] === null)
.map(label)
.sort();
const unexpectedInPayload = payloadKeys
.filter((key) => !expectedSet.has(key))
.map(label)
.sort();
return { missingInPayload, unexpectedInPayload };
}Report the drift, and only touch the query behind a reviewed dry run
Under DRY_RUN=true, the default, the script only logs each webhook's drift report, missing fields, unexpected fields, and the sample delivery id it diffed. There is no safe automatic rewrite, the app's handler code depends on the exact fields it currently requests. If a human reviews the report and supplies a specific corrected query string, the only mutating call is webhookUpdate, and even then the script prints the old versus new query diff first and only applies it when DRY_RUN=false.
Never let a script guess a corrected subscription query. Report the drift, show a human the exact missing and unexpected fields plus a real delivery id to inspect, and only call webhookUpdate with a query string that person reviewed and approved, always with DRY_RUN=true first to print the diff.
The full code
Here is the complete script in one file for each language. It reads settings from the environment, lists webhooks, extracts expected fields from each subscription query, fetches a recent delivery, diffs the parsed payload, and reports drift. It only calls webhookUpdate when a human supplies a reviewed replacement query and dry run is off.
"""Find Saleor webhooks whose delivered payload no longer matches the
fields their own subscriptionQuery asks for.
Saleor supports two incompatible webhook payload mechanisms: a legacy
hard-coded shape documented with a sample payload per event, and a
subscription-defined shape set by the query field on Webhook, which
delivers exactly whatever that GraphQL query selects. There is no fixed
schema for a subscription webhook. Drift shows up when the query goes
stale after a Saleor field is renamed, deprecated, or moved behind a new
type (saleor/saleor#8054, #9500, discussion #14194), including behavior
changes like the 3.22 useLegacyUpdateWebhookEmission setting that altered
whether metadata-only updates fire *_UPDATED events at all.
This script never rewrites a subscription query on its own. Under
DRY_RUN=true (the default) it only reports drift per webhook: missing
fields, unexpected fields, and a sample delivery id. When DRY_RUN=false
and NEW_SUBSCRIPTION_QUERY is set for a specific WEBHOOK_ID a human has
reviewed, it prints the old versus new query and calls webhookUpdate.
Run on a schedule. Safe to run again and again.
"""
import os
import re
import json
import logging
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("detect_webhook_payload_drift")
API_URL = os.environ["SALEOR_API_URL"]
TOKEN = os.environ["SALEOR_AUTH_TOKEN"]
DRY_RUN = os.environ.get("DRY_RUN", "true").lower() == "true"
WEBHOOK_ID = os.environ.get("WEBHOOK_ID", "")
NEW_SUBSCRIPTION_QUERY = os.environ.get("NEW_SUBSCRIPTION_QUERY", "")
GRAPHQL_KEYWORDS = {"query", "mutation", "subscription", "fragment", "on"}
FIELD_TOKEN = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)\s*(?:\([^)]*\))?\s*\{")
# Documented sample fields for legacy (non-subscription) webhooks, per event
# type. Extend this as needed for events you run as legacy webhooks.
LEGACY_SAMPLE_FIELDS = {
"PRODUCT_UPDATED": ["id", "name", "slug", "category"],
"ORDER_CREATED": ["id", "number", "status", "userEmail", "total"],
}
WEBHOOKS_QUERY = """
query {
webhooks(first: 100) {
edges {
node { id name targetUrl isActive subscriptionQuery events }
}
}
}"""
DELIVERIES_QUERY = """
query($webhookId: ID!) {
webhook(id: $webhookId) {
eventDeliveries(first: 50, sortBy: { field: CREATED_AT, direction: DESC }) {
edges {
node {
id eventType payload createdAt
attempts(first: 1) { edges { node { responseStatusCode response } } }
}
}
}
}
}"""
WEBHOOK_UPDATE = """
mutation($id: ID!, $query: String!) {
webhookUpdate(id: $id, input: { query: $query }) {
webhook { id subscriptionQuery }
errors { field message code }
}
}"""
def gql(query, variables=None):
r = requests.post(
API_URL,
json={"query": query, "variables": variables or {}},
headers={"Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json"},
timeout=30,
)
r.raise_for_status()
body = r.json()
if body.get("errors"):
raise RuntimeError(body["errors"])
return body["data"]
def extract_selection_fields(fragment_body):
"""Return the flat field names selected at the top level of one
braces-delimited selection set, ignoring nested sub-selections."""
depth = 0
fields = []
i = 0
n = len(fragment_body)
while i < n:
ch = fragment_body[i]
if ch == "{":
depth += 1
i += 1
continue
if ch == "}":
depth -= 1
i += 1
continue
if depth == 1:
m = FIELD_TOKEN.match(fragment_body, i)
if m and m.group(1) not in GRAPHQL_KEYWORDS:
fields.append(m.group(1))
i = m.end() - 1
continue
m2 = re.match(r"[A-Za-z_][A-Za-z0-9_]*", fragment_body[i:])
if m2:
token = m2.group(0)
nxt = fragment_body[i + len(token):].lstrip()
if not nxt.startswith("{") and token not in GRAPHQL_KEYWORDS:
fields.append(token)
i += len(token)
continue
i += 1
return sorted(set(fields))
def expected_fields_for(webhook, event_type):
subscription_query = webhook.get("subscriptionQuery")
if not subscription_query:
return LEGACY_SAMPLE_FIELDS.get(event_type, [])
marker = f"on {event_type.title().replace('_', '')}"
idx = subscription_query.find(f"on {event_type}")
body = subscription_query
if idx != -1:
body = subscription_query[idx:]
return extract_selection_fields(body)
def diff_payload_against_schema(payload, expected_fields, path=""):
missing_in_payload = []
unexpected_in_payload = []
if not isinstance(payload, dict):
return {
"missingInPayload": [f"{path}.{f}" if path else f for f in expected_fields],
"unexpectedInPayload": [],
}
expected_set = set(expected_fields)
payload_keys = set(payload.keys())
for field in expected_fields:
label = f"{path}.{field}" if path else field
if field not in payload or payload[field] is None:
missing_in_payload.append(label)
for key in payload_keys:
label = f"{path}.{key}" if path else key
if key not in expected_set:
unexpected_in_payload.append(label)
return {
"missingInPayload": sorted(missing_in_payload),
"unexpectedInPayload": sorted(unexpected_in_payload),
}
def list_webhooks():
data = gql(WEBHOOKS_QUERY)["webhooks"]
return [edge["node"] for edge in data["edges"]]
def recent_deliveries(webhook_id):
data = gql(DELIVERIES_QUERY, {"webhookId": webhook_id})["webhook"]
return [edge["node"] for edge in data["eventDeliveries"]["edges"]]
def apply_new_query(webhook_id, old_query, new_query):
log.info("Old query for %s:\n%s", webhook_id, old_query)
log.info("New query for %s:\n%s", webhook_id, new_query)
if DRY_RUN:
log.info("Dry run, not calling webhookUpdate.")
return
result = gql(WEBHOOK_UPDATE, {"id": webhook_id, "query": new_query})["webhookUpdate"]
if result["errors"]:
raise RuntimeError(result["errors"])
log.info("webhookUpdate applied for %s.", webhook_id)
def run():
reports = []
for webhook in list_webhooks():
deliveries = recent_deliveries(webhook["id"])
for delivery in deliveries[:5]:
try:
payload = json.loads(delivery["payload"])
except (TypeError, ValueError):
log.warning("Webhook %s delivery %s: payload did not parse as JSON.",
webhook["name"], delivery["id"])
continue
expected = expected_fields_for(webhook, delivery["eventType"])
if not expected:
continue
result = diff_payload_against_schema(payload, expected)
if result["missingInPayload"] or result["unexpectedInPayload"]:
report = {
"webhookId": webhook["id"],
"webhookName": webhook["name"],
"eventType": delivery["eventType"],
"sampleDeliveryId": delivery["id"],
**result,
}
reports.append(report)
log.warning(
"DRIFT webhook=%s event=%s missing=%s unexpected=%s delivery=%s",
webhook["name"], delivery["eventType"],
result["missingInPayload"], result["unexpectedInPayload"], delivery["id"],
)
if WEBHOOK_ID and NEW_SUBSCRIPTION_QUERY:
target = next((w for w in list_webhooks() if w["id"] == WEBHOOK_ID), None)
if target:
apply_new_query(WEBHOOK_ID, target.get("subscriptionQuery") or "", NEW_SUBSCRIPTION_QUERY)
log.info("Done. %d webhook delivery report(s) with drift.", len(reports))
return reports
if __name__ == "__main__":
run()/**
* Find Saleor webhooks whose delivered payload no longer matches the
* fields their own subscriptionQuery asks for.
*
* Saleor supports two incompatible webhook payload mechanisms: a legacy
* hard-coded shape documented with a sample payload per event, and a
* subscription-defined shape set by the query field on Webhook, which
* delivers exactly whatever that GraphQL query selects. There is no fixed
* schema for a subscription webhook. Drift shows up when the query goes
* stale after a Saleor field is renamed, deprecated, or moved behind a new
* type (saleor/saleor#8054, #9500, discussion #14194), including behavior
* changes like the 3.22 useLegacyUpdateWebhookEmission setting that altered
* whether metadata-only updates fire *_UPDATED events at all.
*
* This script never rewrites a subscription query on its own. Under
* DRY_RUN=true (the default) it only reports drift per webhook: missing
* fields, unexpected fields, and a sample delivery id. When DRY_RUN=false
* and NEW_SUBSCRIPTION_QUERY is set for a specific WEBHOOK_ID a human has
* reviewed, it prints the old versus new query and calls webhookUpdate.
* Run on a schedule.
*
* Guide: https://www.allanninal.dev/saleor/webhook-payload-diverges-from-schema/
*/
import { pathToFileURL } from "node:url";
const API_URL = process.env.SALEOR_API_URL || "https://store.saleor.cloud/graphql/";
const TOKEN = process.env.SALEOR_AUTH_TOKEN || "dummy-token";
const DRY_RUN = (process.env.DRY_RUN || "true").toLowerCase() === "true";
const WEBHOOK_ID = process.env.WEBHOOK_ID || "";
const NEW_SUBSCRIPTION_QUERY = process.env.NEW_SUBSCRIPTION_QUERY || "";
const GRAPHQL_KEYWORDS = new Set(["query", "mutation", "subscription", "fragment", "on"]);
// Documented sample fields for legacy (non-subscription) webhooks, per event
// type. Extend this as needed for events you run as legacy webhooks.
const LEGACY_SAMPLE_FIELDS = {
PRODUCT_UPDATED: ["id", "name", "slug", "category"],
ORDER_CREATED: ["id", "number", "status", "userEmail", "total"],
};
export function extractSelectionFields(fragmentBody) {
// Flat field names selected at the top level of one braces-delimited
// selection set, ignoring nested sub-selections.
let depth = 0;
const fields = new Set();
let i = 0;
const n = fragmentBody.length;
while (i < n) {
const ch = fragmentBody[i];
if (ch === "{") { depth++; i++; continue; }
if (ch === "}") { depth--; i++; continue; }
if (depth === 1) {
const m = /^([A-Za-z_][A-Za-z0-9_]*)\s*(?:\([^)]*\))?\s*\{/.exec(fragmentBody.slice(i));
if (m && !GRAPHQL_KEYWORDS.has(m[1])) {
fields.add(m[1]);
i += m[0].length - 1;
continue;
}
const m2 = /^[A-Za-z_][A-Za-z0-9_]*/.exec(fragmentBody.slice(i));
if (m2) {
const token = m2[0];
const rest = fragmentBody.slice(i + token.length).trimStart();
if (!rest.startsWith("{") && !GRAPHQL_KEYWORDS.has(token)) {
fields.add(token);
}
i += token.length;
continue;
}
}
i++;
}
return Array.from(fields).sort();
}
export function expectedFieldsFor(webhook, eventType) {
const subscriptionQuery = webhook.subscriptionQuery;
if (!subscriptionQuery) return LEGACY_SAMPLE_FIELDS[eventType] || [];
const idx = subscriptionQuery.indexOf(`on ${eventType}`);
const body = idx !== -1 ? subscriptionQuery.slice(idx) : subscriptionQuery;
return extractSelectionFields(body);
}
export function diffPayloadAgainstSchema(payload, expectedFields, options = {}) {
const path = options.path || "";
const label = (name) => (path ? `${path}.${name}` : name);
if (typeof payload !== "object" || payload === null || Array.isArray(payload)) {
return {
missingInPayload: expectedFields.map(label).sort(),
unexpectedInPayload: [],
};
}
const expectedSet = new Set(expectedFields);
const payloadKeys = Object.keys(payload);
const missingInPayload = expectedFields
.filter((field) => !(field in payload) || payload[field] === null)
.map(label)
.sort();
const unexpectedInPayload = payloadKeys
.filter((key) => !expectedSet.has(key))
.map(label)
.sort();
return { missingInPayload, unexpectedInPayload };
}
async function gql(query, variables = {}) {
const res = await fetch(API_URL, {
method: "POST",
headers: { Authorization: `Bearer ${TOKEN}`, "Content-Type": "application/json" },
body: JSON.stringify({ query, variables }),
});
if (!res.ok) throw new Error(`Saleor ${res.status}`);
const body = await res.json();
if (body.errors) throw new Error(JSON.stringify(body.errors));
return body.data;
}
const WEBHOOKS_QUERY = `
query {
webhooks(first: 100) {
edges {
node { id name targetUrl isActive subscriptionQuery events }
}
}
}`;
const DELIVERIES_QUERY = `
query($webhookId: ID!) {
webhook(id: $webhookId) {
eventDeliveries(first: 50, sortBy: { field: CREATED_AT, direction: DESC }) {
edges {
node {
id eventType payload createdAt
attempts(first: 1) { edges { node { responseStatusCode response } } }
}
}
}
}
}`;
const WEBHOOK_UPDATE = `
mutation($id: ID!, $query: String!) {
webhookUpdate(id: $id, input: { query: $query }) {
webhook { id subscriptionQuery }
errors { field message code }
}
}`;
async function listWebhooks() {
const data = (await gql(WEBHOOKS_QUERY)).webhooks;
return data.edges.map((edge) => edge.node);
}
async function recentDeliveries(webhookId) {
const data = (await gql(DELIVERIES_QUERY, { webhookId })).webhook;
return data.eventDeliveries.edges.map((edge) => edge.node);
}
async function applyNewQuery(webhookId, oldQuery, newQuery) {
console.log(`Old query for ${webhookId}:\n${oldQuery}`);
console.log(`New query for ${webhookId}:\n${newQuery}`);
if (DRY_RUN) {
console.log("Dry run, not calling webhookUpdate.");
return;
}
const result = (await gql(WEBHOOK_UPDATE, { id: webhookId, query: newQuery })).webhookUpdate;
if (result.errors.length) throw new Error(JSON.stringify(result.errors));
console.log(`webhookUpdate applied for ${webhookId}.`);
}
export async function run() {
const reports = [];
const webhooks = await listWebhooks();
for (const webhook of webhooks) {
const deliveries = await recentDeliveries(webhook.id);
for (const delivery of deliveries.slice(0, 5)) {
let payload;
try {
payload = JSON.parse(delivery.payload);
} catch {
console.warn(`Webhook ${webhook.name} delivery ${delivery.id}: payload did not parse as JSON.`);
continue;
}
const expected = expectedFieldsFor(webhook, delivery.eventType);
if (!expected.length) continue;
const result = diffPayloadAgainstSchema(payload, expected);
if (result.missingInPayload.length || result.unexpectedInPayload.length) {
reports.push({
webhookId: webhook.id,
webhookName: webhook.name,
eventType: delivery.eventType,
sampleDeliveryId: delivery.id,
...result,
});
console.warn(
`DRIFT webhook=${webhook.name} event=${delivery.eventType} missing=${JSON.stringify(result.missingInPayload)} unexpected=${JSON.stringify(result.unexpectedInPayload)} delivery=${delivery.id}`
);
}
}
}
if (WEBHOOK_ID && NEW_SUBSCRIPTION_QUERY) {
const target = webhooks.find((w) => w.id === WEBHOOK_ID);
if (target) {
await applyNewQuery(WEBHOOK_ID, target.subscriptionQuery || "", NEW_SUBSCRIPTION_QUERY);
}
}
console.log(`Done. ${reports.length} webhook delivery report(s) with drift.`);
return reports;
}
if (import.meta.url === pathToFileURL(process.argv[1]).href) {
run().catch((err) => { console.error(err); process.exit(1); });
}Add a test
The decision rule is the part most worth testing, because it decides what counts as drift. Because diff_payload_against_schema is pure, the test needs no network and no Saleor account. It just feeds in a payload and a field list, then checks the answer against fixtures for a renamed field, a deprecated field returning null, and an extra field a newer Saleor version added.
from detect_webhook_payload_drift import diff_payload_against_schema, extract_selection_fields
def test_no_drift_when_payload_matches_expected_fields():
payload = {"id": "gid://saleor/Product/1", "name": "Mug", "slug": "mug"}
result = diff_payload_against_schema(payload, ["id", "name", "slug"])
assert result == {"missingInPayload": [], "unexpectedInPayload": []}
def test_detects_renamed_field_as_missing():
# query expects "category", payload was produced by an older query that
# only ever had "categoryId", simulating a field rename drift
payload = {"id": "1", "name": "Mug", "categoryId": "9"}
result = diff_payload_against_schema(payload, ["id", "name", "category"])
assert result["missingInPayload"] == ["category"]
assert result["unexpectedInPayload"] == ["categoryId"]
def test_detects_deprecated_field_returning_null():
payload = {"id": "1", "name": "Mug", "chargeTaxes": None}
result = diff_payload_against_schema(payload, ["id", "name", "chargeTaxes"])
assert result["missingInPayload"] == ["chargeTaxes"]
assert result["unexpectedInPayload"] == []
def test_detects_extra_field_from_newer_saleor_version():
payload = {"id": "1", "name": "Mug", "slug": "mug", "externalReference": "ext-1"}
result = diff_payload_against_schema(payload, ["id", "name", "slug"])
assert result["missingInPayload"] == []
assert result["unexpectedInPayload"] == ["externalReference"]
def test_non_dict_payload_flags_all_fields_missing():
result = diff_payload_against_schema(None, ["id", "name"])
assert result == {"missingInPayload": ["id", "name"], "unexpectedInPayload": []}
def test_nested_path_is_used_in_labels():
payload = {"id": "1"}
result = diff_payload_against_schema(payload, ["id", "name"], path="product")
assert result["missingInPayload"] == ["product.name"]
assert result["unexpectedInPayload"] == []
def test_extract_selection_fields_reads_top_level_only():
fragment = "on ProductUpdated { product { id name category { id } } }"
assert extract_selection_fields(fragment) == ["product"]import { test } from "node:test";
import assert from "node:assert/strict";
import { diffPayloadAgainstSchema, extractSelectionFields } from "./detect-webhook-payload-drift.js";
test("no drift when payload matches expected fields", () => {
const payload = { id: "gid://saleor/Product/1", name: "Mug", slug: "mug" };
const result = diffPayloadAgainstSchema(payload, ["id", "name", "slug"]);
assert.deepEqual(result, { missingInPayload: [], unexpectedInPayload: [] });
});
test("detects renamed field as missing", () => {
const payload = { id: "1", name: "Mug", categoryId: "9" };
const result = diffPayloadAgainstSchema(payload, ["id", "name", "category"]);
assert.deepEqual(result.missingInPayload, ["category"]);
assert.deepEqual(result.unexpectedInPayload, ["categoryId"]);
});
test("detects deprecated field returning null", () => {
const payload = { id: "1", name: "Mug", chargeTaxes: null };
const result = diffPayloadAgainstSchema(payload, ["id", "name", "chargeTaxes"]);
assert.deepEqual(result.missingInPayload, ["chargeTaxes"]);
assert.deepEqual(result.unexpectedInPayload, []);
});
test("detects extra field from newer Saleor version", () => {
const payload = { id: "1", name: "Mug", slug: "mug", externalReference: "ext-1" };
const result = diffPayloadAgainstSchema(payload, ["id", "name", "slug"]);
assert.deepEqual(result.missingInPayload, []);
assert.deepEqual(result.unexpectedInPayload, ["externalReference"]);
});
test("non-object payload flags all fields missing", () => {
const result = diffPayloadAgainstSchema(null, ["id", "name"]);
assert.deepEqual(result, { missingInPayload: ["id", "name"], unexpectedInPayload: [] });
});
test("nested path is used in labels", () => {
const payload = { id: "1" };
const result = diffPayloadAgainstSchema(payload, ["id", "name"], { path: "product" });
assert.deepEqual(result.missingInPayload, ["product.name"]);
assert.deepEqual(result.unexpectedInPayload, []);
});
test("extractSelectionFields reads top level only", () => {
const fragment = "on ProductUpdated { product { id name category { id } } }";
assert.deepEqual(extractSelectionFields(fragment), ["product"]);
});Case studies
A pricing sync quietly stopped reading discount data
A pricing sync app subscribed to PRODUCT_VARIANT_UPDATED with a query written against an early Saleor release. A later release moved a discount-related field behind a new type, and the app's query still referenced the old shape. The webhook kept firing, kept returning 200 responses, and the sync silently stopped updating discount data for weeks, because the field it needed was simply absent from every delivery.
Running the drift report against a week of stored deliveries flagged the exact missing field on every PRODUCT_VARIANT_UPDATED delivery, with a sample delivery id the team could open and inspect by hand before touching the query.
A new hire debugged the wrong schema for two days
A developer new to a Saleor integration built a payload validator against the sample ORDER_CREATED payload shown on the legacy webhooks documentation page, not realizing the production webhook was subscription-based with a much narrower query. Every real delivery failed validation, and two days went into suspecting a Saleor bug that did not exist.
Pulling the webhook's actual subscriptionQuery and running the diff against that, instead of the legacy sample, showed the delivery matched its own query perfectly. The lesson stuck: a subscription webhook's schema is its query, not the docs.
After this runs on a schedule, a payload shift caused by a schema change, a deprecation, or a stale query gets caught as a clear report, exact missing and unexpected fields, a real delivery id to inspect, instead of surfacing weeks later as silently missing data downstream. The subscription query itself only ever changes when a human reads the diff, decides it is correct, and applies it deliberately through webhookUpdate.
FAQ
Why does my Saleor webhook payload not match the sample in the docs?
The documented sample payload only applies to legacy, hard-coded webhooks. A webhook created with a subscription query on the Webhook query field delivers whatever fields that specific query selects, not a fixed schema. If the query is stale after a field was renamed, deprecated, or moved behind a new type, or if the app compares a subscription delivery against the generic legacy sample, the shapes will not line up and Saleor never flags it.
Does Saleor validate a webhook payload against its own schema before sending it?
No. Saleor builds the payload by executing the webhook's stored subscription query at delivery time and sends whatever comes back. There is no check that the resulting JSON still matches an older sample, a changelog note, or the app's own expectations, so a schema change, a deprecated field returning null, or a dashboard-created webhook with a different query silently changes the delivered shape with no error surfaced anywhere.
Is it safe to auto-fix a subscription query when a payload drifts?
No, rewriting a subscription query automatically is not safe, because the app's own webhook handler code depends on the exact fields it currently requests. The safe pattern is to detect and report the drift, missing fields, unexpected fields, and a sample delivery id, and only apply a corrected query through webhookUpdate after a human reviews the diff, always checked first with a dry run.
Related field notes
Citations
On the problem:
- Webhooks payloads do not match documentation. github.com/saleor/saleor/issues/8054
- Customizable webhook payloads. github.com/saleor/saleor/issues/9500
- Webhooks & Subscriptions discussion. github.com/saleor/saleor/discussions/14194
On the solution:
- Saleor Commerce Documentation: Subscription Webhook Payloads. docs.saleor.io/developer/extending/webhooks/subscription-webhook-payloads
- Saleor Commerce Documentation: How to Update App Webhooks. docs.saleor.io/developer/extending/apps/updating-app-webhooks
- Saleor Commerce Documentation: Webhooks Troubleshooting. docs.saleor.io/developer/extending/webhooks/troubleshooting
Stuck on a tricky one?
If you have a problem in Saleor checkout, stock, channels, or fulfillment that you would rather hand off, this is the kind of work I do. Message me and we can work through it together.
Did this catch a payload drift for you?
If this saved you from chasing a bug that was really just a stale subscription query, you can buy me a coffee. It is the best way to keep these field notes free and growing.
Buy me a coffee on Ko-fi