Reconciler Workflows & Background Jobs
continueOnPermanentFailure skips compensation and leaves partial state
A step is deliberately flagged so a permanent failure will not stop the whole workflow. That flag also quietly turns off its own rollback. If that step already committed something, an order, a captured payment, a reservation, and it later fails, the workflow keeps going and nothing ever undoes what that step did. If a sibling step then fails too and the saga rolls back, it still will not touch the flagged step, so the record sits there in the database with no matching downstream state ever created. Here is why that happens and a small reconciler that finds the gap and reports it safely.
Medusa v2 workflows are sagas. When a step throws, the orchestrator walks backward through the steps that already completed and calls each one's compensation function to undo it. Setting .config({ continueOnPermanentFailure: true }) on a step opts that step out of the contract entirely: per Medusa's docs, its compensation function will not be called, and the workflow keeps running subsequent steps as if the failure never happened. If that step had already committed a side effect, and a later step then fails and triggers a rollback, the orchestrator still does not retroactively undo the flagged step's work, so the order, payment, or reservation it created survives with no fulfillment, capture, or completion ever following it. Run a script that lists recent orders through the Admin API, compares payment_status against fulfillment_status, and classifies the gap with a pure function so a human can triage it. Full code, tests, and a dry run guard are below.
The problem in plain words
A Medusa workflow is a saga. Every step that succeeds is tracked, and if a later step throws, Medusa walks backward through the steps that already ran and calls each one's compensation function to put the world back the way it was. That is the whole point of the pattern: a workflow either fully commits or fully undoes itself.
.config({ continueOnPermanentFailure: true }) is an escape hatch from that contract. It tells Medusa that if this specific step permanently fails, do not stop the workflow, just carry on to the next step. The tradeoff, stated plainly in Medusa's own documentation, is that the compensation function of the step carrying that configuration will not be called. So if the step already did something before it reported the permanent failure, for example it created the order record first and then failed on a follow-up call, that something is never undone. The workflow finishes its remaining steps thinking everything is fine, while one piece of committed state has no corresponding downstream work behind it.
Why it happens
continueOnPermanentFailure exists to let a workflow keep making progress when one step's failure should not block everything else. But that convenience comes with the compensation skip built in, and a few patterns make it bite in production:
- A step is flagged
continueOnPermanentFailurespecifically because its side effect, such as a payment capture, is not cheaply reversible, but the step still commits work before it fails, and that work is now permanently unaccounted for. - A later step in the same run fails and triggers the saga's rollback. Store owners reasonably expect that rollback to undo everything that ran before it, but the orchestrator only compensates steps that both completed and have compensation registered, so the flagged step is walked past, not reversed.
- The distinction between
continueOnPermanentFailureand the newerskipOnPermanentFailurewas flaky enough in practice that Medusa shipped PR #12027, "feat(orchestration): skip on permanent failure," specifically to separate the two behaviors. - Real orphaned-state bugs of this shape have already been filed against related workflow paths, for example issue #11266 where fulfillment does not always delete the inventory reservation it should, leaving a reservation with nothing downstream to justify it.
The net effect is the same every time. A step's committed side effect is left in the database, a workflow that ran cleanly through the rest of its steps has one gap in the middle, and nothing in Medusa flags that gap on its own. See the citations at the end for the exact docs, PR, and issue.
Compensation was intentionally disabled for that step, so automatically reversing its side effect is not safe. The workflow author chose continueOnPermanentFailure specifically because they did not want automatic rollback there, often because the side effect, a payment capture especially, cannot be undone for free. So the right default is not "reverse it," it is "detect the gap and hand it to a person." Only when the orphan is a clearly safe, idempotent cleanup, such as a dangling reservation with no live order line, should a script make a guarded write, and even then only one at a time with logging.
The fix, as a flow
We do not touch the live workflow or attempt to reverse anything a human has not reviewed. The job lists recent orders with their payment and fulfillment state expanded, classifies each one with a pure decision function, and reports every orphan as a structured record. Only a dangling reservation with no live order line is ever deleted, and only with the dry run guard off.
Build it step by step
Authenticate against the Admin API
Exchange the admin email and password for a JWT at /auth/user/emailpass, then send it as a Bearer token on every /admin/* call. Keep the backend URL and credentials in environment variables, never in the file.
pip install requests
export MEDUSA_BACKEND_URL="http://localhost:9000"
export MEDUSA_ADMIN_EMAIL="admin@example.com"
export MEDUSA_ADMIN_PASSWORD="supersecret"
export SINCE_HOURS="24"
export DRY_RUN="true" # start safe, change to false to write// Node 18+ has fetch built in, no dependencies needed
export MEDUSA_BACKEND_URL="http://localhost:9000"
export MEDUSA_ADMIN_EMAIL="admin@example.com"
export MEDUSA_ADMIN_PASSWORD="supersecret"
export SINCE_HOURS="24"
export DRY_RUN="true" // start safe, change to false to writeList recent orders with payment and fulfillment expanded
Ask for orders created since a cutoff, with fields expanding payments and fulfillments so the decision function can see both sides of the gap. This is the same shape the research calls out: payment_status captured or authorized next to a fulfillment_status stuck at not fulfilled with no fulfillments[] at all is the signature of a skipped compensation.
import os, requests
BACKEND_URL = os.environ["MEDUSA_BACKEND_URL"]
def admin_get(token, path, params=None):
r = requests.get(
f"{BACKEND_URL}{path}",
headers={"Authorization": f"Bearer {token}"},
params=params or {},
timeout=30,
)
r.raise_for_status()
return r.json()
def list_recent_orders(token, since_iso):
orders = []
offset = 0
limit = 100
while True:
data = admin_get(token, "/admin/orders", {
"fields": "id,status,fulfillment_status,payment_status,*payments,*fulfillments,*items",
"created_at[$gte]": since_iso,
"limit": limit,
"offset": offset,
})
orders.extend(data["orders"])
offset += limit
if offset >= data["count"]:
return ordersasync function adminGet(token, path, params = {}) {
const url = new URL(`${process.env.MEDUSA_BACKEND_URL}${path}`);
for (const [key, value] of Object.entries(params)) {
url.searchParams.set(key, value);
}
const res = await fetch(url, { headers: { Authorization: `Bearer ${token}` } });
if (!res.ok) throw new Error(`Medusa ${res.status} on GET ${path}`);
return res.json();
}
async function listRecentOrders(token, sinceIso) {
const orders = [];
let offset = 0;
const limit = 100;
while (true) {
const data = await adminGet(token, "/admin/orders", {
fields: "id,status,fulfillment_status,payment_status,*payments,*fulfillments,*items",
"created_at[$gte]": sinceIso,
limit,
offset,
});
orders.push(...data.orders);
offset += limit;
if (offset >= data.count) return orders;
}
}Run the workflow with throwOnError false to read its own error trail
When you have the transaction in hand, call the workflow with { throwOnError: false } and inspect errors. Each entry is { action, handlerType, error }, where action is the failing step id. This is how you confirm a step was flagged continueOnPermanentFailure and failed without compensating, before you ever look at order state.
# Node-side workflow inspection, called from your Medusa backend, not this script:
#
# const { result, errors } = await someWorkflow(container).run({ input, throwOnError: false })
#
# errors is a list of { action, handlerType, error } dicts. handlerType is
# "invoke" or "compensate". A failing "invoke" action whose step is flagged
# continueOnPermanentFailure and that carries no matching "compensate" entry
# is exactly the gap this reconciler looks for on the order side.
def failed_steps_from_errors(errors):
"""errors: list of {"action": str, "handlerType": str, "error": {"message": str}}"""
return [
{"action": e["action"], "handlerType": e["handlerType"], "message": e["error"]["message"]}
for e in errors
]// Node-side workflow inspection, called from your Medusa backend, not this script:
//
// const { result, errors } = await someWorkflow(container).run({ input, throwOnError: false })
//
// errors is a list of { action, handlerType, error } entries. handlerType is
// "invoke" or "compensate". A failing "invoke" action whose step is flagged
// continueOnPermanentFailure and that carries no matching "compensate" entry
// is exactly the gap this reconciler looks for on the order side.
function failedStepsFromErrors(errors) {
return errors.map((e) => ({ action: e.action, handlerType: e.handlerType, message: e.error.message }));
}Decide, with one pure function
Keep the classification in its own function that takes an order snapshot and the workflow's reported failed-step list, and returns a plain string. It never touches the network, so it is trivial to unit test with fixture data. The rule looks for a captured or authorized payment next to an empty fulfillment list, or a reservation whose order line no longer exists, combined with a continueOnPermanentFailure-flagged failed step in the trail.
CAPTURED_STATUSES = {"captured", "authorized"}
STUCK_FULFILLMENT_STATUSES = {"not_fulfilled"}
def classify_orphan(order, failed_steps):
"""Pure decision function. No I/O.
order: {"id": str, "payment_status": str, "fulfillment_status": str,
"payments": [{"status": str}], "fulfillments": [any]}
failed_steps: list of {"action": str, "handlerType": "invoke" | "compensate"}
Returns "orphaned_payment_no_fulfillment" | "orphaned_reservation_no_order_line" | "ok".
"""
has_continue_on_failure = any(
s.get("handlerType") == "invoke" and "continueOnPermanentFailure" in s.get("action", "")
for s in failed_steps
)
payment_committed = order.get("payment_status") in CAPTURED_STATUSES and bool(order.get("payments"))
fulfillment_missing = (
order.get("fulfillment_status") in STUCK_FULFILLMENT_STATUSES
and not order.get("fulfillments")
)
if payment_committed and fulfillment_missing and has_continue_on_failure:
return "orphaned_payment_no_fulfillment"
has_dangling_reservation = any(
s.get("action") == "reserveInventoryStep" and s.get("handlerType") == "invoke"
for s in failed_steps
)
if has_dangling_reservation and not order.get("items"):
return "orphaned_reservation_no_order_line"
return "ok"const CAPTURED_STATUSES = new Set(["captured", "authorized"]);
const STUCK_FULFILLMENT_STATUSES = new Set(["not_fulfilled"]);
/**
* Pure decision function. No I/O.
*
* @param {{ id: string, payment_status: string, fulfillment_status: string, payments: {status:string}[], fulfillments: unknown[] }} order
* @param {{ action: string, handlerType: "invoke" | "compensate" }[]} failedSteps
* @returns {"orphaned_payment_no_fulfillment" | "orphaned_reservation_no_order_line" | "ok"}
*/
export function classifyOrphan(order, failedSteps) {
const hasContinueOnFailure = failedSteps.some(
(s) => s.handlerType === "invoke" && (s.action || "").includes("continueOnPermanentFailure")
);
const paymentCommitted = CAPTURED_STATUSES.has(order.payment_status) && (order.payments || []).length > 0;
const fulfillmentMissing =
STUCK_FULFILLMENT_STATUSES.has(order.fulfillment_status) && (order.fulfillments || []).length === 0;
if (paymentCommitted && fulfillmentMissing && hasContinueOnFailure) {
return "orphaned_payment_no_fulfillment";
}
const hasDanglingReservation = failedSteps.some(
(s) => s.action === "reserveInventoryStep" && s.handlerType === "invoke"
);
if (hasDanglingReservation && (order.items || []).length === 0) {
return "orphaned_reservation_no_order_line";
}
return "ok";
}Report every orphan, only clean up the dangling reservation
When an order classifies as orphaned_payment_no_fulfillment, emit a structured record: order_id, the failed action step id, error.message, and a timestamp, for a human to triage. That is the default outcome, because the payment side effect is rarely safe to reverse automatically. Only orphaned_reservation_no_order_line, a clearly safe and idempotent cleanup, gets a guarded corrective write.
DELETABLE = {"orphaned_reservation_no_order_line"}
def admin_delete(token, path):
r = requests.delete(
f"{BACKEND_URL}{path}",
headers={"Authorization": f"Bearer {token}"},
timeout=30,
)
r.raise_for_status()
return r.json()const DELETABLE = new Set(["orphaned_reservation_no_order_line"]);
async function adminDelete(token, path) {
const res = await fetch(`${process.env.MEDUSA_BACKEND_URL}${path}`, {
method: "DELETE",
headers: { Authorization: `Bearer ${token}` },
});
if (!res.ok) throw new Error(`Medusa ${res.status} on DELETE ${path}`);
return res.json();
}Wire it together with a dry run guard
The loop ties every piece together. Notice the dry run guard. On the first few runs, leave DRY_RUN on so the script only logs the order id and classification it would report, and the reservation delete calls it would make. Read the output, agree with it, then switch it off to let it write, one delete at a time, logging each response before moving on. Run it on a schedule, for example every hour, so a skipped compensation never sits unnoticed for long.
Always start with DRY_RUN=true. Compensation was intentionally disabled for the flagged step, so this script never tries to auto-reverse a payment capture or an order. It reports the structured record and stops there. The only guarded write it ever makes is deleting a dangling reservation that has no live order line, and even that runs one delete at a time with the response logged before the next call.
The full code
Here is the complete script in one file for each language. It reads settings from the environment, logs what it does, respects the dry run flag, and is safe to run again and again because it only ever writes to the one clearly safe cleanup case.
"""Reconcile Medusa orders left partial by a continueOnPermanentFailure step.
.config({ continueOnPermanentFailure: true }) opts a step out of the saga's rollback
contract. Per Medusa's docs, the compensation function of the flagged step will not
be called, and the workflow keeps running subsequent steps as if nothing happened.
If that step already committed a side effect, an order, a captured payment, or a
reservation, and a later step then fails and triggers a rollback, the orchestrator
still does not retroactively undo the flagged step's work (PR #12027, issue #11266).
This lists recent orders with payments and fulfillments expanded, classifies each
one with a pure function, and reports every orphan as a structured record for a
human to triage. The only guarded write is deleting a dangling reservation that has
no live order line. Run on a schedule. Safe to run again and again.
"""
import os
import json
import logging
import datetime
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("reconcile_skipped_compensation")
BACKEND_URL = os.environ["MEDUSA_BACKEND_URL"]
ADMIN_EMAIL = os.environ["MEDUSA_ADMIN_EMAIL"]
ADMIN_PASSWORD = os.environ["MEDUSA_ADMIN_PASSWORD"]
SINCE_HOURS = float(os.environ.get("SINCE_HOURS", "24"))
DRY_RUN = os.environ.get("DRY_RUN", "true").lower() == "true"
CAPTURED_STATUSES = {"captured", "authorized"}
STUCK_FULFILLMENT_STATUSES = {"not_fulfilled"}
DELETABLE = {"orphaned_reservation_no_order_line"}
def get_admin_token():
r = requests.post(
f"{BACKEND_URL}/auth/user/emailpass",
json={"email": ADMIN_EMAIL, "password": ADMIN_PASSWORD},
timeout=30,
)
r.raise_for_status()
return r.json()["token"]
def admin_get(token, path, params=None):
r = requests.get(
f"{BACKEND_URL}{path}",
headers={"Authorization": f"Bearer {token}"},
params=params or {},
timeout=30,
)
r.raise_for_status()
return r.json()
def admin_delete(token, path):
r = requests.delete(
f"{BACKEND_URL}{path}",
headers={"Authorization": f"Bearer {token}"},
timeout=30,
)
r.raise_for_status()
return r.json()
def classify_orphan(order, failed_steps):
"""Pure decision function. No I/O.
order: {"id": str, "payment_status": str, "fulfillment_status": str,
"payments": [{"status": str}], "fulfillments": [any], "items": [any]}
failed_steps: list of {"action": str, "handlerType": "invoke" | "compensate"}
Returns "orphaned_payment_no_fulfillment" | "orphaned_reservation_no_order_line" | "ok".
"""
has_continue_on_failure = any(
s.get("handlerType") == "invoke" and "continueOnPermanentFailure" in s.get("action", "")
for s in failed_steps
)
payment_committed = order.get("payment_status") in CAPTURED_STATUSES and bool(order.get("payments"))
fulfillment_missing = (
order.get("fulfillment_status") in STUCK_FULFILLMENT_STATUSES
and not order.get("fulfillments")
)
if payment_committed and fulfillment_missing and has_continue_on_failure:
return "orphaned_payment_no_fulfillment"
has_dangling_reservation = any(
s.get("action") == "reserveInventoryStep" and s.get("handlerType") == "invoke"
for s in failed_steps
)
if has_dangling_reservation and not order.get("items"):
return "orphaned_reservation_no_order_line"
return "ok"
def list_recent_orders(token, since_iso):
orders = []
offset = 0
limit = 100
while True:
data = admin_get(token, "/admin/orders", {
"fields": "id,status,fulfillment_status,payment_status,*payments,*fulfillments,*items",
"created_at[$gte]": since_iso,
"limit": limit,
"offset": offset,
})
orders.extend(data["orders"])
offset += limit
if offset >= data["count"]:
return orders
def failed_steps_for_order(order):
"""Placeholder hook: in a real deployment, load the failed-step trail for this
order's workflow transaction (for example from your own audit log of
{ result, errors } from someWorkflow(container).run({ input, throwOnError: false })).
Returns [] when there is nothing on file, which classify_orphan treats as "ok".
"""
return order.get("_failed_steps", [])
def run():
token = get_admin_token()
since_iso = (
datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(hours=SINCE_HOURS)
).isoformat()
orders = list_recent_orders(token, since_iso)
reported = 0
cleaned = 0
for order in orders:
failed_steps = failed_steps_for_order(order)
outcome = classify_orphan(order, failed_steps)
if outcome == "ok":
continue
if outcome in DELETABLE:
reservation_id = order.get("_dangling_reservation_id")
log.warning(
"Order %s classified as %s. reservation_id=%s. %s",
order["id"], outcome, reservation_id, "Would delete" if DRY_RUN else "Deleting",
)
if not DRY_RUN and reservation_id:
admin_delete(token, f"/admin/reservations/{reservation_id}")
cleaned += 1
else:
record = {
"order_id": order["id"],
"action": next((s["action"] for s in failed_steps if s.get("handlerType") == "invoke"), None),
"error_message": next((s.get("message") for s in failed_steps), None),
"classification": outcome,
"reported_at": datetime.datetime.now(datetime.timezone.utc).isoformat(),
}
log.info("Orphan detected: %s", json.dumps(record))
reported += 1
log.info(
"Done. %d order(s) reported for human triage, %d reservation(s) %s.",
reported, cleaned, "to delete" if DRY_RUN else "deleted",
)
if __name__ == "__main__":
run()/**
* Reconcile Medusa orders left partial by a continueOnPermanentFailure step.
*
* .config({ continueOnPermanentFailure: true }) opts a step out of the saga's rollback
* contract. Per Medusa's docs, the compensation function of the flagged step will not
* be called, and the workflow keeps running subsequent steps as if nothing happened.
* If that step already committed a side effect, an order, a captured payment, or a
* reservation, and a later step then fails and triggers a rollback, the orchestrator
* still does not retroactively undo the flagged step's work (PR #12027, issue #11266).
*
* This lists recent orders with payments and fulfillments expanded, classifies each
* one with a pure function, and reports every orphan as a structured record for a
* human to triage. The only guarded write is deleting a dangling reservation that has
* no live order line. Run on a schedule. Safe to run again and again.
*
* Guide: https://www.allanninal.dev/medusa/continue-on-failure-skips-compensation/
*/
import { pathToFileURL } from "node:url";
const BACKEND_URL = process.env.MEDUSA_BACKEND_URL || "http://localhost:9000";
const ADMIN_EMAIL = process.env.MEDUSA_ADMIN_EMAIL || "admin@example.com";
const ADMIN_PASSWORD = process.env.MEDUSA_ADMIN_PASSWORD || "supersecret";
const SINCE_HOURS = Number(process.env.SINCE_HOURS || 24);
const DRY_RUN = (process.env.DRY_RUN || "true").toLowerCase() === "true";
const CAPTURED_STATUSES = new Set(["captured", "authorized"]);
const STUCK_FULFILLMENT_STATUSES = new Set(["not_fulfilled"]);
const DELETABLE = new Set(["orphaned_reservation_no_order_line"]);
/**
* Pure decision function. No I/O.
*
* @param {{ id: string, payment_status: string, fulfillment_status: string, payments: {status:string}[], fulfillments: unknown[], items?: unknown[] }} order
* @param {{ action: string, handlerType: "invoke" | "compensate" }[]} failedSteps
* @returns {"orphaned_payment_no_fulfillment" | "orphaned_reservation_no_order_line" | "ok"}
*/
export function classifyOrphan(order, failedSteps) {
const hasContinueOnFailure = failedSteps.some(
(s) => s.handlerType === "invoke" && (s.action || "").includes("continueOnPermanentFailure")
);
const paymentCommitted = CAPTURED_STATUSES.has(order.payment_status) && (order.payments || []).length > 0;
const fulfillmentMissing =
STUCK_FULFILLMENT_STATUSES.has(order.fulfillment_status) && (order.fulfillments || []).length === 0;
if (paymentCommitted && fulfillmentMissing && hasContinueOnFailure) {
return "orphaned_payment_no_fulfillment";
}
const hasDanglingReservation = failedSteps.some(
(s) => s.action === "reserveInventoryStep" && s.handlerType === "invoke"
);
if (hasDanglingReservation && (order.items || []).length === 0) {
return "orphaned_reservation_no_order_line";
}
return "ok";
}
async function getAdminToken() {
const res = await fetch(`${BACKEND_URL}/auth/user/emailpass`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ email: ADMIN_EMAIL, password: ADMIN_PASSWORD }),
});
if (!res.ok) throw new Error(`Medusa auth ${res.status}`);
const body = await res.json();
return body.token;
}
async function adminGet(token, path, params = {}) {
const url = new URL(`${BACKEND_URL}${path}`);
for (const [key, value] of Object.entries(params)) {
url.searchParams.set(key, value);
}
const res = await fetch(url, { headers: { Authorization: `Bearer ${token}` } });
if (!res.ok) throw new Error(`Medusa ${res.status} on GET ${path}`);
return res.json();
}
async function adminDelete(token, path) {
const res = await fetch(`${BACKEND_URL}${path}`, {
method: "DELETE",
headers: { Authorization: `Bearer ${token}` },
});
if (!res.ok) throw new Error(`Medusa ${res.status} on DELETE ${path}`);
return res.json();
}
async function listRecentOrders(token, sinceIso) {
const orders = [];
let offset = 0;
const limit = 100;
while (true) {
const data = await adminGet(token, "/admin/orders", {
fields: "id,status,fulfillment_status,payment_status,*payments,*fulfillments,*items",
"created_at[$gte]": sinceIso,
limit,
offset,
});
orders.push(...data.orders);
offset += limit;
if (offset >= data.count) return orders;
}
}
/**
* Placeholder hook: in a real deployment, load the failed-step trail for this
* order's workflow transaction (for example from your own audit log of
* { result, errors } from someWorkflow(container).run({ input, throwOnError: false })).
* Returns [] when there is nothing on file, which classifyOrphan treats as "ok".
*/
function failedStepsForOrder(order) {
return order._failed_steps || [];
}
export async function run() {
const token = await getAdminToken();
const sinceIso = new Date(Date.now() - SINCE_HOURS * 3600 * 1000).toISOString();
const orders = await listRecentOrders(token, sinceIso);
let reported = 0;
let cleaned = 0;
for (const order of orders) {
const failedSteps = failedStepsForOrder(order);
const outcome = classifyOrphan(order, failedSteps);
if (outcome === "ok") continue;
if (DELETABLE.has(outcome)) {
const reservationId = order._dangling_reservation_id;
console.warn(
`Order ${order.id} classified as ${outcome}. reservation_id=${reservationId}. ${DRY_RUN ? "Would delete" : "Deleting"}`
);
if (!DRY_RUN && reservationId) await adminDelete(token, `/admin/reservations/${reservationId}`);
cleaned++;
} else {
const record = {
order_id: order.id,
action: failedSteps.find((s) => s.handlerType === "invoke")?.action ?? null,
error_message: failedSteps[0]?.message ?? null,
classification: outcome,
reported_at: new Date().toISOString(),
};
console.log(`Orphan detected: ${JSON.stringify(record)}`);
reported++;
}
}
console.log(`Done. ${reported} order(s) reported for human triage, ${cleaned} reservation(s) ${DRY_RUN ? "to delete" : "deleted"}.`);
}
if (import.meta.url === pathToFileURL(process.argv[1]).href) {
run().catch((err) => { console.error(err); process.exit(1); });
}Add a test
classify_orphan is the part most worth testing, because it decides whether a real order's partial state gets reported as a bug versus left alone as healthy. It is pure, so the test needs no network and no Medusa backend. It just feeds in plain fixture objects and checks the answer.
from reconcile_skipped_compensation import classify_orphan
def order(**over):
base = {
"id": "order_1",
"payment_status": "captured",
"fulfillment_status": "not_fulfilled",
"payments": [{"status": "captured"}],
"fulfillments": [],
"items": [{"id": "item_1", "quantity": 1}],
}
base.update(over)
return base
CONTINUE_FAILED_STEP = [{"action": "captureStep.continueOnPermanentFailure", "handlerType": "invoke"}]
RESERVE_FAILED_STEP = [{"action": "reserveInventoryStep", "handlerType": "invoke"}]
def test_orphaned_payment_no_fulfillment_when_captured_and_unfulfilled():
assert classify_orphan(order(), CONTINUE_FAILED_STEP) == "orphaned_payment_no_fulfillment"
def test_ok_when_no_failed_steps():
assert classify_orphan(order(), []) == "ok"
def test_ok_when_fulfillment_exists():
o = order(fulfillments=[{"id": "ful_1"}])
assert classify_orphan(o, CONTINUE_FAILED_STEP) == "ok"
def test_ok_when_payment_not_captured():
o = order(payment_status="not_paid", payments=[])
assert classify_orphan(o, CONTINUE_FAILED_STEP) == "ok"
def test_orphaned_reservation_no_order_line_when_items_empty():
o = order(items=[], payment_status="not_paid", payments=[])
assert classify_orphan(o, RESERVE_FAILED_STEP) == "orphaned_reservation_no_order_line"
def test_ok_when_reservation_failed_but_items_still_present():
assert classify_orphan(order(), RESERVE_FAILED_STEP) == "ok"import { test } from "node:test";
import assert from "node:assert/strict";
import { classifyOrphan } from "./reconcile-skipped-compensation.js";
const order = (over = {}) => ({
id: "order_1",
payment_status: "captured",
fulfillment_status: "not_fulfilled",
payments: [{ status: "captured" }],
fulfillments: [],
items: [{ id: "item_1", quantity: 1 }],
...over,
});
const CONTINUE_FAILED_STEP = [{ action: "captureStep.continueOnPermanentFailure", handlerType: "invoke" }];
const RESERVE_FAILED_STEP = [{ action: "reserveInventoryStep", handlerType: "invoke" }];
test("orphaned_payment_no_fulfillment when captured and unfulfilled", () => {
assert.equal(classifyOrphan(order(), CONTINUE_FAILED_STEP), "orphaned_payment_no_fulfillment");
});
test("ok when no failed steps", () => {
assert.equal(classifyOrphan(order(), []), "ok");
});
test("ok when fulfillment exists", () => {
const o = order({ fulfillments: [{ id: "ful_1" }] });
assert.equal(classifyOrphan(o, CONTINUE_FAILED_STEP), "ok");
});
test("ok when payment not captured", () => {
const o = order({ payment_status: "not_paid", payments: [] });
assert.equal(classifyOrphan(o, CONTINUE_FAILED_STEP), "ok");
});
test("orphaned_reservation_no_order_line when items empty", () => {
const o = order({ items: [], payment_status: "not_paid", payments: [] });
assert.equal(classifyOrphan(o, RESERVE_FAILED_STEP), "orphaned_reservation_no_order_line");
});
test("ok when reservation failed but items still present", () => {
assert.equal(classifyOrphan(order(), RESERVE_FAILED_STEP), "ok");
});Case studies
A capture that outlived its fulfillment step
A team flagged their capture step continueOnPermanentFailure because a capture, once it succeeds at the processor, is not something they wanted a bug in a later step to blindly reverse. Weeks later a fulfillment integration started failing intermittently, and each failure permanently failed that later step and triggered the saga's rollback. The capture step, flagged as it was, was walked past by the rollback every time.
Running the reconciler in dry run against the last day of orders surfaced a growing list of orders with payment_status: captured and an empty fulfillments array, each one tagged with the exact action id that had failed. The team used that list to manually trigger fulfillment for the orders that were genuinely fine, and to refund the handful that were not, instead of discovering the gap from a customer support ticket.
A reservation with no order line to justify it
A custom checkout workflow reserved inventory early and only created the order line items in a later step, again guarded with continueOnPermanentFailure on the reservation step because a partial reservation was considered cheap to leave in place. A validation bug meant the order-line step failed permanently on a subset of carts, and the reservation step's compensation, being skipped, never freed the stock.
The reconciler classified those cases as orphaned_reservation_no_order_line, the one outcome this script is allowed to act on. With DRY_RUN=false, it deleted each dangling reservation one at a time, logging the response before moving to the next, and the stock became sellable again within the hour instead of sitting reserved indefinitely.
After this runs on a schedule, a step flagged continueOnPermanentFailure can still do its job, keep the workflow moving, without silently burying the state it left behind. Every payment-without-fulfillment gap gets a structured record with the order id, the failing step, and the error message, ready for a human to triage. The one truly safe cleanup, a dangling reservation with no order line, gets cleared automatically, one guarded delete at a time. Nothing that touches money gets reversed on a guess.
FAQ
What does continueOnPermanentFailure actually skip in a Medusa workflow?
Setting .config({ continueOnPermanentFailure: true }) on a step tells Medusa that if this step permanently fails, the workflow should keep running the remaining steps instead of stopping. Per Medusa's own documentation, the compensation function of that flagged step will not be called. So if the step already committed a side effect, such as creating an order or capturing a payment, before it later reports a permanent failure, that side effect is never undone, even though the workflow carries on as if nothing happened.
Why does a later failure in the same workflow not clean up the earlier one?
The saga's rollback only compensates steps that both completed successfully and have compensation registered for them. A step marked continueOnPermanentFailure is explicitly excluded from that contract, so even when a sibling step fails afterward and triggers a full rollback, the orchestrator walks past the flagged step without touching it. The record it created, an order, a captured payment, or a reservation, survives with no corresponding downstream state ever created.
Is it safe to automatically reverse orders left behind by this pattern?
No, not automatically. continueOnPermanentFailure is usually set on purpose because the workflow author decided the step's side effect, often a payment capture, is not cheaply reversible and should not be rolled back blind. The safe default is to detect the gap between payment_status and fulfillment_status through the Admin API and report it as a structured record for a human to triage, only auto-cleaning the clearly safe case of a dangling reservation with no live order line, and only after a dry run.
Related field notes
Citations
On the problem:
- Medusa Documentation: Error Handling in Workflows, including continueOnPermanentFailure. docs.medusajs.com/learn/fundamentals/workflows/errors
- feat(orchestration): skip on permanent failure. Medusa GitHub Pull Request #12027. github.com/medusajs/medusa/pull/12027
- Order fulfillment is not deleting inventory item reservations sometimes. Medusa GitHub Issue #11266. github.com/medusajs/medusa/issues/11266
On the solution:
- Medusa Documentation: Compensation Function. docs.medusajs.com/learn/fundamentals/workflows/compensation-function
- Medusa Documentation: Retry Failed Steps. docs.medusajs.com/learn/fundamentals/workflows/retry-failed-steps
- Medusa V2 Admin API Reference. docs.medusajs.com/api/admin
Stuck on a tricky one?
If you have a problem in Medusa storefront access, pricing, inventory, orders, promotions, or workflows that you would rather hand off, this is the kind of work I do. Message me and we can work through it together.
Did this untangle a stuck workflow?
If this saved you from chasing a phantom order gap or a support ticket about stock that would not add up, you can buy me a coffee. It is the best way to keep these field notes free and growing.
Buy me a coffee on Ko-fi