allanninal.dev / shopify

Field notes / Orders, payments, and webhooks

Reconciler Orders, payments, and webhooks

Missed webhooks with no backfill

Your app went down, or a deploy misconfigured the endpoint, or a certificate expired quietly over a long weekend. Shopify kept trying to deliver its webhooks the whole time, but its patience runs out. Once your outage crosses that limit, the events that fired during it are gone for good, and no setting brings them back. Here is why the gap is unrecoverable and a small script that polls what changed and applies it anyway.

Python and Node.js Admin GraphQL API Safe by default (dry run)
Blue utp network cord
Photo by Jordan Harrison on Unsplash
The short answer

Shopify retries a failing webhook delivery for up to 48 hours, then drops it permanently. There is no replay endpoint and no history of the exact payload. The fix is to stop depending on webhooks alone for anything you cannot afford to lose: poll the Admin GraphQL API for orders whose updatedAt falls inside your outage window, read each order's current state directly, and apply the update yourself. Tag each order once it is reprocessed so a second run never repeats it. Full code, tests, and a dry run guard are below.

The problem in plain words

Webhooks feel like a guarantee. Something changes on an order, Shopify posts it to your endpoint, your app reacts. Most of the time that loop just works, so it is easy to build an entire pipeline that assumes every event will land eventually.

It will not. Shopify's delivery system retries a failed POST on a backoff schedule, but only for a fixed window, currently up to 48 hours. If your endpoint returns errors, times out, or is simply unreachable for the whole window, whether from a crashed server, a bad deploy, an expired TLS certificate, or a hosting outage, the retries stop and the event is discarded. Shopify does not keep a log you can replay from and does not resend it later just because your endpoint came back. The order changed. Your system never heard about it, and by the time you notice, there is nothing left to ask Shopify for.

Order changes Shopify fires webhook Retries on backoff endpoint still down up to 48 hours ✕ window runs out Event dropped no replay, no log App never finds out
Once the retry window closes, the specific webhook payload is gone. Shopify will not resend it and keeps no history you can pull it from.

Why it happens

Webhooks are a delivery mechanism, not a durable event log on Shopify's side. A few common ways stores end up with a real gap:

None of these are unusual. What makes this particular failure sharp is the asymmetry: the outage might last an hour, but if it straddles the 48 hour cutoff for even one order's events, that order's history has a permanent hole in it. See the citations at the end for Shopify's own documentation on the retry schedule.

The key insight

You cannot recover the exact webhook payload that was dropped, so do not try to reconstruct it. Instead, ask Shopify for the current state of every order that changed during the window, using updated_at as your filter, and treat that current state as good enough to react to. The gap in your event stream closes even though the individual events never arrive.

The fix, as a flow

We do not touch the webhook subscriptions themselves. We add a one-time or on-demand job that lists orders updated inside the known outage window, keeps only the ones that have not already been marked reprocessed, reads their current financial and fulfillment state, and applies that state the same way a webhook handler would, then tags the order so a second run never replays it.

Known outage window GAP_START, GAP_END List updated orders updated_at inside window Read status and tags financial, fulfillment, tags In window and not tagged? yes no, skip Apply and tag webhook-backfilled
The script only reprocesses orders that changed inside the known outage window and that have not already been marked backfilled. Everything else is left untouched.

Build it step by step

1

Get an Admin API access token and pin down the window

Create a custom app in your Shopify admin under Settings, Apps and sales channels, Develop apps. Give it read_orders and write_orders scopes and install it to get an Admin API access token that starts with shpat_. Pull the exact outage start and end from your monitoring or hosting logs, in ISO 8601, and widen each edge by a few minutes since clocks and webhook queues are never perfectly aligned.

setup (shell)
pip install requests

export SHOPIFY_SHOP="yourstore.myshopify.com"
export SHOPIFY_ACCESS_TOKEN="shpat_..."
export GAP_START="2026-07-05T00:00:00Z"
export GAP_END="2026-07-06T00:00:00Z"
export BACKFILL_TAG="webhook-backfilled"
export DRY_RUN="true"   # start safe, change to false to write
setup (shell)
// Node 18+ has fetch built in, no dependencies needed

export SHOPIFY_SHOP="yourstore.myshopify.com"
export SHOPIFY_ACCESS_TOKEN="shpat_..."
export GAP_START="2026-07-05T00:00:00Z"
export GAP_END="2026-07-06T00:00:00Z"
export BACKFILL_TAG="webhook-backfilled"
export DRY_RUN="true"   // start safe, change to false to write
2

Talk to the Admin GraphQL API

Every call goes to one GraphQL endpoint with your token in the X-Shopify-Access-Token header. A small helper sends a query and returns the data, and raises if Shopify reports an error. We use this same helper for the poll and for the tag write.

step2.py
import os, requests

SHOP = os.environ["SHOPIFY_SHOP"]
TOKEN = os.environ["SHOPIFY_ACCESS_TOKEN"]
API_VERSION = os.environ.get("SHOPIFY_API_VERSION", "2025-01")
ENDPOINT = f"https://{SHOP}/admin/api/{API_VERSION}/graphql.json"

def gql(query, variables=None):
    r = requests.post(
        ENDPOINT,
        json={"query": query, "variables": variables or {}},
        headers={"X-Shopify-Access-Token": TOKEN, "Content-Type": "application/json"},
        timeout=30,
    )
    r.raise_for_status()
    body = r.json()
    if body.get("errors"):
        raise RuntimeError(body["errors"])
    return body["data"]
step2.js
const SHOP = process.env.SHOPIFY_SHOP;
const TOKEN = process.env.SHOPIFY_ACCESS_TOKEN;
const API_VERSION = process.env.SHOPIFY_API_VERSION || "2025-01";
const ENDPOINT = `https://${SHOP}/admin/api/${API_VERSION}/graphql.json`;

async function gql(query, variables = {}) {
  const res = await fetch(ENDPOINT, {
    method: "POST",
    headers: { "X-Shopify-Access-Token": TOKEN, "Content-Type": "application/json" },
    body: JSON.stringify({ query, variables }),
  });
  if (!res.ok) throw new Error(`Shopify ${res.status}`);
  const body = await res.json();
  if (body.errors) throw new Error(JSON.stringify(body.errors));
  return body.data;
}
3

List orders updated during the gap

Filter with updated_at for the exact window, sorted so the results come back in order. Read the fields the decision needs: the name, when it last changed, its current tags, financial and fulfillment status, and the amount actually received. We page through with a cursor so the job handles a wide window without missing anything.

step3.py
ORDERS_QUERY = """
query($cursor: String, $q: String!) {
  orders(first: 50, after: $cursor, query: $q, sortKey: UPDATED_AT) {
    pageInfo { hasNextPage endCursor }
    nodes {
      id name tags updatedAt
      displayFinancialStatus
      displayFulfillmentStatus
      cancelledAt
      totalReceivedSet { shopMoney { amount currencyCode } }
    }
  }
}"""

def updated_orders_in_gap(gap_start, gap_end):
    q = f"updated_at:>='{gap_start}' AND updated_at:<='{gap_end}'"
    cursor = None
    while True:
        data = gql(ORDERS_QUERY, {"cursor": cursor, "q": q})["orders"]
        for node in data["nodes"]:
            yield node
        if not data["pageInfo"]["hasNextPage"]:
            return
        cursor = data["pageInfo"]["endCursor"]
step3.js
const ORDERS_QUERY = `
query($cursor: String, $q: String!) {
  orders(first: 50, after: $cursor, query: $q, sortKey: UPDATED_AT) {
    pageInfo { hasNextPage endCursor }
    nodes {
      id name tags updatedAt
      displayFinancialStatus
      displayFulfillmentStatus
      cancelledAt
      totalReceivedSet { shopMoney { amount currencyCode } }
    }
  }
}`;

async function* updatedOrdersInGap(gapStart, gapEnd) {
  const q = `updated_at:>='${gapStart}' AND updated_at:<='${gapEnd}'`;
  let cursor = null;
  while (true) {
    const data = (await gql(ORDERS_QUERY, { cursor, q })).orders;
    for (const node of data.nodes) yield node;
    if (!data.pageInfo.hasNextPage) return;
    cursor = data.pageInfo.endCursor;
  }
}
4

Decide, with one pure function

Keep the decision in its own function that takes an order, the gap boundaries, and the name of the done tag, and returns true or false. A pure function like this is easy to read and easy to test, which we do later. An order needs backfilling only when its own updatedAt falls inside the outage window and it has not already been marked reprocessed. Money is read out in cents to keep any later comparison free of floating point drift.

decide.py
def to_cents(amount):
    return round(float(amount) * 100)

def needs_backfill(order, gap_start, gap_end, done_tag):
    updated_at = order.get("updatedAt")
    if not updated_at:
        return False
    if not (gap_start <= updated_at <= gap_end):
        return False
    return done_tag not in (order.get("tags") or [])
decide.js
export function toCents(amount) {
  return Math.round(parseFloat(amount) * 100);
}

export function needsBackfill(order, gapStart, gapEnd, doneTag) {
  const updatedAt = order.updatedAt;
  if (!updatedAt) return false;
  if (!(updatedAt >= gapStart && updatedAt <= gapEnd)) return false;
  return !(order.tags || []).includes(doneTag);
}
5

Read the current state instead of guessing the missed payload

You cannot recover the exact event Shopify tried to send, so do not try. Read the order's current financial status, fulfillment status, cancellation, and amount received, and treat that as the reprocessed truth. Then write a single review tag so the same order is never replayed on a later run.

apply.py
TAGS_ADD = """
mutation($id: ID!, $tags: [String!]!) {
  tagsAdd(id: $id, tags: $tags) { node { id } userErrors { field message } }
}"""

def summarize(order):
    received = order.get("totalReceivedSet") or {}
    amount = (received.get("shopMoney") or {}).get("amount", "0")
    return {
        "id": order["id"],
        "name": order["name"],
        "financial_status": order.get("displayFinancialStatus"),
        "fulfillment_status": order.get("displayFulfillmentStatus"),
        "cancelled": bool(order.get("cancelledAt")),
        "total_received_cents": to_cents(amount),
    }

def mark_backfilled(order_id, done_tag):
    result = gql(TAGS_ADD, {"id": order_id, "tags": [done_tag]})["tagsAdd"]
    if result["userErrors"]:
        raise RuntimeError(result["userErrors"])
apply.js
const TAGS_ADD = `
mutation($id: ID!, $tags: [String!]!) {
  tagsAdd(id: $id, tags: $tags) { node { id } userErrors { field message } }
}`;

export function summarize(order) {
  const amount = order.totalReceivedSet?.shopMoney?.amount ?? "0";
  return {
    id: order.id,
    name: order.name,
    financialStatus: order.displayFinancialStatus,
    fulfillmentStatus: order.displayFulfillmentStatus,
    cancelled: Boolean(order.cancelledAt),
    totalReceivedCents: toCents(amount),
  };
}

async function markBackfilled(orderId, doneTag) {
  const result = (await gql(TAGS_ADD, { id: orderId, tags: [doneTag] })).tagsAdd;
  if (result.userErrors.length) throw new Error(JSON.stringify(result.userErrors));
}
6

Wire it together with a dry run guard

The loop ties every piece together. Notice the dry run guard. On the first run, leave DRY_RUN on so the script only logs which orders it would reprocess and what their current state looks like. Read the output, confirm it lines up with the outage window, then switch it off to let it write the tag. This is a one-off run against a known window, not a recurring schedule, though nothing stops you from running it again if you discover the window was wider than you thought.

Run it safe

Always start with DRY_RUN=true, and double check GAP_START and GAP_END against your incident timeline before writing anything. A window that is too wide will tag healthy orders that had nothing to do with the outage.

The full code

Here is the complete script in one file for each language. It reads settings from the environment, logs what it does, respects the dry run flag, and is safe to run again and again because it only reprocesses orders inside the stated window that have not already been tagged.

View this code on GitHub Full runnable folder with tests in the shopify-fixes repo.

backfill_missed_webhooks.py
"""Backfill Shopify orders whose webhooks were missed during downtime.

Shopify retries a failing webhook for up to 48 hours, then drops it for good.
If your endpoint was down longer than that, some orders never told you they
were paid, fulfilled, or cancelled. This job polls orders updated during the
outage window, keeps only the ones whose updatedAt falls inside that window
and that have not already been reprocessed, and re-applies the update by
tagging the order and logging what would have shipped in the missed webhook.
Read heavy, one small write. Safe to run again and again.
"""
import os
import logging
import requests

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("backfill_missed_webhooks")

SHOP = os.environ["SHOPIFY_SHOP"]
TOKEN = os.environ["SHOPIFY_ACCESS_TOKEN"]
API_VERSION = os.environ.get("SHOPIFY_API_VERSION", "2025-01")
ENDPOINT = f"https://{SHOP}/admin/api/{API_VERSION}/graphql.json"

# The window your app was unreachable, in ISO 8601. Widen it a little on
# both sides, since Shopify's retry schedule is not instant either.
GAP_START = os.environ.get("GAP_START", "2026-07-05T00:00:00Z")
GAP_END = os.environ.get("GAP_END", "2026-07-06T00:00:00Z")
BACKFILL_TAG = os.environ.get("BACKFILL_TAG", "webhook-backfilled")
DRY_RUN = os.environ.get("DRY_RUN", "true").lower() == "true"

ORDERS_QUERY = """
query($cursor: String, $q: String!) {
  orders(first: 50, after: $cursor, query: $q, sortKey: UPDATED_AT) {
    pageInfo { hasNextPage endCursor }
    nodes {
      id name tags updatedAt
      displayFinancialStatus
      displayFulfillmentStatus
      cancelledAt
      totalReceivedSet { shopMoney { amount currencyCode } }
    }
  }
}"""

TAGS_ADD = """
mutation($id: ID!, $tags: [String!]!) {
  tagsAdd(id: $id, tags: $tags) { node { id } userErrors { field message } }
}"""


def gql(query, variables=None):
    r = requests.post(
        ENDPOINT,
        json={"query": query, "variables": variables or {}},
        headers={"X-Shopify-Access-Token": TOKEN, "Content-Type": "application/json"},
        timeout=30,
    )
    r.raise_for_status()
    body = r.json()
    if body.get("errors"):
        raise RuntimeError(body["errors"])
    return body["data"]


def to_cents(amount):
    return round(float(amount) * 100)


def needs_backfill(order, gap_start, gap_end, done_tag):
    """Pure decision. True when an order's own update happened inside the
    outage window and it has not already been marked as reprocessed.
    """
    updated_at = order.get("updatedAt")
    if not updated_at:
        return False
    if not (gap_start <= updated_at <= gap_end):
        return False
    return done_tag not in (order.get("tags") or [])


def summarize(order):
    """What the missed webhook would have told us, in plain fields we can
    log or replay into a local system. Money is kept in cents.
    """
    received = order.get("totalReceivedSet") or {}
    amount = (received.get("shopMoney") or {}).get("amount", "0")
    return {
        "id": order["id"],
        "name": order["name"],
        "financial_status": order.get("displayFinancialStatus"),
        "fulfillment_status": order.get("displayFulfillmentStatus"),
        "cancelled": bool(order.get("cancelledAt")),
        "total_received_cents": to_cents(amount),
    }


def mark_backfilled(order_id, done_tag):
    result = gql(TAGS_ADD, {"id": order_id, "tags": [done_tag]})["tagsAdd"]
    if result["userErrors"]:
        raise RuntimeError(result["userErrors"])


def updated_orders_in_gap():
    q = f"updated_at:>='{GAP_START}' AND updated_at:<='{GAP_END}'"
    cursor = None
    while True:
        data = gql(ORDERS_QUERY, {"cursor": cursor, "q": q})["orders"]
        for node in data["nodes"]:
            yield node
        if not data["pageInfo"]["hasNextPage"]:
            return
        cursor = data["pageInfo"]["endCursor"]


def run():
    backfilled = 0
    for order in updated_orders_in_gap():
        if not needs_backfill(order, GAP_START, GAP_END, BACKFILL_TAG):
            continue
        state = summarize(order)
        log.info(
            "Order %s missed a webhook. financial=%s fulfillment=%s received_cents=%d. %s",
            state["name"], state["financial_status"], state["fulfillment_status"],
            state["total_received_cents"], "would backfill" if DRY_RUN else "backfilling",
        )
        if not DRY_RUN:
            # Apply your own side effect here: sync to your database, send
            # an internal event, trigger fulfillment, etc. Then mark it done
            # so a later run never replays the same order twice.
            mark_backfilled(order["id"], BACKFILL_TAG)
        backfilled += 1
    log.info("Done. %d order(s) %s.", backfilled, "to backfill" if DRY_RUN else "backfilled")


if __name__ == "__main__":
    run()
backfill-missed-webhooks.js
/**
 * Backfill Shopify orders whose webhooks were missed during downtime.
 *
 * Shopify retries a failing webhook for up to 48 hours, then drops it for
 * good. If your endpoint was down longer than that, some orders never told
 * you they were paid, fulfilled, or cancelled. This job polls orders updated
 * during the outage window, keeps only the ones whose updatedAt falls inside
 * that window and that have not already been reprocessed, and re-applies the
 * update by tagging the order and logging what would have shipped in the
 * missed webhook. Read heavy, one small write. Safe to run again and again.
 */
import { pathToFileURL } from "node:url";

const SHOP = process.env.SHOPIFY_SHOP || "example.myshopify.com";
const TOKEN = process.env.SHOPIFY_ACCESS_TOKEN || "shpat_dummy";
const API_VERSION = process.env.SHOPIFY_API_VERSION || "2025-01";
const ENDPOINT = `https://${SHOP}/admin/api/${API_VERSION}/graphql.json`;

// The window your app was unreachable, in ISO 8601. Widen it a little on
// both sides, since Shopify's retry schedule is not instant either.
const GAP_START = process.env.GAP_START || "2026-07-05T00:00:00Z";
const GAP_END = process.env.GAP_END || "2026-07-06T00:00:00Z";
const BACKFILL_TAG = process.env.BACKFILL_TAG || "webhook-backfilled";
const DRY_RUN = (process.env.DRY_RUN || "true").toLowerCase() === "true";

export function toCents(amount) {
  return Math.round(parseFloat(amount) * 100);
}

export function needsBackfill(order, gapStart, gapEnd, doneTag) {
  const updatedAt = order.updatedAt;
  if (!updatedAt) return false;
  if (!(updatedAt >= gapStart && updatedAt <= gapEnd)) return false;
  return !(order.tags || []).includes(doneTag);
}

export function summarize(order) {
  const amount = order.totalReceivedSet?.shopMoney?.amount ?? "0";
  return {
    id: order.id,
    name: order.name,
    financialStatus: order.displayFinancialStatus,
    fulfillmentStatus: order.displayFulfillmentStatus,
    cancelled: Boolean(order.cancelledAt),
    totalReceivedCents: toCents(amount),
  };
}

async function gql(query, variables = {}) {
  const res = await fetch(ENDPOINT, {
    method: "POST",
    headers: { "X-Shopify-Access-Token": TOKEN, "Content-Type": "application/json" },
    body: JSON.stringify({ query, variables }),
  });
  if (!res.ok) throw new Error(`Shopify ${res.status}`);
  const body = await res.json();
  if (body.errors) throw new Error(JSON.stringify(body.errors));
  return body.data;
}

const ORDERS_QUERY = `
query($cursor: String, $q: String!) {
  orders(first: 50, after: $cursor, query: $q, sortKey: UPDATED_AT) {
    pageInfo { hasNextPage endCursor }
    nodes {
      id name tags updatedAt
      displayFinancialStatus
      displayFulfillmentStatus
      cancelledAt
      totalReceivedSet { shopMoney { amount currencyCode } }
    }
  }
}`;

const TAGS_ADD = `
mutation($id: ID!, $tags: [String!]!) {
  tagsAdd(id: $id, tags: $tags) { node { id } userErrors { field message } }
}`;

async function* updatedOrdersInGap() {
  const q = `updated_at:>='${GAP_START}' AND updated_at:<='${GAP_END}'`;
  let cursor = null;
  while (true) {
    const data = (await gql(ORDERS_QUERY, { cursor, q })).orders;
    for (const node of data.nodes) yield node;
    if (!data.pageInfo.hasNextPage) return;
    cursor = data.pageInfo.endCursor;
  }
}

async function markBackfilled(orderId, doneTag) {
  const result = (await gql(TAGS_ADD, { id: orderId, tags: [doneTag] })).tagsAdd;
  if (result.userErrors.length) throw new Error(JSON.stringify(result.userErrors));
}

export async function run() {
  let backfilled = 0;
  for await (const order of updatedOrdersInGap()) {
    if (!needsBackfill(order, GAP_START, GAP_END, BACKFILL_TAG)) continue;
    const state = summarize(order);
    console.log(
      `Order ${state.name} missed a webhook. financial=${state.financialStatus} ` +
      `fulfillment=${state.fulfillmentStatus} received_cents=${state.totalReceivedCents}. ` +
      `${DRY_RUN ? "would backfill" : "backfilling"}`
    );
    if (!DRY_RUN) {
      // Apply your own side effect here: sync to your database, send an
      // internal event, trigger fulfillment, etc. Then mark it done so a
      // later run never replays the same order twice.
      await markBackfilled(order.id, BACKFILL_TAG);
    }
    backfilled++;
  }
  console.log(`Done. ${backfilled} order(s) ${DRY_RUN ? "to backfill" : "backfilled"}.`);
}

if (import.meta.url === pathToFileURL(process.argv[1]).href) {
  run().catch((err) => { console.error(err); process.exit(1); });
}

Add a test

The decision rule is the part most worth testing, because it decides which orders get reprocessed. Because we kept needs_backfill pure, the test needs no network and no Shopify account. It just feeds in plain objects and checks the answer.

test_missed_webhooks_backfill.py
from backfill_missed_webhooks import needs_backfill, summarize, to_cents


def order(**over):
    base = {
        "id": "gid://shopify/Order/1",
        "name": "#1001",
        "tags": [],
        "updatedAt": "2026-07-05T12:00:00Z",
        "displayFinancialStatus": "PAID",
        "displayFulfillmentStatus": "UNFULFILLED",
        "cancelledAt": None,
        "totalReceivedSet": {"shopMoney": {"amount": "50.00", "currencyCode": "USD"}},
    }
    base.update(over)
    return base


GAP_START = "2026-07-05T00:00:00Z"
GAP_END = "2026-07-06T00:00:00Z"


def test_to_cents_rounds():
    assert to_cents("50.00") == 5000
    assert to_cents("9.99") == 999


def test_needs_backfill_when_updated_inside_gap_and_untagged():
    assert needs_backfill(order(), GAP_START, GAP_END, "webhook-backfilled") is True


def test_skip_when_updated_before_gap():
    o = order(updatedAt="2026-07-04T23:00:00Z")
    assert needs_backfill(o, GAP_START, GAP_END, "webhook-backfilled") is False


def test_skip_when_updated_after_gap():
    o = order(updatedAt="2026-07-06T01:00:00Z")
    assert needs_backfill(o, GAP_START, GAP_END, "webhook-backfilled") is False


def test_skip_when_already_tagged():
    o = order(tags=["webhook-backfilled"])
    assert needs_backfill(o, GAP_START, GAP_END, "webhook-backfilled") is False


def test_skip_when_no_updated_at():
    o = order(updatedAt=None)
    assert needs_backfill(o, GAP_START, GAP_END, "webhook-backfilled") is False


def test_boundary_timestamps_are_inclusive():
    assert needs_backfill(order(updatedAt=GAP_START), GAP_START, GAP_END, "webhook-backfilled") is True
    assert needs_backfill(order(updatedAt=GAP_END), GAP_START, GAP_END, "webhook-backfilled") is True


def test_summarize_reads_money_in_cents():
    state = summarize(order(totalReceivedSet={"shopMoney": {"amount": "129.99", "currencyCode": "USD"}}))
    assert state["total_received_cents"] == 12999
    assert state["financial_status"] == "PAID"
    assert state["cancelled"] is False
backfill-missed-webhooks.test.js
import { test } from "node:test";
import assert from "node:assert/strict";
import { needsBackfill, summarize, toCents } from "./backfill-missed-webhooks.js";

const GAP_START = "2026-07-05T00:00:00Z";
const GAP_END = "2026-07-06T00:00:00Z";

const order = (over = {}) => ({
  id: "gid://shopify/Order/1",
  name: "#1001",
  tags: [],
  updatedAt: "2026-07-05T12:00:00Z",
  displayFinancialStatus: "PAID",
  displayFulfillmentStatus: "UNFULFILLED",
  cancelledAt: null,
  totalReceivedSet: { shopMoney: { amount: "50.00", currencyCode: "USD" } },
  ...over,
});

test("needs backfill when updated inside gap and untagged", () => {
  assert.equal(needsBackfill(order(), GAP_START, GAP_END, "webhook-backfilled"), true);
});

test("skip when updated before gap", () => {
  const o = order({ updatedAt: "2026-07-04T23:00:00Z" });
  assert.equal(needsBackfill(o, GAP_START, GAP_END, "webhook-backfilled"), false);
});

test("skip when updated after gap", () => {
  const o = order({ updatedAt: "2026-07-06T01:00:00Z" });
  assert.equal(needsBackfill(o, GAP_START, GAP_END, "webhook-backfilled"), false);
});

test("skip when already tagged", () => {
  const o = order({ tags: ["webhook-backfilled"] });
  assert.equal(needsBackfill(o, GAP_START, GAP_END, "webhook-backfilled"), false);
});

test("summarize reads money in cents", () => {
  const state = summarize(order({ totalReceivedSet: { shopMoney: { amount: "129.99", currencyCode: "USD" } } }));
  assert.equal(state.totalReceivedCents, 12999);
  assert.equal(state.financialStatus, "PAID");
});

Case studies

Expired certificate

The weekend the certificate lapsed

A small apparel store's webhook endpoint sat behind a TLS certificate that nobody had put on an auto-renew schedule. It expired at midnight on a Friday, and every delivery attempt for the next 30 hours failed the handshake before it reached the app. Orders kept coming in and getting paid all weekend, but the fulfillment queue that reacted to payment webhooks never grew.

By Monday the gap was 30 hours wide, safely under the 48 hour cutoff, but nobody wanted to check hundreds of orders by hand. The team pointed GAP_START and GAP_END at the exact outage from their monitoring dashboard, ran the script in dry run, confirmed the list matched the paid-but-unqueued orders, and let it tag them so fulfillment picked every one back up.

Bad deploy

A deploy that quietly broke the webhook route

A subscription app shipped a refactor that renamed a route parameter. The webhook endpoint started returning 500 for every request, but the rest of the app worked fine, so the on-call engineer did not notice for almost two days. Cancellations and refunds that happened during that stretch never reached the app's internal ledger.

Once the bug was found and fixed, the team ran the backfill script against the deploy-to-fix window. It surfaced every order whose status had moved during the outage, including several cancellations the ledger had never recorded, and tagging them closed the gap without anyone guessing at what the missing webhooks might have said.

What good looks like

After the backfill runs, every order that changed during the outage has been read and reprocessed once, and each carries a tag proving it. The gap in your event history is closed even though the original webhook payloads are gone forever. The bigger fix is treating webhooks as a convenience, not the only source of truth, so the next outage is a known, closeable gap instead of a permanent blind spot.

FAQ

Does Shopify resend a webhook forever until it succeeds?

No. Shopify retries a failing webhook delivery on a backoff schedule for up to 48 hours, then stops and drops it for good. If your endpoint was unreachable longer than that window, the event is gone and nothing will ever resend it to you.

How do I find the orders a missed webhook would have covered?

Query orders with the Admin GraphQL API filtered by updated_at for the exact start and end of your downtime window. Any order whose updatedAt falls inside that window changed state while you were unreachable, which is the same set your webhooks would have described.

Is it safe to replay every order updated during the outage?

Yes, when the script reads the order's current state instead of guessing what the missed event said, tags each order once it has been reprocessed so a second run never repeats it, and starts in dry run so you can review the exact list before anything writes.

Related field notes

Orders stuck Unpaid after a payment made outside checkout Transactions total does not match the order Browse all Shopify field notes

Citations

On the problem:

  1. Shopify Developer Docs: webhook delivery and the retry schedule, including the retirement of unrecoverable deliveries. shopify.dev/docs/apps/build/webhooks
  2. Shopify Developer Docs: troubleshooting webhook delivery failures. shopify.dev/docs/apps/build/webhooks/troubleshooting
  3. Shopify Community: apps that missed webhooks during downtime and had no way to recover the payloads. community.shopify.com graphql admin api

On the solution:

  1. Shopify Admin GraphQL: the orders query and its search syntax, including filtering by updated_at. shopify.dev/docs/api/admin-graphql/latest/queries/orders
  2. Shopify Admin GraphQL: the Order object, including updatedAt, displayFinancialStatus, and displayFulfillmentStatus. shopify.dev/docs/api/admin-graphql/latest/objects/Order
  3. Shopify Admin GraphQL: the tagsAdd mutation. shopify.dev/docs/api/admin-graphql/latest/mutations/tagsAdd

Stuck on a tricky one?

If you have a problem in Shopify orders, payments, subscriptions, inventory, or fulfillment that you would rather hand off, this is the kind of work I do. Message me and we can work through it together.

Contact me on LinkedIn

Did this close a webhook gap for you?

If this saved you from guessing at a missed payload or chasing orders by hand, you can buy me a coffee. It is the best way to keep these field notes free and growing.

Buy me a coffee on Ko-fi

Back to all Shopify field notes