Field notes / Orders and Grid Sync

Reconciler Orders and Grid Sync

Sales order grid permanently out of sync with sales_order

Support asks about an order that the customer swears they placed. You look it up with the REST API and there it is, entity_id, correct status, correct total, sitting right there in sales_order. But the admin order grid does not show it, or shows it stuck on an old status with a stale total. The order was never lost. The grid, a separate table that a cron job is supposed to keep in sync, just stopped picking it up, and a documented race condition means it may never pick it up again on its own. Here is why that happens and a small script that finds exactly which orders drifted.

Python and Node.js Orders REST API Safe by default (report only)

Papers with sticky tabs — Photo by Tanja Tepavac on Unsplash

The short answer

When Stores > Configuration > Advanced > Developer > Grid Settings has Asynchronous Indexing enabled (dev/grid/async_indexing=1), orders are written to sales_order immediately, but the admin grid reads from sales_order_grid, a copy that a scheduled cron job fills in later within a window bounded by a cached watermark on updated_at. A documented race, magento/magento2 issue #40803, lets one cron run advance that watermark past an order's updated_at while the order's grid row write is still in flight or fails. Because the watermark's cache is refreshed every cycle, that order's timestamp stays permanently below the new floor and is never synced again. The same pipeline can also drop rows if cron is killed mid-batch or the grid re-save throws. There is no REST endpoint to rebuild a single grid row, so the safe move is to diff sales_order against what the grid-backed list endpoint returns, and report exactly which entity_id values need bin/magento indexer:reindex sales_order_grid. Full code, tests, and a dry run guard are below.

The problem in plain words

The admin order grid is not a live view of sales_order. It is a denormalized copy, sales_order_grid, built so the grid can filter and sort quickly without joining half a dozen tables on every page load. When an order is created or changes, checkout and the REST API write straight to sales_order. The grid table only gets that same update through a separate step.

With synchronous grid indexing, that separate step happens in the same request, so it is slow but never falls behind. Asynchronous grid indexing, enabled with dev/grid/async_indexing=1, trades that for speed. The order save returns immediately, and a scheduled cron job comes along afterward, looks at what changed since the last time it ran, and copies those changes into sales_order_grid. That cron job tracks its own progress with a cached watermark on updated_at, essentially "I have already synced everything up to this timestamp."

The problem is that watermark does not wait for every row in its own batch to actually finish writing before it advances. If one order's grid write is still in flight, or throws partway through, for example because of missing customer or address data, the cron run can still move the watermark forward past that order's updated_at. The next cron cycle only looks at rows updated after the new watermark, so an order sitting just behind it is invisible to every future run. It is not that the order is slow to sync. It is permanently below the floor cron now starts from, and cron never looks backward on its own.

sales_order is never wrong. The grid copy just stopped being told about this one order, and the cron watermark that tracks progress moved past it for good.

Why it happens

The store has Stores > Configuration > Advanced > Developer > Grid Settings set to Asynchronous Indexing (dev/grid/async_indexing=1), so grid rows are only ever written by cron, never inline with the order save.
The documented race in magento/magento2 issue #40803: the cron's cached sync watermark on updated_at advances past an order whose grid row write was still in flight or failed in that same batch, and every later cron cycle only looks forward from that watermark.
The cron process is killed mid-batch, by a crash, a timeout, or a deployment restarting PHP-FPM or the queue consumer, leaving some rows in the batch never written to sales_order_grid at all.
The grid re-save throws partway through, for example missing customer or address data referenced by the order, so that one order's row is skipped while the rest of the batch, and the watermark, move on without it.
A related but distinct failure mode covered in issue #36334: auto-invoiced orders under async processing can also desync between the order and its derived grid state.

Store owners usually notice this the same way: an order that unmistakably exists, confirmed by email, by payment gateway, by the customer, simply is not in the admin grid, or is there with a status and total that stopped updating days ago. See the citations at the end for the exact issue threads and forum posts.

The key insight

sales_order_grid is a database table with no REST endpoint of its own, and there is no public API to force a rebuild of a single order's grid row. What the REST API does expose is two different views of the same order: GET /V1/orders/{id}, which always reads sales_order directly, and a searchCriteria filter on entity_id against the list endpoint, which is what the grid-backed admin list effectively surfaces. Diffing those two views over the same set of ids is the only way to prove drift from the outside, and it is exactly what a script can do safely, since it never has to guess at cron internals.

The fix, as a flow

We do not try to patch sales_order_grid directly over the API, because there is no safe public way to do that. Instead we add a job that pulls the entity source of truth, cross checks every id against the grid-backed list view, classifies each drift by type, and reports the exact entity_id values that need a targeted indexer:reindex sales_order_grid, leaving the actual repair to the CLI where it belongs.

The script only ever reports drifted entity_id values. It never tries to write sales_order_grid itself, because the real fix runs on the CLI.

Build it step by step

Get an admin bearer token

Authenticate the same way as any other Magento REST client. Either call POST /rest/V1/integration/admin/token with an admin username and password, or create an integration and use its token directly. Keep the store URL and token in environment variables, never in the file.

setup (shell)

pip install requests

export MAGENTO_URL="https://your-store.example.com"
export MAGENTO_ADMIN_TOKEN="eyJraWQ..."
export SYNC_SINCE="2026-07-01 00:00:00"
export WATERMARK="2026-07-09 00:00:00"
export DRY_RUN="true"   # report-only either way, this only affects log verbosity

setup (shell)

// Node 18+ has fetch built in, no dependencies needed

export MAGENTO_URL="https://your-store.example.com"
export MAGENTO_ADMIN_TOKEN="eyJraWQ..."
export SYNC_SINCE="2026-07-01 00:00:00"
export WATERMARK="2026-07-09 00:00:00"
export DRY_RUN="true"   // report-only either way, this only affects log verbosity

Talk to the Magento REST API

Every call sends Authorization: Bearer <token> to a /rest/V1 route. A small helper wraps the request and raises on a non 200 response, since Magento returns structured error bodies worth surfacing as is.

step2.py

import os, requests

MAGENTO_URL = os.environ["MAGENTO_URL"].rstrip("/")
TOKEN = os.environ["MAGENTO_ADMIN_TOKEN"]

def magento_get(path, params=None):
    r = requests.get(
        f"{MAGENTO_URL}/rest/V1{path}",
        params=params or {},
        headers={"Authorization": f"Bearer {TOKEN}"},
        timeout=30,
    )
    r.raise_for_status()
    return r.json()

step2.js

const MAGENTO_URL = (process.env.MAGENTO_URL || "").replace(/\/$/, "");
const TOKEN = process.env.MAGENTO_ADMIN_TOKEN;

async function magentoGet(path, params = {}) {
  const url = new URL(`${MAGENTO_URL}/rest/V1${path}`);
  for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
  const res = await fetch(url, { headers: { Authorization: `Bearer ${TOKEN}` } });
  if (!res.ok) throw new Error(`Magento ${res.status}`);
  return res.json();
}

Pull the entity source of truth, then the grid-backed view per id

First page through GET /rest/V1/orders filtered by updated_at gteq <since>, collecting entity_id, increment_id, status, updated_at, and grand_total for every item, since this always reads sales_order. Then for each id, call GET /rest/V1/orders/{id} for the direct entity read, and separately query the list endpoint filtered by entity_id eq to see what the grid-backed admin list surfaces for that same id. Any id missing from that filtered result, or disagreeing on status or total, is a candidate for drift.

step3.py

def entity_orders_since(since, page_size=200, current_page=1):
    params = {
        "searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
        "searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
        "searchCriteria[filterGroups][0][filters][0][value]": since,
        "searchCriteria[pageSize]": page_size,
        "searchCriteria[currentPage]": current_page,
    }
    return magento_get("/orders", params)["items"]


def grid_view_for_id(entity_id):
    params = {
        "searchCriteria[filterGroups][0][filters][0][field]": "entity_id",
        "searchCriteria[filterGroups][0][filters][0][conditionType]": "eq",
        "searchCriteria[filterGroups][0][filters][0][value]": entity_id,
    }
    items = magento_get("/orders", params)["items"]
    return items[0] if items else None

step3.js

async function entityOrdersSince(since, pageSize = 200, currentPage = 1) {
  const params = {
    "searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
    "searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
    "searchCriteria[filterGroups][0][filters][0][value]": since,
    "searchCriteria[pageSize]": pageSize,
    "searchCriteria[currentPage]": currentPage,
  };
  const data = await magentoGet("/orders", params);
  return data.items;
}

async function gridViewForId(entityId) {
  const params = {
    "searchCriteria[filterGroups][0][filters][0][field]": "entity_id",
    "searchCriteria[filterGroups][0][filters][0][conditionType]": "eq",
    "searchCriteria[filterGroups][0][filters][0][value]": entityId,
  };
  const data = await magentoGet("/orders", params);
  return data.items[0] || null;
}

Decide, with one pure function

Keep the decision in its own function that takes the entity row, the grid row, or null, and the current watermark, and returns a drift type plus an action. A pure function like this is easy to read and easy to test, which we do later. A null grid row older than the watermark is the watermark-race signature, an order old enough that it should have synced but never did. A null grid row newer than the watermark is simply not due yet. A present grid row that disagrees on status or total is stale and needs the same reindex.

decide.py

def classify_order_sync(entity_row, grid_row, watermark):
    entity_id = entity_row["entityId"]

    if grid_row is None:
        if entity_row["updatedAt"] <= watermark:
            return {"entityId": entity_id, "driftType": "MISSING_FROM_GRID", "action": "FLAG_REINDEX"}
        return {"entityId": entity_id, "driftType": "OK", "action": "NONE"}

    if grid_row["status"] != entity_row["status"]:
        return {"entityId": entity_id, "driftType": "STALE_STATUS", "action": "FLAG_REINDEX"}

    if grid_row["grandTotal"] != entity_row["grandTotal"]:
        return {"entityId": entity_id, "driftType": "STALE_TOTAL", "action": "FLAG_REINDEX"}

    return {"entityId": entity_id, "driftType": "OK", "action": "NONE"}

decide.js

export function classifyOrderSync(entityRow, gridRow, watermark) {
  const entityId = entityRow.entityId;

  if (gridRow === null) {
    if (entityRow.updatedAt <= watermark) {
      return { entityId, driftType: "MISSING_FROM_GRID", action: "FLAG_REINDEX" };
    }
    return { entityId, driftType: "OK", action: "NONE" };
  }

  if (gridRow.status !== entityRow.status) {
    return { entityId, driftType: "STALE_STATUS", action: "FLAG_REINDEX" };
  }

  if (gridRow.grandTotal !== entityRow.grandTotal) {
    return { entityId, driftType: "STALE_TOTAL", action: "FLAG_REINDEX" };
  }

  return { entityId, driftType: "OK", action: "NONE" };
}

Report by default, never fake a repair

The output is a structured report per drifted entity_id, its increment_id, last-known-good entity fields, and the drift type. There is no code path in this script that touches sales_order_grid, because there is no safe REST way to do that. The recommended repair is always bin/magento indexer:reindex sales_order_grid for exactly the flagged ids, or the admin Update by Schedule cron.

An explicit, opt-in nudge, off by default

Only if DRY_RUN=false and an explicit allowlist of ids is supplied does the script attempt a benign nudge: a no-op PUT /V1/orders/{id}/comments with a system comment, which bumps updated_at and lets the async cron re-pick the row on its next cycle. This is logged, rate-limited, and skipped by default, since the correct fix is still the CLI reindex.

Run it safe

This script never writes to sales_order_grid and never marks an order paid, shipped, or invoiced. DRY_RUN defaults to true and only report mode ever runs unattended. The nudge path requires both DRY_RUN=false and an explicit id allowlist, and even then it only adds a system comment, nothing else.

The full code

Here is the complete script in one file for each language. It reads settings from the environment, pulls the entity source of truth, cross checks the grid-backed view per id, classifies drift with the pure function, and prints a structured report. It never writes to sales_order_grid, so it is safe to run again and again.

Get this script on GitHub Follow @allanninal Python and Node.js, with tests. Dry run by default. One of 59 Magento fixes, free and open source.

flag_grid_sync_drift.py

"""Flag Magento 2 sales_order_grid rows that fell out of sync with sales_order.

With asynchronous grid indexing (dev/grid/async_indexing=1), orders are written
to sales_order immediately but only copied into sales_order_grid by a scheduled
cron job bounded by a cached watermark on updated_at. A documented race
(magento/magento2 issue #40803) lets a cron run advance that watermark past an
order whose grid row write was still in flight or failed, permanently skipping
it. sales_order_grid has no REST endpoint and there is no public API to force a
single order's grid row rebuild, so this only reports the drift. Run on a
schedule. Safe to run again and again.
"""
import os
import logging
import requests

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("flag_grid_sync_drift")

MAGENTO_URL = os.environ["MAGENTO_URL"].rstrip("/")
TOKEN = os.environ["MAGENTO_ADMIN_TOKEN"]
SYNC_SINCE = os.environ.get("SYNC_SINCE", "2026-01-01 00:00:00")
WATERMARK = os.environ.get("WATERMARK", "2026-01-01 00:00:00")
DRY_RUN = os.environ.get("DRY_RUN", "true").lower() == "true"
NUDGE_ALLOWLIST = {
    s.strip() for s in os.environ.get("NUDGE_ALLOWLIST", "").split(",") if s.strip()
}


def magento_get(path, params=None):
    r = requests.get(
        f"{MAGENTO_URL}/rest/V1{path}",
        params=params or {},
        headers={"Authorization": f"Bearer {TOKEN}"},
        timeout=30,
    )
    r.raise_for_status()
    return r.json()


def magento_put(path, payload):
    r = requests.put(
        f"{MAGENTO_URL}/rest/V1{path}",
        json=payload,
        headers={"Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json"},
        timeout=30,
    )
    r.raise_for_status()
    return r.json()


def entity_orders_since(since, page_size=200, current_page=1):
    params = {
        "searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
        "searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
        "searchCriteria[filterGroups][0][filters][0][value]": since,
        "searchCriteria[pageSize]": page_size,
        "searchCriteria[currentPage]": current_page,
    }
    return magento_get("/orders", params)["items"]


def grid_view_for_id(entity_id):
    params = {
        "searchCriteria[filterGroups][0][filters][0][field]": "entity_id",
        "searchCriteria[filterGroups][0][filters][0][conditionType]": "eq",
        "searchCriteria[filterGroups][0][filters][0][value]": entity_id,
    }
    items = magento_get("/orders", params)["items"]
    return items[0] if items else None


def normalize_order(item):
    return {
        "entityId": item.get("entity_id"),
        "incrementId": item.get("increment_id"),
        "status": item.get("status"),
        "updatedAt": item.get("updated_at"),
        "grandTotal": item.get("grand_total"),
    }


def classify_order_sync(entity_row, grid_row, watermark):
    entity_id = entity_row["entityId"]

    if grid_row is None:
        if entity_row["updatedAt"] <= watermark:
            return {"entityId": entity_id, "driftType": "MISSING_FROM_GRID", "action": "FLAG_REINDEX"}
        return {"entityId": entity_id, "driftType": "OK", "action": "NONE"}

    if grid_row["status"] != entity_row["status"]:
        return {"entityId": entity_id, "driftType": "STALE_STATUS", "action": "FLAG_REINDEX"}

    if grid_row["grandTotal"] != entity_row["grandTotal"]:
        return {"entityId": entity_id, "driftType": "STALE_TOTAL", "action": "FLAG_REINDEX"}

    return {"entityId": entity_id, "driftType": "OK", "action": "NONE"}


def nudge_order(entity_id):
    """No-op comment PUT that bumps updated_at so async cron can re-pick the row."""
    payload = {
        "statusHistory": {
            "comment": "Reconciler: no-op comment to refresh updated_at for grid re-sync.",
            "isCustomerNotified": False,
            "isVisibleOnFront": False,
        }
    }
    return magento_put(f"/orders/{entity_id}/comments", payload)


def run():
    raw_entities = entity_orders_since(SYNC_SINCE)
    entities = [normalize_order(item) for item in raw_entities]

    drifted = []
    for entity_row in entities:
        raw_grid = grid_view_for_id(entity_row["entityId"])
        grid_row = normalize_order(raw_grid) if raw_grid else None
        result = classify_order_sync(entity_row, grid_row, WATERMARK)
        if result["action"] == "FLAG_REINDEX":
            drifted.append({**result, "incrementId": entity_row["incrementId"], "lastKnownGood": entity_row})

    for d in drifted:
        log.warning(
            "Order %s (id %s) drifted: %s.",
            d["incrementId"], d["entityId"], d["driftType"],
        )

    if drifted:
        ids = ",".join(str(d["entityId"]) for d in drifted)
        log.error(
            "%d order(s) out of sync with sales_order_grid. Run: "
            "bin/magento indexer:reindex sales_order_grid  (affected ids: %s)",
            len(drifted), ids,
        )
    else:
        log.info("Done. No drift found between sales_order and sales_order_grid.")

    if not DRY_RUN and NUDGE_ALLOWLIST:
        for d in drifted:
            if str(d["entityId"]) in NUDGE_ALLOWLIST:
                log.warning("Nudging order %s to bump updated_at for re-sync.", d["entityId"])
                nudge_order(d["entityId"])


if __name__ == "__main__":
    run()

flag-grid-sync-drift.js

/**
 * Flag Magento 2 sales_order_grid rows that fell out of sync with sales_order.
 *
 * With asynchronous grid indexing (dev/grid/async_indexing=1), orders are written
 * to sales_order immediately but only copied into sales_order_grid by a scheduled
 * cron job bounded by a cached watermark on updated_at. A documented race
 * (magento/magento2 issue #40803) lets a cron run advance that watermark past an
 * order whose grid row write was still in flight or failed, permanently skipping
 * it. sales_order_grid has no REST endpoint and there is no public API to force a
 * single order's grid row rebuild, so this only reports the drift. Run on a
 * schedule. Safe to run again and again.
 *
 * Guide: https://www.allanninal.dev/magento/sales-order-grid-out-of-sync/
 */
import { pathToFileURL } from "node:url";

const MAGENTO_URL = (process.env.MAGENTO_URL || "https://demo.example.com").replace(/\/$/, "");
const TOKEN = process.env.MAGENTO_ADMIN_TOKEN || "token_dummy";
const SYNC_SINCE = process.env.SYNC_SINCE || "2026-01-01 00:00:00";
const WATERMARK = process.env.WATERMARK || "2026-01-01 00:00:00";
const DRY_RUN = (process.env.DRY_RUN || "true").toLowerCase() === "true";
const NUDGE_ALLOWLIST = new Set(
  (process.env.NUDGE_ALLOWLIST || "").split(",").map((s) => s.trim()).filter(Boolean)
);

export function classifyOrderSync(entityRow, gridRow, watermark) {
  const entityId = entityRow.entityId;

  if (gridRow === null) {
    if (entityRow.updatedAt <= watermark) {
      return { entityId, driftType: "MISSING_FROM_GRID", action: "FLAG_REINDEX" };
    }
    return { entityId, driftType: "OK", action: "NONE" };
  }

  if (gridRow.status !== entityRow.status) {
    return { entityId, driftType: "STALE_STATUS", action: "FLAG_REINDEX" };
  }

  if (gridRow.grandTotal !== entityRow.grandTotal) {
    return { entityId, driftType: "STALE_TOTAL", action: "FLAG_REINDEX" };
  }

  return { entityId, driftType: "OK", action: "NONE" };
}

async function magentoGet(path, params = {}) {
  const url = new URL(`${MAGENTO_URL}/rest/V1${path}`);
  for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
  const res = await fetch(url, { headers: { Authorization: `Bearer ${TOKEN}` } });
  if (!res.ok) throw new Error(`Magento ${res.status}`);
  return res.json();
}

async function magentoPut(path, payload) {
  const res = await fetch(`${MAGENTO_URL}/rest/V1${path}`, {
    method: "PUT",
    headers: { Authorization: `Bearer ${TOKEN}`, "Content-Type": "application/json" },
    body: JSON.stringify(payload),
  });
  if (!res.ok) throw new Error(`Magento ${res.status}`);
  return res.json();
}

async function entityOrdersSince(since, pageSize = 200, currentPage = 1) {
  const params = {
    "searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
    "searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
    "searchCriteria[filterGroups][0][filters][0][value]": since,
    "searchCriteria[pageSize]": pageSize,
    "searchCriteria[currentPage]": currentPage,
  };
  const data = await magentoGet("/orders", params);
  return data.items;
}

async function gridViewForId(entityId) {
  const params = {
    "searchCriteria[filterGroups][0][filters][0][field]": "entity_id",
    "searchCriteria[filterGroups][0][filters][0][conditionType]": "eq",
    "searchCriteria[filterGroups][0][filters][0][value]": entityId,
  };
  const data = await magentoGet("/orders", params);
  return data.items[0] || null;
}

function normalizeOrder(item) {
  return {
    entityId: item.entity_id,
    incrementId: item.increment_id,
    status: item.status,
    updatedAt: item.updated_at,
    grandTotal: item.grand_total,
  };
}

async function nudgeOrder(entityId) {
  const payload = {
    statusHistory: {
      comment: "Reconciler: no-op comment to refresh updated_at for grid re-sync.",
      isCustomerNotified: false,
      isVisibleOnFront: false,
    },
  };
  return magentoPut(`/orders/${entityId}/comments`, payload);
}

export async function run() {
  const rawEntities = await entityOrdersSince(SYNC_SINCE);
  const entities = rawEntities.map(normalizeOrder);

  const drifted = [];
  for (const entityRow of entities) {
    const rawGrid = await gridViewForId(entityRow.entityId);
    const gridRow = rawGrid ? normalizeOrder(rawGrid) : null;
    const result = classifyOrderSync(entityRow, gridRow, WATERMARK);
    if (result.action === "FLAG_REINDEX") {
      drifted.push({ ...result, incrementId: entityRow.incrementId, lastKnownGood: entityRow });
    }
  }

  for (const d of drifted) {
    console.warn(`Order ${d.incrementId} (id ${d.entityId}) drifted: ${d.driftType}.`);
  }

  if (drifted.length) {
    const ids = drifted.map((d) => d.entityId).join(",");
    console.error(
      `${drifted.length} order(s) out of sync with sales_order_grid. Run: ` +
      `bin/magento indexer:reindex sales_order_grid  (affected ids: ${ids})`
    );
  } else {
    console.log("Done. No drift found between sales_order and sales_order_grid.");
  }

  if (!DRY_RUN && NUDGE_ALLOWLIST.size) {
    for (const d of drifted) {
      if (NUDGE_ALLOWLIST.has(String(d.entityId))) {
        console.warn(`Nudging order ${d.entityId} to bump updated_at for re-sync.`);
        await nudgeOrder(d.entityId);
      }
    }
  }
}

if (import.meta.url === pathToFileURL(process.argv[1]).href) {
  run().catch((err) => { console.error(err); process.exit(1); });
}

Add a test

The classification rule is the part most worth testing, because it decides whether an order gets flagged for reindex. Because we kept classify_order_sync pure, the test needs no network and no Magento store. It just feeds in plain fixture rows for each branch and checks the answer.

test_grid_sync_classify.py

from flag_grid_sync_drift import classify_order_sync

WATERMARK = "2026-07-10 00:00:00"


def entity(**over):
    base = {
        "entityId": 501,
        "incrementId": "100000501",
        "status": "processing",
        "updatedAt": "2026-07-09 12:00:00",
        "grandTotal": 129.99,
    }
    base.update(over)
    return base


def grid(**over):
    base = {
        "entityId": 501,
        "incrementId": "100000501",
        "status": "processing",
        "updatedAt": "2026-07-09 12:00:00",
        "grandTotal": 129.99,
    }
    base.update(over)
    return base


def test_missing_and_due_is_flagged():
    result = classify_order_sync(entity(), None, WATERMARK)
    assert result["driftType"] == "MISSING_FROM_GRID"
    assert result["action"] == "FLAG_REINDEX"


def test_missing_but_not_due_is_ok():
    result = classify_order_sync(entity(updatedAt="2026-07-10 08:00:00"), None, WATERMARK)
    assert result["driftType"] == "OK"
    assert result["action"] == "NONE"


def test_status_drift_is_flagged():
    result = classify_order_sync(entity(), grid(status="pending"), WATERMARK)
    assert result["driftType"] == "STALE_STATUS"
    assert result["action"] == "FLAG_REINDEX"


def test_total_drift_is_flagged():
    result = classify_order_sync(entity(), grid(grandTotal=89.99), WATERMARK)
    assert result["driftType"] == "STALE_TOTAL"
    assert result["action"] == "FLAG_REINDEX"


def test_matched_rows_are_ok():
    result = classify_order_sync(entity(), grid(), WATERMARK)
    assert result["driftType"] == "OK"
    assert result["action"] == "NONE"


def test_entity_id_is_preserved_in_result():
    result = classify_order_sync(entity(entityId=777), None, WATERMARK)
    assert result["entityId"] == 777


def test_exactly_at_watermark_is_flagged():
    result = classify_order_sync(entity(updatedAt=WATERMARK), None, WATERMARK)
    assert result["driftType"] == "MISSING_FROM_GRID"

grid-sync-classify.test.js

import { test } from "node:test";
import assert from "node:assert/strict";
import { classifyOrderSync } from "./flag-grid-sync-drift.js";

const WATERMARK = "2026-07-10 00:00:00";

const entity = (over = {}) => ({
  entityId: 501,
  incrementId: "100000501",
  status: "processing",
  updatedAt: "2026-07-09 12:00:00",
  grandTotal: 129.99,
  ...over,
});

const grid = (over = {}) => ({
  entityId: 501,
  incrementId: "100000501",
  status: "processing",
  updatedAt: "2026-07-09 12:00:00",
  grandTotal: 129.99,
  ...over,
});

test("missing and due is flagged", () => {
  const result = classifyOrderSync(entity(), null, WATERMARK);
  assert.equal(result.driftType, "MISSING_FROM_GRID");
  assert.equal(result.action, "FLAG_REINDEX");
});

test("missing but not due is ok", () => {
  const result = classifyOrderSync(entity({ updatedAt: "2026-07-10 08:00:00" }), null, WATERMARK);
  assert.equal(result.driftType, "OK");
  assert.equal(result.action, "NONE");
});

test("status drift is flagged", () => {
  const result = classifyOrderSync(entity(), grid({ status: "pending" }), WATERMARK);
  assert.equal(result.driftType, "STALE_STATUS");
  assert.equal(result.action, "FLAG_REINDEX");
});

test("total drift is flagged", () => {
  const result = classifyOrderSync(entity(), grid({ grandTotal: 89.99 }), WATERMARK);
  assert.equal(result.driftType, "STALE_TOTAL");
  assert.equal(result.action, "FLAG_REINDEX");
});

test("matched rows are ok", () => {
  const result = classifyOrderSync(entity(), grid(), WATERMARK);
  assert.equal(result.driftType, "OK");
  assert.equal(result.action, "NONE");
});

test("entity id is preserved in result", () => {
  const result = classifyOrderSync(entity({ entityId: 777 }), null, WATERMARK);
  assert.equal(result.entityId, 777);
});

test("exactly at watermark is flagged", () => {
  const result = classifyOrderSync(entity({ updatedAt: WATERMARK }), null, WATERMARK);
  assert.equal(result.driftType, "MISSING_FROM_GRID");
});

Case studies

Missing from grid

The order support could not find

A mid-size store had asynchronous grid indexing on for months without issue, until a spike in checkout traffic overlapped with a deployment restart. A handful of orders placed in that narrow window were confirmed by the payment gateway and by customer emails, but simply were not in the admin grid days later. Support searched by increment id and found nothing, then escalated it as a lost order.

The reconciliation job, run against the entity source of truth and cross checked per id, flagged exactly those orders as MISSING_FROM_GRID, each one older than the watermark and never present in the grid-backed list. A targeted bin/magento indexer:reindex sales_order_grid brought every one of them back into the grid within minutes, with no data loss, since sales_order had been correct the whole time.

Stale status

The grid that stopped believing an order shipped

An order moved from processing to complete after its shipment was created, and the REST API reflected that immediately. The admin grid, though, kept showing it as processing for over a week, which meant it kept appearing on a report of orders awaiting fulfillment that a warehouse team checked every morning.

The script's per-id cross check caught the mismatch as STALE_STATUS, comparing the grid row's status against the direct entity read for the same id. Reindexing the flagged id resolved it, and the team added the job as a daily check so a stale status in the grid never sits unnoticed for a week again.

What good looks like

After this runs on a schedule, a permanently skipped or stale grid row is caught within one polling cycle instead of surviving as a support mystery. The report carries the exact entity_id values affected, the drift type, and the last-known-good entity fields, so whoever responds can run bin/magento indexer:reindex sales_order_grid for precisely those ids rather than a full store-wide reindex or a guess.

FAQ

Why does an order show correctly in the API but wrong or missing in the Magento admin grid?

The admin order grid reads from a separate table, sales_order_grid, that is only kept in sync with the real sales_order table by a scheduled cron job when asynchronous grid indexing is enabled. A documented race condition lets that cron job advance its sync watermark past an order's updated_at while that order's grid row write is still in flight or fails, and once the watermark moves past it, the order is never picked up again. sales_order itself is always correct because the REST API and checkout write to it directly.

Can I fix a missing or stale sales_order_grid row through the REST API?

No. There is no public REST endpoint that rebuilds a single order's grid row. The real fix is a CLI command, bin/magento indexer:reindex sales_order_grid, or triggering the admin Update by Schedule cron. A script can only detect the drift by comparing sales_order against what the grid-backed list endpoint returns, and then report which entity_id values need that reindex.

What is the watermark race that causes orders to permanently disappear from the grid?

Each async grid indexing cron run reads a cached watermark, syncs sales_order rows updated since that point, and then advances the watermark. If one order's grid row write is still in flight, fails, or the cron is killed mid batch, but the cron run still advances the watermark past that order's updated_at, the order's timestamp is now permanently below the new floor. Every later cron cycle only looks forward from the watermark, so that order is skipped forever unless it is updated again or reindexed directly.

Related field notes

Citations

On the problem:

GitHub Issue: bug, async sales order grid sync permanently skips orders due to watermark race condition. github.com/magento/magento2/issues/40803
GitHub Issue: fix best practice "Asynchronous order data processing" for auto invoiced orders. github.com/magento/magento2/issues/36334
Magento Forums: some orders not appearing in order grid. community.magento.com some orders not appearing in order grid

On the solution:

Adobe Commerce: configuration best practices for order processing. experienceleague.adobe.com order processing configuration
Adobe Commerce: manage the indexers. experienceleague.adobe.com manage indexers
Adobe Commerce: search using REST endpoints. developer.adobe.com performing searches

Stuck on a tricky one?

If you have a problem in Magento 2 or Adobe Commerce orders, cron, catalog data, or inventory that you would rather hand off, this is the kind of work I do. Message me and we can work through it together.

Contact me on LinkedIn

Did this find an order support could not?

If this saved you a confusing support escalation or a full store-wide reindex you did not need, you can buy me a coffee. It is the best way to keep these field notes free and growing.

Buy me a coffee on Ko-fi

Back to all Magento field notes