Diagnostic Orders and Grid Sync
Grid refresh by schedule caps at batch size, backlog grows
The admin order grid is missing rows that clearly exist in the orders table. Same story on the invoice, shipment, and credit memo grids. Nothing crashed, nothing errored, and the cron job that syncs them is running on schedule exactly as configured. The problem is quieter than that: the sync query itself never asks for more than 100 rows at a time, so when more than 100 orders fall out of sync between ticks, the extra rows are simply invisible to that run. Here is why the batch cap leaks a backlog instead of draining it, and a small script that watches the symptom from the REST API and reports it.
Magento's Update by Schedule grid sync for sales_order_grid, sales_invoice_grid, sales_shipment_grid, and sales_creditmemo_grid runs through Magento\Sales\Model\ResourceModel\Grid::refreshBySchedule(). It asks a provider for the list of not-yet-synced entity ids, then chunks that list with array_chunk() into batches of Grid::BATCH_SIZE, which is 100. The catch is that the provider's own SQL select carries a LIMIT equal to that same 100, so the query never returns more than 100 candidate ids in the first place, no matter how many rows are actually out of sync. During a bulk order import or a high-traffic sale, the backlog can grow faster than 100 rows per tick drains it. There is no REST endpoint to force a resync or remove the cap, so a script cannot repair this over the API. It can, however, poll /rest/V1/orders for the stream of recently updated orders, spot the signature of the batch cap, the count of orders updated since the last poll staying at or above 100 for consecutive runs, and report the affected increment_ids for a human to reconcile with bin/magento indexer:reindex sales_order_grid or a re-run of the grid sync cron. Full code, tests, and a dry run guard are below.
The problem in plain words
The sales_order_grid table (and its invoice, shipment, and credit memo siblings) is a flattened, admin-friendly copy of order data, kept separate from the normalized sales_order table so the admin grid can search and sort quickly. When you set grid indexing to Update on Schedule, Magento does not rebuild that copy the instant an order changes. Instead a cron job periodically asks, in effect, "which entity ids have changed since I last looked," then writes just those rows into the grid table.
That question is answered by UpdatedIdListProvider and its IdListBuilder, and the SQL select behind it carries a LIMIT that happens to equal Grid::BATCH_SIZE, 100. refreshBySchedule() then takes whatever the provider returned and slices it into chunks of 100 with array_chunk() for processing. On a quiet store that is invisible, since there are rarely more than 100 unsynced rows waiting at once. But the moment a bulk import lands, a flash sale generates a burst of orders, or the grid table gets truncated and needs to rebuild from scratch, more than 100 rows fall out of sync between cron ticks. The provider's query caps at 100 candidates no matter how many actually need syncing, so each scheduled run drains at most 100 rows and the remainder is entirely invisible to that run. The backlog does not shrink proportionally to the real gap, it just leaks out 100 rows per tick, and during sustained high volume it can grow faster than it drains.
Why it happens
- A bulk order import, through the Admin API, a migration tool, or a data feed, that creates or updates far more than 100 orders between cron ticks.
- A high-traffic sales event where checkout volume outpaces the grid sync cron's 100-row-per-tick draining rate.
- A
sales_order_gridtruncate or manual rebuild that leaves the entire order history unsynced at once, so the first several cron ticks each only claw back 100 rows. - The provider's SQL select (in
UpdatedIdListProvider/IdListBuilder) carries aLIMITthat matchesGrid::BATCH_SIZE, so the cap exists one layer before thearray_chunk()most people assume is the only limit.
This is tracked upstream as magento/magento2#39602, describing refreshBySchedule not working properly once the backlog exceeds the batch size, with a follow-up issue #39621 and a proposed fix in PR #39603 to remove the limit from the id-list query. A related performance issue, #40282, covers the same async order grid processing path. See the citations at the end for the specific threads.
The grid tables and the changelog behind them are database-internal; there is no REST resource that exposes sales_order_grid, so a script cannot query the sync backlog directly. What it can do is watch the one thing REST does expose: the stream of updated_at timestamps on /rest/V1/orders. A healthy cron drains its whole delta every run, so the count of orders updated since the previous poll should fall back under 100 once it catches up. A capped refreshBySchedule instead leaves that count at or above 100 run after run, because the underlying query can never return more than 100 candidates no matter how many are actually pending. That pattern, not a single high reading but a streak of them, is the detectable signature.
The fix, as a flow
We do not touch the grid tables or the sync cron directly, both are out of reach for a REST-only client. We add a job that polls the orders endpoint on an interval, records how many orders changed since the last poll, and feeds that history into a pure classifier. When the classifier sees two or more consecutive polls at or above the batch size, it reports a suspected backlog with the affected order ids, for an operator to run the real repair.
Build it step by step
Get an admin bearer token
The script authenticates like any other Magento REST client. Either call POST /rest/V1/integration/admin/token with an admin username and password, or create an integration and use its token directly. Keep the store URL and token in environment variables, never in the file.
pip install requests
export MAGENTO_URL="https://your-store.example.com"
export MAGENTO_ADMIN_TOKEN="eyJraWQ..."
export GRID_BATCH_SIZE="100"
export CONSECUTIVE_THRESHOLD="2"
export POLL_WINDOW_MINUTES="15"
export DRY_RUN="true" # start safe, this script only reports either way// Node 18+ has fetch built in, no dependencies needed
export MAGENTO_URL="https://your-store.example.com"
export MAGENTO_ADMIN_TOKEN="eyJraWQ..."
export GRID_BATCH_SIZE="100"
export CONSECUTIVE_THRESHOLD="2"
export POLL_WINDOW_MINUTES="15"
export DRY_RUN="true" // start safe, this script only reports either wayTalk to the Magento REST API
Every call sends Authorization: Bearer <token> to a /rest/V1 route. A small helper wraps the request and raises on a non 200 response, since Magento returns structured error bodies worth surfacing as is.
import os, requests
MAGENTO_URL = os.environ["MAGENTO_URL"].rstrip("/")
TOKEN = os.environ["MAGENTO_ADMIN_TOKEN"]
def magento_get(path, params=None):
r = requests.get(
f"{MAGENTO_URL}/rest/V1{path}",
params=params or {},
headers={"Authorization": f"Bearer {TOKEN}"},
timeout=30,
)
r.raise_for_status()
return r.json()const MAGENTO_URL = (process.env.MAGENTO_URL || "").replace(/\/$/, "");
const TOKEN = process.env.MAGENTO_ADMIN_TOKEN;
async function magentoGet(path, params = {}) {
const url = new URL(`${MAGENTO_URL}/rest/V1${path}`);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
const res = await fetch(url, { headers: { Authorization: `Bearer ${TOKEN}` } });
if (!res.ok) throw new Error(`Magento ${res.status}`);
return res.json();
}Poll orders updated since the last checkpoint
Grid tables are not exposed over REST, so the script uses the closest proxy: how many orders report an updated_at newer than the last poll's checkpoint. Filter with conditionType=gteq on updated_at, page with pageSize and currentPage, and read total_count from the search envelope. Sorting by updated_at descending on a one-row page is a cheap way to get the newest timestamp to use as the next checkpoint.
def orders_updated_since(since_iso, page_size=200, current_page=1):
params = {
"searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
"searchCriteria[filterGroups][0][filters][0][value]": since_iso,
"searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
"searchCriteria[pageSize]": page_size,
"searchCriteria[currentPage]": current_page,
}
return magento_get("/orders", params)
def newest_order_updated_at():
params = {
"searchCriteria[pageSize]": 1,
"searchCriteria[currentPage]": 1,
"searchCriteria[sortOrders][0][field]": "updated_at",
"searchCriteria[sortOrders][0][direction]": "DESC",
}
items = magento_get("/orders", params)["items"]
return items[0]["updated_at"] if items else Noneasync function ordersUpdatedSince(sinceIso, pageSize = 200, currentPage = 1) {
const params = {
"searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
"searchCriteria[filterGroups][0][filters][0][value]": sinceIso,
"searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
"searchCriteria[pageSize]": pageSize,
"searchCriteria[currentPage]": currentPage,
};
return magentoGet("/orders", params);
}
async function newestOrderUpdatedAt() {
const params = {
"searchCriteria[pageSize]": 1,
"searchCriteria[currentPage]": 1,
"searchCriteria[sortOrders][0][field]": "updated_at",
"searchCriteria[sortOrders][0][direction]": "DESC",
};
const data = await magentoGet("/orders", params);
return data.items[0]?.updated_at || null;
}Decide, with one pure function
Keep the decision in its own function that takes the chronological poll history and the batch size, and returns whether a backlog is suspected. A pure function like this is easy to read and easy to test, which we do later. A healthy cron drains the whole delta each run, so the count should fall below the batch size once it catches up. A capped refreshBySchedule instead leaves the count at or above the batch size run after run, so we look for a streak of consecutive polls at or above that ceiling.
def classify_grid_sync_backlog(poll_history, batch_size=100, consecutive_threshold=2):
consecutive = 0
best_streak = 0
streak_excess = 0
best_excess = 0
for sample in poll_history:
count = sample["updatedSinceLastPollCount"]
if count >= batch_size:
consecutive += 1
streak_excess += count - batch_size
else:
consecutive = 0
streak_excess = 0
if consecutive >= best_streak:
best_streak = consecutive
best_excess = streak_excess
return {
"backlogSuspected": best_streak >= consecutive_threshold,
"consecutiveOverBatchRuns": best_streak,
"estimatedBacklogRows": best_excess if best_streak >= consecutive_threshold else 0,
}export function classifyGridSyncBacklog(pollHistory, batchSize = 100, consecutiveThreshold = 2) {
let consecutive = 0;
let bestStreak = 0;
let streakExcess = 0;
let bestExcess = 0;
for (const sample of pollHistory) {
const count = sample.updatedSinceLastPollCount;
if (count >= batchSize) {
consecutive += 1;
streakExcess += count - batchSize;
} else {
consecutive = 0;
streakExcess = 0;
}
if (consecutive >= bestStreak) {
bestStreak = consecutive;
bestExcess = streakExcess;
}
}
return {
backlogSuspected: bestStreak >= consecutiveThreshold,
consecutiveOverBatchRuns: bestStreak,
estimatedBacklogRows: bestStreak >= consecutiveThreshold ? bestExcess : 0,
};
}Report by default, never mutate order data to force a resync
There is no REST endpoint that forces a sales_order_grid sync or removes the BATCH_SIZE cap. Re-saving or touching orders through PUT /rest/V1/orders just to trigger a resync is not safe, since it risks unintended side effects on order state. So the script's write path is limited to alerting: it logs the suspected backlog, the streak length, the estimated row count, and the affected increment_id list, for a human or an ops pipeline to run the real repair, bin/magento indexer:reindex sales_order_grid, re-triggering the grid async-insert cron, or switching to Update on Save in Stores > Configuration > Advanced > Developer > Grid Settings.
DRY_RUN defaults to true and the script never writes to an order either way, since there is no safe REST-only repair for this issue. Treat a flagged backlog as a lead for the admin or CLI team to reconcile with bin/magento indexer:reindex sales_order_grid, not as something this script should attempt on its own.
The full code
Here is the complete script in one file for each language. It reads settings from the environment, polls the orders endpoint on each run, keeps a small in-memory poll history, classifies it, and logs a report. It never writes to Magento, because the only safe action available over REST here is detection.
"""Flag a Magento 2 sales_order_grid (and invoice/shipment/creditmemo grid)
sync backlog caused by the refreshBySchedule batch size cap, safely.
Magento's Update by Schedule grid sync asks a provider for not-yet-synced
entity ids, and that provider's own SQL select carries a LIMIT equal to
Grid::BATCH_SIZE (100). When more than 100 rows fall out of sync between
cron ticks (bulk import, a busy sale, a grid rebuild), each scheduled run
only ever drains 100 rows, and the backlog can grow faster than it shrinks.
The grid tables are database-internal and are not exposed over REST, so
this script infers the backlog from the REST-visible order updated_at
stream: it polls how many orders changed since the last checkpoint and
looks for a streak of consecutive polls at or above the batch size, which
is the signature of the batch cap rather than a merely busy cron.
This script never writes to Magento. There is no REST endpoint that forces
a grid resync or removes the batch cap, and touching orders through PUT
just to force a resync is not a safe workaround. It only reports. Run on a
schedule. Safe to run again and again.
"""
import os
import logging
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("flag_grid_sync_backlog")
MAGENTO_URL = os.environ["MAGENTO_URL"].rstrip("/")
TOKEN = os.environ["MAGENTO_ADMIN_TOKEN"]
GRID_BATCH_SIZE = int(os.environ.get("GRID_BATCH_SIZE", "100"))
CONSECUTIVE_THRESHOLD = int(os.environ.get("CONSECUTIVE_THRESHOLD", "2"))
DRY_RUN = os.environ.get("DRY_RUN", "true").lower() == "true"
def magento_get(path, params=None):
r = requests.get(
f"{MAGENTO_URL}/rest/V1{path}",
params=params or {},
headers={"Authorization": f"Bearer {TOKEN}"},
timeout=30,
)
r.raise_for_status()
return r.json()
def orders_updated_since(since_iso, page_size=200, current_page=1):
params = {
"searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
"searchCriteria[filterGroups][0][filters][0][value]": since_iso,
"searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
"searchCriteria[pageSize]": page_size,
"searchCriteria[currentPage]": current_page,
}
return magento_get("/orders", params)
def newest_order_updated_at():
params = {
"searchCriteria[pageSize]": 1,
"searchCriteria[currentPage]": 1,
"searchCriteria[sortOrders][0][field]": "updated_at",
"searchCriteria[sortOrders][0][direction]": "DESC",
}
items = magento_get("/orders", params)["items"]
return items[0]["updated_at"] if items else None
def classify_grid_sync_backlog(poll_history, batch_size=100, consecutive_threshold=2):
consecutive = 0
best_streak = 0
streak_excess = 0
best_excess = 0
for sample in poll_history:
count = sample["updatedSinceLastPollCount"]
if count >= batch_size:
consecutive += 1
streak_excess += count - batch_size
else:
consecutive = 0
streak_excess = 0
if consecutive >= best_streak:
best_streak = consecutive
best_excess = streak_excess
return {
"backlogSuspected": best_streak >= consecutive_threshold,
"consecutiveOverBatchRuns": best_streak,
"estimatedBacklogRows": best_excess if best_streak >= consecutive_threshold else 0,
}
def poll_once(since_iso, page_size=200):
data = orders_updated_since(since_iso, page_size=page_size)
increment_ids = [o.get("increment_id") for o in data.get("items", [])]
return {
"count": data.get("total_count", len(data.get("items", []))),
"incrementIds": increment_ids,
}
def run(checkpoint_iso=None, poll_history=None):
poll_history = poll_history if poll_history is not None else []
checkpoint = checkpoint_iso or newest_order_updated_at()
if not checkpoint:
log.info("No orders found. Nothing to poll yet.")
return poll_history
sample = poll_once(checkpoint)
poll_history.append({"timestampMs": 0, "updatedSinceLastPollCount": sample["count"]})
result = classify_grid_sync_backlog(poll_history, GRID_BATCH_SIZE, CONSECUTIVE_THRESHOLD)
if result["backlogSuspected"]:
log.warning(
"Suspected grid sync backlog: %d consecutive poll(s) at or above batch size %d, "
"estimated %d row(s) behind. Sample increment_ids: %s. %s",
result["consecutiveOverBatchRuns"], GRID_BATCH_SIZE,
result["estimatedBacklogRows"], sample["incrementIds"][:20],
"DRY_RUN, reporting only" if DRY_RUN else "reporting only, no auto-repair available over REST",
)
else:
log.info("Grid sync looks healthy. %d order(s) updated since checkpoint.", sample["count"])
return poll_history
if __name__ == "__main__":
run()/**
* Flag a Magento 2 sales_order_grid (and invoice/shipment/creditmemo grid)
* sync backlog caused by the refreshBySchedule batch size cap, safely.
*
* Magento's Update by Schedule grid sync asks a provider for not-yet-synced
* entity ids, and that provider's own SQL select carries a LIMIT equal to
* Grid::BATCH_SIZE (100). When more than 100 rows fall out of sync between
* cron ticks (bulk import, a busy sale, a grid rebuild), each scheduled run
* only ever drains 100 rows, and the backlog can grow faster than it
* shrinks. The grid tables are database-internal and are not exposed over
* REST, so this script infers the backlog from the REST-visible order
* updated_at stream: it polls how many orders changed since the last
* checkpoint and looks for a streak of consecutive polls at or above the
* batch size, which is the signature of the batch cap rather than a merely
* busy cron.
*
* This script never writes to Magento. There is no REST endpoint that
* forces a grid resync or removes the batch cap, and touching orders
* through PUT just to force a resync is not a safe workaround. It only
* reports. Run on a schedule. Safe to run again and again.
*
* Guide: https://www.allanninal.dev/magento/grid-refresh-batch-size-backlog/
*/
import { pathToFileURL } from "node:url";
const MAGENTO_URL = (process.env.MAGENTO_URL || "https://demo.example.com").replace(/\/$/, "");
const TOKEN = process.env.MAGENTO_ADMIN_TOKEN || "token_dummy";
const GRID_BATCH_SIZE = Number(process.env.GRID_BATCH_SIZE || 100);
const CONSECUTIVE_THRESHOLD = Number(process.env.CONSECUTIVE_THRESHOLD || 2);
const DRY_RUN = (process.env.DRY_RUN || "true").toLowerCase() === "true";
export function classifyGridSyncBacklog(pollHistory, batchSize = 100, consecutiveThreshold = 2) {
let consecutive = 0;
let bestStreak = 0;
let streakExcess = 0;
let bestExcess = 0;
for (const sample of pollHistory) {
const count = sample.updatedSinceLastPollCount;
if (count >= batchSize) {
consecutive += 1;
streakExcess += count - batchSize;
} else {
consecutive = 0;
streakExcess = 0;
}
if (consecutive >= bestStreak) {
bestStreak = consecutive;
bestExcess = streakExcess;
}
}
return {
backlogSuspected: bestStreak >= consecutiveThreshold,
consecutiveOverBatchRuns: bestStreak,
estimatedBacklogRows: bestStreak >= consecutiveThreshold ? bestExcess : 0,
};
}
async function magentoGet(path, params = {}) {
const url = new URL(`${MAGENTO_URL}/rest/V1${path}`);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
const res = await fetch(url, { headers: { Authorization: `Bearer ${TOKEN}` } });
if (!res.ok) throw new Error(`Magento ${res.status}`);
return res.json();
}
async function ordersUpdatedSince(sinceIso, pageSize = 200, currentPage = 1) {
const params = {
"searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
"searchCriteria[filterGroups][0][filters][0][value]": sinceIso,
"searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
"searchCriteria[pageSize]": pageSize,
"searchCriteria[currentPage]": currentPage,
};
return magentoGet("/orders", params);
}
async function newestOrderUpdatedAt() {
const params = {
"searchCriteria[pageSize]": 1,
"searchCriteria[currentPage]": 1,
"searchCriteria[sortOrders][0][field]": "updated_at",
"searchCriteria[sortOrders][0][direction]": "DESC",
};
const data = await magentoGet("/orders", params);
return data.items[0]?.updated_at || null;
}
async function pollOnce(sinceIso, pageSize = 200) {
const data = await ordersUpdatedSince(sinceIso, pageSize);
const items = data.items || [];
const incrementIds = items.map((o) => o.increment_id);
return { count: data.total_count ?? items.length, incrementIds };
}
export async function run(checkpointIso, pollHistory = []) {
const checkpoint = checkpointIso || (await newestOrderUpdatedAt());
if (!checkpoint) {
console.log("No orders found. Nothing to poll yet.");
return pollHistory;
}
const sample = await pollOnce(checkpoint);
pollHistory.push({ timestampMs: Date.now(), updatedSinceLastPollCount: sample.count });
const result = classifyGridSyncBacklog(pollHistory, GRID_BATCH_SIZE, CONSECUTIVE_THRESHOLD);
if (result.backlogSuspected) {
console.warn(
`Suspected grid sync backlog: ${result.consecutiveOverBatchRuns} consecutive poll(s) at or above batch size ${GRID_BATCH_SIZE}, ` +
`estimated ${result.estimatedBacklogRows} row(s) behind. Sample increment_ids: ${sample.incrementIds.slice(0, 20)}. ` +
`${DRY_RUN ? "DRY_RUN, reporting only" : "reporting only, no auto-repair available over REST"}`
);
} else {
console.log(`Grid sync looks healthy. ${sample.count} order(s) updated since checkpoint.`);
}
return pollHistory;
}
if (import.meta.url === pathToFileURL(process.argv[1]).href) {
run().catch((err) => { console.error(err); process.exit(1); });
}Add a test
The classification rule is the part most worth testing, because it decides whether the poll history looks like a healthy draining cron or a capped one leaking a backlog. Because we kept classify_grid_sync_backlog pure, the test needs no network and no Magento store. It just feeds in a fixed list of poll samples and checks the answer.
from flag_grid_sync_backlog import classify_grid_sync_backlog
def samples(*counts):
return [{"timestampMs": i, "updatedSinceLastPollCount": c} for i, c in enumerate(counts)]
def test_healthy_when_all_polls_under_batch_size():
result = classify_grid_sync_backlog(samples(10, 20, 5), 100, 2)
assert result["backlogSuspected"] is False
assert result["consecutiveOverBatchRuns"] == 0
assert result["estimatedBacklogRows"] == 0
def test_single_spike_is_not_enough():
result = classify_grid_sync_backlog(samples(150, 30, 10), 100, 2)
assert result["backlogSuspected"] is False
def test_two_consecutive_over_batch_size_flags_backlog():
result = classify_grid_sync_backlog(samples(120, 140), 100, 2)
assert result["backlogSuspected"] is True
assert result["consecutiveOverBatchRuns"] == 2
assert result["estimatedBacklogRows"] == 20 + 40
def test_streak_resets_after_a_healthy_poll():
result = classify_grid_sync_backlog(samples(150, 40, 150, 160), 100, 2)
assert result["backlogSuspected"] is True
assert result["consecutiveOverBatchRuns"] == 2
assert result["estimatedBacklogRows"] == 50 + 60
def test_exactly_at_batch_size_counts_as_over():
result = classify_grid_sync_backlog(samples(100, 100), 100, 2)
assert result["backlogSuspected"] is True
assert result["consecutiveOverBatchRuns"] == 2
def test_empty_history_is_healthy():
result = classify_grid_sync_backlog([], 100, 2)
assert result["backlogSuspected"] is False
assert result["consecutiveOverBatchRuns"] == 0import { test } from "node:test";
import assert from "node:assert/strict";
import { classifyGridSyncBacklog } from "./flag-grid-sync-backlog.js";
const samples = (...counts) => counts.map((c, i) => ({ timestampMs: i, updatedSinceLastPollCount: c }));
test("healthy when all polls under batch size", () => {
const result = classifyGridSyncBacklog(samples(10, 20, 5), 100, 2);
assert.equal(result.backlogSuspected, false);
assert.equal(result.consecutiveOverBatchRuns, 0);
assert.equal(result.estimatedBacklogRows, 0);
});
test("single spike is not enough", () => {
const result = classifyGridSyncBacklog(samples(150, 30, 10), 100, 2);
assert.equal(result.backlogSuspected, false);
});
test("two consecutive over batch size flags backlog", () => {
const result = classifyGridSyncBacklog(samples(120, 140), 100, 2);
assert.equal(result.backlogSuspected, true);
assert.equal(result.consecutiveOverBatchRuns, 2);
assert.equal(result.estimatedBacklogRows, 20 + 40);
});
test("streak resets after a healthy poll", () => {
const result = classifyGridSyncBacklog(samples(150, 40, 150, 160), 100, 2);
assert.equal(result.backlogSuspected, true);
assert.equal(result.consecutiveOverBatchRuns, 2);
assert.equal(result.estimatedBacklogRows, 50 + 60);
});
test("exactly at batch size counts as over", () => {
const result = classifyGridSyncBacklog(samples(100, 100), 100, 2);
assert.equal(result.backlogSuspected, true);
assert.equal(result.consecutiveOverBatchRuns, 2);
});
test("empty history is healthy", () => {
const result = classifyGridSyncBacklog([], 100, 2);
assert.equal(result.backlogSuspected, false);
assert.equal(result.consecutiveOverBatchRuns, 0);
});Case studies
The migration that quietly hid three days of orders
A merchant migrating from another platform imported 4,000 historical orders in one overnight batch. The orders table filled up fine, but the admin order grid only ever showed 100 new rows per hour, matching the grid sync cron's tick. Support staff spent two days insisting orders were missing before anyone thought to check the raw order count against the grid.
Running the poller against /rest/V1/orders would have shown the pattern in the first two ticks: order counts updated since the last checkpoint sitting well above 100 run after run, instead of tapering off. The team ended up running bin/magento indexer:reindex sales_order_grid as a full rebuild, which cleared the backlog in one pass instead of waiting on the scheduled cron to leak it out 100 rows at a time.
The sale that outran its own grid sync
A flash sale pushed order volume to roughly 400 orders an hour for a six hour window. The grid sync cron, ticking every few minutes but capped at 100 ids per run by the provider's own query, never caught up in real time. Invoices for orders placed early in the sale did not show up in the admin invoice grid until well after the sale ended.
The team added the poller as a lightweight early warning ahead of future sales. Seeing two or more consecutive polls read at or above 100 updated orders became the trigger to manually re-run the grid async-insert cron more frequently during the peak window, instead of discovering the lag from a support ticket the next morning.
After this runs on a schedule, a grid sync backlog is caught within a couple of polling intervals instead of surviving silently until someone notices missing rows in the admin. The report carries the streak length, an estimated row count, and a sample of affected increment_ids, so whoever responds has evidence in hand before reaching for bin/magento indexer:reindex sales_order_grid. Keep the write path limited to alerting, since there is no safe way to force a grid resync through the REST API alone.
FAQ
Why does my sales_order_grid fall behind the orders table?
Magento's Update by Schedule grid sync reads not-yet-synced ids with a query that itself carries a LIMIT equal to Grid::BATCH_SIZE, which is 100, then chunks that list into batches of 100 for processing. If more than 100 orders fall out of sync between cron ticks, such as during a bulk import or a busy sale, the query never sees the rows past the first 100, so each run drains at most 100 and the rest is invisible until the backlog shrinks below that ceiling on its own.
How can I tell the batch size cap is the cause instead of a slow cron?
Poll how many orders have an updated_at newer than your last check. A healthy cron drains the whole delta each run, so that count should fall back under 100 once it catches up. If the count stays at or above 100 for two or more consecutive polls in a row while the backlog does not shrink to zero, that is the signature of the refreshBySchedule batch cap rather than an indexer that is merely busy.
Can a script fix the grid backlog automatically over the REST API?
No. There is no REST endpoint that forces a sales_order_grid sync or removes the BATCH_SIZE limit, and writing to orders through PUT just to force a resync is not a safe or reliable workaround. The real fixes are CLI or admin actions, bin/magento indexer:reindex sales_order_grid, repeatedly running the grid async-insert cron, or switching to Update on Save, so the script's job is to detect and report the backlog, not to mutate order data.
Related field notes
Citations
On the problem:
- GitHub Issue: sales_order_grid refreshBySchedule not working properly for more items than the batch size. github.com/magento/magento2/issues/39602
- GitHub Issue: remove limit in IdListBuilder. github.com/magento/magento2/issues/39621
- GitHub Issue: minor performance improvement of async order grid processing. github.com/magento/magento2/issues/40282
On the solution:
- Adobe Commerce: indexing components and Update by Schedule mode. developer.adobe.com/commerce/php/development/components/indexing
- Adobe Commerce: indexer optimization. developer.adobe.com/commerce/php/development/components/indexing/optimization
- Adobe Commerce: index management in the Admin. experienceleague.adobe.com index management
Stuck on a tricky one?
If you have a problem in Magento 2 or Adobe Commerce indexing, catalog data, orders, or inventory that you would rather hand off, this is the kind of work I do. Message me and we can work through it together.
Did this clear your grid backlog?
If this saved you a confusing missing-orders report or a support ticket you could not explain, you can buy me a coffee. It is the best way to keep these field notes free and growing.
Buy me a coffee on Ko-fi