Reconciler Catalog and Visibility
Duplicate SKU created through concurrent API or import saves
Two REST POST /V1/products calls land within the same instant, or an import bunch and an async bulk save race each other while touching the same SKU, and Magento lets both of them believe the SKU does not exist yet. One wins the unique key check outright, the other either errors out or, on some code paths, quietly commits a second entity_id under the same SKU string. Search and the storefront start showing two rows for what should be one product. Here is why the unique index does not prevent this and a small script that finds every collision and reports it without touching your data.
Magento 2 and Adobe Commerce enforce SKU uniqueness with a database-level unique index on catalog_product_entity.sku, but ProductRepository::save() and the import and bulk API code paths first do an application-level lookup, an in-memory instance cache or a SELECT by sku, to decide whether to insert or update, before they ever touch that index. When two requests race, two REST POST /V1/products calls, or a concurrent import bunch and an async bulk /rest/async/bulk/V1/products call, both can pass the "SKU not found yet" check in the same window and both proceed to insert. One of the two either hits the unique key error, which shows up as bulk operation status 3, or in older or inconsistent code paths actually commits a second catalog_product_entity row with its own entity_id, leaving two IDs that both resolve to the same SKU string in search and the indexes. You cannot repair this safely by merging or deleting an entity blind, since orders, CMS links, and inventory reservations may point at either one. The safe pattern is to page GET /rest/V1/products by updated_at, group by a normalized SKU, flag any group with more than one id, and only ever report it, disabling a confirmed zero-order orphan as a reversible first step, never a delete, and only when explicitly asked to write. Full code, tests, and a dry run guard are below.
The problem in plain words
Every SKU in Magento is supposed to be one product. The database backs that up with a unique index on catalog_product_entity.sku, so in theory two rows can never share a SKU. In practice, that index only ever gets a chance to reject a duplicate at the exact moment a second row tries to insert. Everything that happens before that moment, deciding whether this save is a new product or an update to an existing one, runs in application code first, and application code is not atomic the way a database constraint is.
ProductRepository::save(), and the code paths behind product import and the async bulk product API, look the SKU up first, either against an in-memory cache of instances already loaded in this process or with a SELECT against the table, before deciding to insert or update. If two of these save attempts happen close enough together, two REST POST /V1/products calls for what a merchandiser thinks is the same new product, or a scheduled import bunch running at the same moment an async bulk /rest/async/bulk/V1/products job touches an overlapping SKU, both processes can run that lookup in the same narrow window and both get back "not found." Both then proceed toward an insert, because as far as either one knows, it is the only one creating this SKU.
Why it happens
- Two REST
POST /V1/productscalls for what should be one new SKU arrive close enough together that both passProductRepository::save()'s SKU lookup before either has inserted, a textbook time-of-check to time-of-use gap. - A scheduled
Magento\ImportExportbunch and a concurrent async bulk/rest/async/bulk/V1/productscall both process an overlapping SKU in the same window, since neither process knows about the other's in-flight save. - Magento also allows SKU variants that differ only by leading or trailing whitespace to slip through as if they were different values, so what looks like one SKU in a report can actually be two slightly different strings that both resolve to the same product to a human.
- One of the two racing inserts may hit the unique key error and get reported only as bulk operation status 3, "Unique constraint violation," which is easy to miss in an async bulk job log, while the other insert succeeds normally and looks like nothing went wrong.
This exact race is documented across Magento's own issue tracker, including a case where the repository saves a product twice with the same SKU, a report of duplicate SKUs surfacing specifically through import, and a separate report of the REST API leaving duplicate SKU rows in the database. See the citations at the end for the specific threads.
The unique index on catalog_product_entity.sku guarantees that the database itself never accepts two rows with an identical string. It does not guarantee that only one entity_id ever answers to a SKU in practice, because the check that decides insert-versus-update runs in application code, ahead of that index, and application code is not atomic across two separate requests. A script cannot ask Magento directly whether two saves raced each other. What it can do is ask for every product's id and sku, group by a normalized SKU, and treat any group with more than one distinct id as proof that a collision already happened, then confirm it against the single-SKU lookup endpoint, which only ever resolves one row.
The fix, as a flow
We never merge or delete a duplicate entity blind, because either entity_id could already be referenced by an order line item, a CMS block link, or an inventory reservation that the API cannot see all at once. Instead we add a job that pages recently touched products, groups them by normalized SKU, confirms every collision against the single-SKU lookup, and only when a write is explicitly requested and exactly one of the duplicate IDs has zero orders against it, disables that one entity as a reversible first step.
Build it step by step
Get an admin bearer token
Authenticate the same way as any other Magento REST client. Either call POST /rest/V1/integration/admin/token with an admin username and password, or create an integration and use its token directly. Keep the store URL and token in environment variables, never in the file.
pip install requests
export MAGENTO_URL="https://your-store.example.com"
export MAGENTO_ADMIN_TOKEN="eyJraWQ..."
export LOOKBACK_HOURS="24"
export PAGE_SIZE="200"
export DRY_RUN="true" # start safe, change to false to disable a confirmed orphan// Node 18+ has fetch built in, no dependencies needed
export MAGENTO_URL="https://your-store.example.com"
export MAGENTO_ADMIN_TOKEN="eyJraWQ..."
export LOOKBACK_HOURS="24"
export PAGE_SIZE="200"
export DRY_RUN="true" // start safe, change to false to disable a confirmed orphanTalk to the Magento REST API
Every call sends Authorization: Bearer <token> to a /rest/V1 route. A small helper wraps GET and PUT and raises on a non 200 response, since Magento returns structured error bodies worth surfacing as is.
import os, requests
MAGENTO_URL = os.environ["MAGENTO_URL"].rstrip("/")
TOKEN = os.environ["MAGENTO_ADMIN_TOKEN"]
def magento_get(path, params=None):
r = requests.get(
f"{MAGENTO_URL}/rest/V1{path}",
params=params or {},
headers={"Authorization": f"Bearer {TOKEN}"},
timeout=30,
)
r.raise_for_status()
return r.json()
def magento_put(path, payload):
r = requests.put(
f"{MAGENTO_URL}/rest/V1{path}",
json=payload,
headers={"Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json"},
timeout=30,
)
r.raise_for_status()
return r.json()const MAGENTO_URL = (process.env.MAGENTO_URL || "").replace(/\/$/, "");
const TOKEN = process.env.MAGENTO_ADMIN_TOKEN;
async function magentoGet(path, params = {}) {
const url = new URL(`${MAGENTO_URL}/rest/V1${path}`);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
const res = await fetch(url, { headers: { Authorization: `Bearer ${TOKEN}` } });
if (!res.ok) throw new Error(`Magento ${res.status}`);
return res.json();
}
async function magentoPut(path, payload) {
const res = await fetch(`${MAGENTO_URL}/rest/V1${path}`, {
method: "PUT",
headers: { Authorization: `Bearer ${TOKEN}`, "Content-Type": "application/json" },
body: JSON.stringify(payload),
});
if (!res.ok) throw new Error(`Magento ${res.status}`);
return res.json();
}Page recently touched products
Call GET /rest/V1/products with searchCriteria[filterGroups][0][filters][0][field]=updated_at, [conditionType]=gteq, and [value] set to your lookback window, plus searchCriteria[pageSize] and [currentPage] to page through. Restricting to recently updated products keeps the job fast, since a race between two saves only ever shows up on rows that were just written.
def recent_products(lookback_iso, page_size=200):
current_page = 1
while True:
params = {
"searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
"searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
"searchCriteria[filterGroups][0][filters][0][value]": lookback_iso,
"searchCriteria[pageSize]": page_size,
"searchCriteria[currentPage]": current_page,
}
data = magento_get("/products", params)
for item in data["items"]:
yield item
if current_page * page_size >= data["total_count"]:
return
current_page += 1async function* recentProducts(lookbackIso, pageSize = 200) {
let currentPage = 1;
while (true) {
const params = {
"searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
"searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
"searchCriteria[filterGroups][0][filters][0][value]": lookbackIso,
"searchCriteria[pageSize]": pageSize,
"searchCriteria[currentPage]": currentPage,
};
const data = await magentoGet("/products", params);
for (const item of data.items) yield item;
if (currentPage * pageSize >= data.total_count) return;
currentPage += 1;
}
}Decide, with one pure function
Keep the decision in its own function that takes a plain list of products and groups them by a normalized SKU, trimmed and lowercased, since Magento allows leading and trailing whitespace variants of the same visible SKU. It is a pure map-reduce with no I/O, which makes it easy to test, as we do later. Only groups with more than one distinct id are collisions. Each group's entity_ids and created_at timestamps come back sorted ascending, so index 0 is the presumed original and later entries are the race-created duplicates.
def find_sku_collisions(products):
groups = {}
for p in products:
normalized = p["sku"].strip().lower()
groups.setdefault(normalized, []).append(p)
collisions = []
for normalized_sku, members in groups.items():
distinct_ids = {m["id"] for m in members}
if len(distinct_ids) <= 1:
continue
ordered = sorted(members, key=lambda m: m["created_at"])
collisions.append({
"sku": normalized_sku,
"entity_ids": [m["id"] for m in ordered],
"created_at": [m["created_at"] for m in ordered],
})
collisions.sort(key=lambda c: c["sku"])
return collisionsexport function findSkuCollisions(products) {
const groups = new Map();
for (const p of products) {
const normalized = p.sku.trim().toLowerCase();
const list = groups.get(normalized) || [];
list.push(p);
groups.set(normalized, list);
}
const collisions = [];
for (const [normalizedSku, members] of groups) {
const distinctIds = new Set(members.map((m) => m.id));
if (distinctIds.size <= 1) continue;
const ordered = [...members].sort((a, b) => (a.created_at < b.created_at ? -1 : a.created_at > b.created_at ? 1 : 0));
collisions.push({
sku: normalizedSku,
entity_ids: ordered.map((m) => m.id),
created_at: ordered.map((m) => m.created_at),
});
}
collisions.sort((a, b) => (a.sku < b.sku ? -1 : a.sku > b.sku ? 1 : 0));
return collisions;
}Confirm each collision, then disable, never delete
For each flagged SKU, call GET /rest/V1/products/{sku}, which resolves only one entity_id per exact SKU string. If that single id does not match every id the grouping found, you have a confirmed live duplicate rather than a stale reindex artifact. Only when DRY_RUN=false is explicitly set, and only when exactly one of the duplicate ids has zero orders against it, PUT /rest/V1/products/{sku} with {"product":{"status":2}} to disable that one entity. Disabling is reversible. A delete is never part of this script.
def has_zero_orders(sku):
params = {
"searchCriteria[filterGroups][0][filters][0][field]": "sku",
"searchCriteria[filterGroups][0][filters][0][value]": sku,
"searchCriteria[pageSize]": 1,
}
data = magento_get("/orders", params)
return data.get("total_count", 0) == 0
def disable_orphan(sku):
payload = {"product": {"sku": sku, "status": 2}}
return magento_put(f"/products/{sku}", payload)async function hasZeroOrders(sku) {
const params = {
"searchCriteria[filterGroups][0][filters][0][field]": "sku",
"searchCriteria[filterGroups][0][filters][0][value]": sku,
"searchCriteria[pageSize]": 1,
};
const data = await magentoGet("/orders", params);
return (data.total_count || 0) === 0;
}
async function disableOrphan(sku) {
const payload = { product: { sku, status: 2 } };
return magentoPut(`/products/${encodeURIComponent(sku)}`, payload);
}Wire it together with a dry run guard
The loop pages recent products, groups them with the pure function, and prints one report line per colliding SKU with every member's entity_id and created_at. Leave DRY_RUN on for the first runs so nothing is written, just reported. Only flip it to false once you have reviewed the report, and even then the script only disables an entity when exactly one of the pair has zero orders. Run it on a schedule that fits how often you run imports or bulk saves, for example hourly.
This script never merges or deletes a product entity. With DRY_RUN=true, the default, it only prints the collision report. With DRY_RUN=false, it additionally disables one entity, only when exactly one of the duplicate ids has zero orders against it, and it does so with a reversible status change, never a save that removes data or a DELETE call. When both entities have orders, or when neither is a clean orphan, it leaves the pair alone for a human to resolve.
The full code
Here is the complete script in one file for each language. It reads settings from the environment, pages recent products, groups them with the pure function, confirms every collision against the single-SKU lookup, prints a full report, and only disables a confirmed zero-order orphan when explicitly told to write. It is safe to run again and again because it never merges or deletes anything.
"""Find and report duplicate SKUs created through concurrent API or import saves.
Magento 2 and Adobe Commerce enforce SKU uniqueness with a unique index on
catalog_product_entity.sku, but ProductRepository::save() and the import
and bulk API code paths first do an application-level lookup to decide
insert versus update, before that index ever runs. When two saves race,
two REST POST /V1/products calls, or a concurrent import bunch and an
async bulk save, both can see "SKU not found" in the same window and both
proceed to insert, leaving two entity_ids that resolve to the same SKU.
This never merges or deletes a product entity. It pages recently touched
products, groups by normalized sku with a pure function, confirms every
collision against the single-SKU lookup, always reports it, and only when
DRY_RUN is explicitly false and exactly one entity has zero orders does it
disable that one entity with status 2 as a reversible step. Run on a
schedule. Safe to run again and again.
"""
import os
import logging
import datetime
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("find_sku_collisions")
MAGENTO_URL = os.environ["MAGENTO_URL"].rstrip("/")
TOKEN = os.environ["MAGENTO_ADMIN_TOKEN"]
LOOKBACK_HOURS = float(os.environ.get("LOOKBACK_HOURS", "24"))
PAGE_SIZE = int(os.environ.get("PAGE_SIZE", "200"))
DRY_RUN = os.environ.get("DRY_RUN", "true").lower() == "true"
def magento_get(path, params=None):
r = requests.get(
f"{MAGENTO_URL}/rest/V1{path}",
params=params or {},
headers={"Authorization": f"Bearer {TOKEN}"},
timeout=30,
)
r.raise_for_status()
return r.json()
def magento_put(path, payload):
r = requests.put(
f"{MAGENTO_URL}/rest/V1{path}",
json=payload,
headers={"Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json"},
timeout=30,
)
r.raise_for_status()
return r.json()
def lookback_iso(hours):
dt = datetime.datetime.utcnow() - datetime.timedelta(hours=hours)
return dt.strftime("%Y-%m-%d %H:%M:%S")
def recent_products(lookback, page_size=200):
current_page = 1
while True:
params = {
"searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
"searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
"searchCriteria[filterGroups][0][filters][0][value]": lookback,
"searchCriteria[pageSize]": page_size,
"searchCriteria[currentPage]": current_page,
}
data = magento_get("/products", params)
for item in data["items"]:
yield item
if current_page * page_size >= data["total_count"]:
return
current_page += 1
def find_sku_collisions(products):
groups = {}
for p in products:
normalized = p["sku"].strip().lower()
groups.setdefault(normalized, []).append(p)
collisions = []
for normalized_sku, members in groups.items():
distinct_ids = {m["id"] for m in members}
if len(distinct_ids) <= 1:
continue
ordered = sorted(members, key=lambda m: m["created_at"])
collisions.append({
"sku": normalized_sku,
"entity_ids": [m["id"] for m in ordered],
"created_at": [m["created_at"] for m in ordered],
})
collisions.sort(key=lambda c: c["sku"])
return collisions
def confirm_collision(sku):
"""GET /products/{sku} resolves only one entity_id for an exact sku string."""
data = magento_get(f"/products/{sku}")
return data.get("id")
def has_zero_orders(sku):
params = {
"searchCriteria[filterGroups][0][filters][0][field]": "sku",
"searchCriteria[filterGroups][0][filters][0][value]": sku,
"searchCriteria[pageSize]": 1,
}
data = magento_get("/orders", params)
return data.get("total_count", 0) == 0
def disable_orphan(sku):
payload = {"product": {"sku": sku, "status": 2}}
return magento_put(f"/products/{sku}", payload)
def run():
raw_items = list(recent_products(lookback_iso(LOOKBACK_HOURS), PAGE_SIZE))
products = [{"id": item["id"], "sku": item["sku"], "created_at": item.get("created_at", "")} for item in raw_items]
collisions = find_sku_collisions(products)
if not collisions:
log.info("Done. 0 duplicate SKU group(s) found.")
return
disabled = 0
for col in collisions:
member_summary = ", ".join(
f"entity_id={eid} created_at={ts}" for eid, ts in zip(col["entity_ids"], col["created_at"])
)
log.warning("sku %s has %d entity_id(s): %s", col["sku"], len(col["entity_ids"]), member_summary)
resolved_id = confirm_collision(col["sku"])
if resolved_id in col["entity_ids"] and len(col["entity_ids"]) == 2:
candidates = [eid for eid in col["entity_ids"] if eid != resolved_id]
orphan_id = candidates[0] if candidates else None
else:
orphan_id = None
if orphan_id is None:
log.warning(" -> could not confirm a single safe orphan for sku %s, skipping.", col["sku"])
continue
if not has_zero_orders(col["sku"]):
log.warning(" -> sku %s has orders on file, leaving both entities alone.", col["sku"])
continue
log.warning(" -> %s entity_id %s (status=2, Disabled).", "would disable" if DRY_RUN else "disabling", orphan_id)
if not DRY_RUN:
disable_orphan(col["sku"])
disabled += 1
log.info(
"Done. %d duplicate SKU group(s), %d orphan(s) %s.",
len(collisions), disabled, "to disable" if DRY_RUN else "disabled",
)
if __name__ == "__main__":
run()/**
* Find and report duplicate SKUs created through concurrent API or import saves.
*
* Magento 2 and Adobe Commerce enforce SKU uniqueness with a unique index on
* catalog_product_entity.sku, but ProductRepository::save() and the import
* and bulk API code paths first do an application-level lookup to decide
* insert versus update, before that index ever runs. When two saves race,
* two REST POST /V1/products calls, or a concurrent import bunch and an
* async bulk save, both can see "SKU not found" in the same window and both
* proceed to insert, leaving two entity_ids that resolve to the same SKU.
* This never merges or deletes a product entity. It pages recently touched
* products, groups by normalized sku with a pure function, confirms every
* collision against the single-SKU lookup, always reports it, and only when
* DRY_RUN is explicitly false and exactly one entity has zero orders does it
* disable that one entity with status 2 as a reversible step. Run on a
* schedule.
*
* Guide: https://www.allanninal.dev/magento/duplicate-sku-race-condition/
*/
import { pathToFileURL } from "node:url";
const MAGENTO_URL = (process.env.MAGENTO_URL || "https://demo.example.com").replace(/\/$/, "");
const TOKEN = process.env.MAGENTO_ADMIN_TOKEN || "token_dummy";
const LOOKBACK_HOURS = Number(process.env.LOOKBACK_HOURS || 24);
const PAGE_SIZE = Number(process.env.PAGE_SIZE || 200);
const DRY_RUN = (process.env.DRY_RUN || "true").toLowerCase() === "true";
export function findSkuCollisions(products) {
const groups = new Map();
for (const p of products) {
const normalized = p.sku.trim().toLowerCase();
const list = groups.get(normalized) || [];
list.push(p);
groups.set(normalized, list);
}
const collisions = [];
for (const [normalizedSku, members] of groups) {
const distinctIds = new Set(members.map((m) => m.id));
if (distinctIds.size <= 1) continue;
const ordered = [...members].sort((a, b) => (a.created_at < b.created_at ? -1 : a.created_at > b.created_at ? 1 : 0));
collisions.push({
sku: normalizedSku,
entity_ids: ordered.map((m) => m.id),
created_at: ordered.map((m) => m.created_at),
});
}
collisions.sort((a, b) => (a.sku < b.sku ? -1 : a.sku > b.sku ? 1 : 0));
return collisions;
}
function lookbackIso(hours) {
const dt = new Date(Date.now() - hours * 3600 * 1000);
return dt.toISOString().slice(0, 19).replace("T", " ");
}
async function magentoGet(path, params = {}) {
const url = new URL(`${MAGENTO_URL}/rest/V1${path}`);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
const res = await fetch(url, { headers: { Authorization: `Bearer ${TOKEN}` } });
if (!res.ok) throw new Error(`Magento ${res.status}`);
return res.json();
}
async function magentoPut(path, payload) {
const res = await fetch(`${MAGENTO_URL}/rest/V1${path}`, {
method: "PUT",
headers: { Authorization: `Bearer ${TOKEN}`, "Content-Type": "application/json" },
body: JSON.stringify(payload),
});
if (!res.ok) throw new Error(`Magento ${res.status}`);
return res.json();
}
async function* recentProducts(lookback, pageSize = 200) {
let currentPage = 1;
while (true) {
const params = {
"searchCriteria[filterGroups][0][filters][0][field]": "updated_at",
"searchCriteria[filterGroups][0][filters][0][conditionType]": "gteq",
"searchCriteria[filterGroups][0][filters][0][value]": lookback,
"searchCriteria[pageSize]": pageSize,
"searchCriteria[currentPage]": currentPage,
};
const data = await magentoGet("/products", params);
for (const item of data.items) yield item;
if (currentPage * pageSize >= data.total_count) return;
currentPage += 1;
}
}
async function confirmCollision(sku) {
const data = await magentoGet(`/products/${encodeURIComponent(sku)}`);
return data.id;
}
async function hasZeroOrders(sku) {
const params = {
"searchCriteria[filterGroups][0][filters][0][field]": "sku",
"searchCriteria[filterGroups][0][filters][0][value]": sku,
"searchCriteria[pageSize]": 1,
};
const data = await magentoGet("/orders", params);
return (data.total_count || 0) === 0;
}
async function disableOrphan(sku) {
const payload = { product: { sku, status: 2 } };
return magentoPut(`/products/${encodeURIComponent(sku)}`, payload);
}
export async function run() {
const rawItems = [];
for await (const item of recentProducts(lookbackIso(LOOKBACK_HOURS), PAGE_SIZE)) rawItems.push(item);
const products = rawItems.map((item) => ({ id: item.id, sku: item.sku, created_at: item.created_at || "" }));
const collisions = findSkuCollisions(products);
if (collisions.length === 0) {
console.log("Done. 0 duplicate SKU group(s) found.");
return;
}
let disabled = 0;
for (const col of collisions) {
const memberSummary = col.entity_ids
.map((eid, i) => `entity_id=${eid} created_at=${col.created_at[i]}`)
.join(", ");
console.warn(`sku ${col.sku} has ${col.entity_ids.length} entity_id(s): ${memberSummary}`);
const resolvedId = await confirmCollision(col.sku);
let orphanId = null;
if (col.entity_ids.includes(resolvedId) && col.entity_ids.length === 2) {
const candidates = col.entity_ids.filter((eid) => eid !== resolvedId);
orphanId = candidates.length ? candidates[0] : null;
}
if (orphanId === null) {
console.warn(` -> could not confirm a single safe orphan for sku ${col.sku}, skipping.`);
continue;
}
if (!(await hasZeroOrders(col.sku))) {
console.warn(` -> sku ${col.sku} has orders on file, leaving both entities alone.`);
continue;
}
console.warn(` -> ${DRY_RUN ? "would disable" : "disabling"} entity_id ${orphanId} (status=2, Disabled).`);
if (!DRY_RUN) await disableOrphan(col.sku);
disabled++;
}
console.log(
`Done. ${collisions.length} duplicate SKU group(s), ${disabled} orphan(s) ${DRY_RUN ? "to disable" : "disabled"}.`
);
}
if (import.meta.url === pathToFileURL(process.argv[1]).href) {
run().catch((err) => { console.error(err); process.exit(1); });
}Add a test
The grouping rule is the part most worth testing, because it decides which products get reported as duplicates. Because we kept find_sku_collisions pure, the test needs no network and no Magento store. It just feeds in plain fixture products and checks the answer.
from find_sku_collisions import find_sku_collisions
def product(**over):
base = {"id": 4501, "sku": "ABC-100", "created_at": "2026-07-01T10:00:00Z"}
base.update(over)
return base
def test_no_collisions_when_all_skus_unique():
products = [product(id=1, sku="A"), product(id=2, sku="B")]
assert find_sku_collisions(products) == []
def test_detects_collision_across_two_entity_ids():
products = [
product(id=4501, sku="ABC-100", created_at="2026-07-01T10:00:00Z"),
product(id=4502, sku="ABC-100", created_at="2026-07-01T10:00:01Z"),
]
result = find_sku_collisions(products)
assert len(result) == 1
assert result[0]["sku"] == "abc-100"
assert result[0]["entity_ids"] == [4501, 4502]
def test_same_entity_id_repeated_is_not_a_collision():
products = [product(id=4501, sku="ABC-100"), product(id=4501, sku="ABC-100")]
assert find_sku_collisions(products) == []
def test_whitespace_and_case_variants_are_treated_as_the_same_sku():
products = [
product(id=1, sku=" ABC-100 ", created_at="2026-07-01T10:00:00Z"),
product(id=2, sku="abc-100", created_at="2026-07-01T10:00:01Z"),
]
result = find_sku_collisions(products)
assert len(result) == 1
assert result[0]["sku"] == "abc-100"
def test_entity_ids_sorted_by_created_at_ascending():
products = [
product(id=4502, sku="ABC-100", created_at="2026-07-01T10:00:01Z"),
product(id=4501, sku="ABC-100", created_at="2026-07-01T10:00:00Z"),
]
result = find_sku_collisions(products)
assert result[0]["entity_ids"] == [4501, 4502]
assert result[0]["created_at"] == ["2026-07-01T10:00:00Z", "2026-07-01T10:00:01Z"]
def test_groups_sorted_by_sku_ascending():
products = [
product(id=1, sku="ZZZ-1"), product(id=2, sku="ZZZ-1"),
product(id=3, sku="AAA-1"), product(id=4, sku="AAA-1"),
]
result = find_sku_collisions(products)
assert [c["sku"] for c in result] == ["aaa-1", "zzz-1"]
def test_three_way_collision_is_one_group_with_three_ids():
products = [
product(id=1, sku="X-1", created_at="2026-07-01T00:00:00Z"),
product(id=2, sku="X-1", created_at="2026-07-01T00:00:01Z"),
product(id=3, sku="X-1", created_at="2026-07-01T00:00:02Z"),
]
result = find_sku_collisions(products)
assert len(result) == 1
assert len(result[0]["entity_ids"]) == 3
def test_empty_input_returns_empty_list():
assert find_sku_collisions([]) == []import { test } from "node:test";
import assert from "node:assert/strict";
import { findSkuCollisions } from "./find-sku-collisions.js";
const product = (over = {}) => ({
id: 4501,
sku: "ABC-100",
created_at: "2026-07-01T10:00:00Z",
...over,
});
test("no collisions when all skus unique", () => {
const products = [product({ id: 1, sku: "A" }), product({ id: 2, sku: "B" })];
assert.deepEqual(findSkuCollisions(products), []);
});
test("detects collision across two entity ids", () => {
const products = [
product({ id: 4501, sku: "ABC-100", created_at: "2026-07-01T10:00:00Z" }),
product({ id: 4502, sku: "ABC-100", created_at: "2026-07-01T10:00:01Z" }),
];
const result = findSkuCollisions(products);
assert.equal(result.length, 1);
assert.equal(result[0].sku, "abc-100");
assert.deepEqual(result[0].entity_ids, [4501, 4502]);
});
test("same entity id repeated is not a collision", () => {
const products = [product({ id: 4501, sku: "ABC-100" }), product({ id: 4501, sku: "ABC-100" })];
assert.deepEqual(findSkuCollisions(products), []);
});
test("whitespace and case variants are treated as the same sku", () => {
const products = [
product({ id: 1, sku: " ABC-100 ", created_at: "2026-07-01T10:00:00Z" }),
product({ id: 2, sku: "abc-100", created_at: "2026-07-01T10:00:01Z" }),
];
const result = findSkuCollisions(products);
assert.equal(result.length, 1);
assert.equal(result[0].sku, "abc-100");
});
test("entity ids sorted by created_at ascending", () => {
const products = [
product({ id: 4502, sku: "ABC-100", created_at: "2026-07-01T10:00:01Z" }),
product({ id: 4501, sku: "ABC-100", created_at: "2026-07-01T10:00:00Z" }),
];
const result = findSkuCollisions(products);
assert.deepEqual(result[0].entity_ids, [4501, 4502]);
assert.deepEqual(result[0].created_at, ["2026-07-01T10:00:00Z", "2026-07-01T10:00:01Z"]);
});
test("groups sorted by sku ascending", () => {
const products = [
product({ id: 1, sku: "ZZZ-1" }), product({ id: 2, sku: "ZZZ-1" }),
product({ id: 3, sku: "AAA-1" }), product({ id: 4, sku: "AAA-1" }),
];
const result = findSkuCollisions(products);
assert.deepEqual(result.map((c) => c.sku), ["aaa-1", "zzz-1"]);
});
test("three way collision is one group with three ids", () => {
const products = [
product({ id: 1, sku: "X-1", created_at: "2026-07-01T00:00:00Z" }),
product({ id: 2, sku: "X-1", created_at: "2026-07-01T00:00:01Z" }),
product({ id: 3, sku: "X-1", created_at: "2026-07-01T00:00:02Z" }),
];
const result = findSkuCollisions(products);
assert.equal(result.length, 1);
assert.equal(result[0].entity_ids.length, 3);
});
test("empty input returns empty list", () => {
assert.deepEqual(findSkuCollisions([]), []);
});Case studies
Two storefront integrations creating the same SKU at once
A retailer connected two separate systems to the same Magento store over the Admin REST API, an ERP feed and a marketing tool that both created new products when a certain signal fired. Both happened to react to the same event and both fired a POST /V1/products call for a brand new SKU within a fraction of a second of each other.
Neither call errored in a way anyone noticed. Search results started showing the same product name twice, one had the ERP's price, the other had a placeholder price the marketing tool had set. Running the collision report over /rest/V1/products found the pair immediately, confirmed against the single-SKU lookup, and because the marketing tool's entity had zero orders against it, the team disabled it and let the ERP-owned entity stand.
A scheduled import overlapping a manual bulk update
A store ran a nightly Magento\ImportExport product import right around the same time an operations engineer kicked off a manual async bulk update through /rest/async/bulk/V1/products to fix a pricing issue on a subset of SKUs. A handful of SKUs happened to be in both jobs.
The bulk operation status for a few items came back 3, "Unique constraint violation," buried in a long list of otherwise successful operations, so it went unnoticed until a customer complained about seeing two listings for the same item. The report caught the exact SKUs, showed which entity had zero orders, and gave the team a safe first step while they adjusted the import and bulk schedules so the two jobs never overlap again.
After this runs on a schedule, a SKU race is caught within one polling cycle instead of surviving until a customer notices two listings for the same product. The report carries every colliding entity_id and its created_at timestamp, and the only write this script ever makes is disabling one entity, and only when it can prove that entity has zero orders against it. Anything murkier, both entities with orders, more than two ids, an unconfirmed collision, is left for a human to resolve by hand, which is exactly how a script that touches live catalog data should behave.
FAQ
How can Magento create two products with the same SKU when there is a unique index?
The unique index on catalog_product_entity.sku only stops a second row from ever landing in the table. It does not stop two requests from both deciding to insert. ProductRepository::save and the import and bulk API code paths first check an in-memory cache or run a SELECT by sku to decide insert versus update, and if two requests, such as two REST POST /V1/products calls or a concurrent import and async bulk save, both run that check in the same narrow window, both can see SKU not found and both proceed to insert. One of the two then hits the unique key error, or in older code paths actually commits a second entity_id under the same SKU string.
How do I find duplicate SKUs over the REST API without database access?
Page GET /rest/V1/products filtered by updated_at with a gteq condition for your lookback window, group the returned items by sku after trimming whitespace and lowercasing it, and flag any group with more than one distinct id. Confirm each flagged group with GET /rest/V1/products/{sku}, which only ever resolves one entity_id for an exact SKU string, so a mismatch between the grouped list and that single lookup confirms a live duplicate rather than a stale reindex artifact.
Is it safe to auto merge or delete a duplicate SKU entity?
No. Deleting or merging a product entity can orphan order line items, CMS block links, and inventory reservations that still point at that entity_id. The safe corrective action is to report every collision and, only when DRY_RUN is explicitly set to false and exactly one of the duplicate entity_ids has zero associated orders, disable that one entity with status 2 as a reversible first step. A DELETE is never part of the same run.
Related field notes
Citations
On the problem:
- GitHub Issue: Magento is able to save a product twice with the same SKU via the repository. github.com/magento/magento2/issues/24799
- GitHub Issue: duplicate SKU possible if importing products. github.com/magento/magento2/issues/10080
- GitHub Issue: REST API, duplicate SKU in database. github.com/magento/magento2/issues/30893
On the solution:
- Adobe Commerce: search using REST endpoints. developer.adobe.com/commerce/webapi/rest/use-rest/performing-searches
- Adobe Commerce: REST API reference. developer.adobe.com/commerce/webapi/rest/reference
- Adobe Commerce: product data attributes reference. experienceleague.adobe.com/docs/commerce-admin/systems/data-transfer/data-attributes-product
Stuck on a tricky one?
If you have a problem in Magento 2 or Adobe Commerce orders, catalog data, cron, or inventory that you would rather hand off, this is the kind of work I do. Message me and we can work through it together.
Did this catch a collision before a customer did?
If this saved you a confusing duplicate listing or a wrong stock count, you can buy me a coffee. It is the best way to keep these field notes free and growing.
Buy me a coffee on Ko-fi