Repair Customers and data
External id metafield dropped on create
Your linking key vanished on order create. A warehouse system, a marketplace, or an ERP relies on a metafield to match a Shopify order back to its own record, and on some orders that metafield never gets written. The two systems quietly lose track of each other, and nobody notices until a fulfillment or a refund cannot be matched. Here is why it happens and a small script that audits recent orders and sets the metafield back only where it is truly missing or wrong.
The external id metafield is written by a second step after the order is created, and some order paths never trigger that step, so the order is left without its linking key. Run a small Python or Node.js script that lists recent orders, reads their external_sync.external_id metafield, compares it against the id you already hold in your own system of record, and calls metafieldsSet only on the orders where the value is missing or does not match. Full code, tests, and a dry run guard are below.
The problem in plain words
Most stores that sync orders to another system need a way to say "this Shopify order is that record over there." The usual way to store that link is a metafield on the order, something like external_sync.external_id, set to the id the other system uses. Once that metafield is on the order, any later lookup can go straight from one system to the other with no guesswork.
The trouble is that metafield is almost never set by Shopify itself. It is written by an app, a webhook handler, or a script, as a separate call right after the order is created. Order creation and metafield writing are two different steps, and anything that breaks the link between them, a webhook that never fires, an app that only watches one creation path, a bulk import that skips the app entirely, leaves the order sitting there with no metafield at all. The order is otherwise completely normal. It just cannot be found by the other system anymore.
Why it happens
Shopify never sets this metafield on its own. It is your integration's job, so the gap opens wherever that integration does not run. A few common ways stores end up here:
- An order created by a bulk import or a migration tool that inserts orders directly and never calls the app that normally tags new orders with their external id.
- A webhook subscription for
orders/createthat is missing, disabled, or was quietly removed when the app was reinstalled, so the metafield write never fires. - A draft order that gets completed through a channel the integration does not watch, such as a POS sale or an admin-created order, instead of the checkout flow the app expects.
- A race between two systems creating the same order at nearly the same time, where the metafield write from one attempt is overwritten or skipped once the order settles.
This is a common source of confusion because everything else about the order looks fine. The customer, the line items, the totals are all correct. It is only when the other system tries to look the order up by its external id, for a shipment update or a refund, that the mismatch shows up, often days later and far from where the original order was created. See the citations at the end for the exact docs on metafields and webhooks.
You already know the correct external id for every order in your own system of record. So the safe repair is not "invent a value" or "clear whatever is there." It is "compare Shopify's metafield to the value you already hold, and only write when they differ." That keeps the script from ever guessing, and it means an order whose metafield is already correct is never touched.
The fix, as a flow
We do not change how the order was created. We add a job that reads recent orders and their external_sync.external_id metafield, checks each one against your source of truth, and calls metafieldsSet only where the value is missing or wrong. An order whose metafield already matches is left completely alone.
Build it step by step
Get an Admin API access token
Create a custom app in your Shopify admin under Settings, Apps and sales channels, Develop apps. Give it the read_orders and write_orders scopes, which cover reading and writing order metafields, and install it to get an Admin API access token that starts with shpat_. Keep the token and the shop domain in environment variables, never in the file.
pip install requests
export SHOPIFY_SHOP="yourstore.myshopify.com"
export SHOPIFY_ACCESS_TOKEN="shpat_..."
export METAFIELD_NAMESPACE="external_sync"
export METAFIELD_KEY="external_id"
export DRY_RUN="true" # start safe, change to false to write// Node 18+ has fetch built in, no dependencies needed
export SHOPIFY_SHOP="yourstore.myshopify.com"
export SHOPIFY_ACCESS_TOKEN="shpat_..."
export METAFIELD_NAMESPACE="external_sync"
export METAFIELD_KEY="external_id"
export DRY_RUN="true" // start safe, change to false to writeTalk to the Admin GraphQL API
Every call goes to one GraphQL endpoint with your token in the X-Shopify-Access-Token header. A small helper sends a query and returns the data, and raises if Shopify reports an error. We use this same helper to read orders and to run the mutation.
import os, requests
SHOP = os.environ["SHOPIFY_SHOP"]
TOKEN = os.environ["SHOPIFY_ACCESS_TOKEN"]
API_VERSION = os.environ.get("SHOPIFY_API_VERSION", "2025-01")
ENDPOINT = f"https://{SHOP}/admin/api/{API_VERSION}/graphql.json"
def gql(query, variables=None):
r = requests.post(
ENDPOINT,
json={"query": query, "variables": variables or {}},
headers={"X-Shopify-Access-Token": TOKEN, "Content-Type": "application/json"},
timeout=30,
)
r.raise_for_status()
body = r.json()
if body.get("errors"):
raise RuntimeError(body["errors"])
return body["data"]const SHOP = process.env.SHOPIFY_SHOP;
const TOKEN = process.env.SHOPIFY_ACCESS_TOKEN;
const API_VERSION = process.env.SHOPIFY_API_VERSION || "2025-01";
const ENDPOINT = `https://${SHOP}/admin/api/${API_VERSION}/graphql.json`;
async function gql(query, variables = {}) {
const res = await fetch(ENDPOINT, {
method: "POST",
headers: { "X-Shopify-Access-Token": TOKEN, "Content-Type": "application/json" },
body: JSON.stringify({ query, variables }),
});
if (!res.ok) throw new Error(`Shopify ${res.status}`);
const body = await res.json();
if (body.errors) throw new Error(JSON.stringify(body.errors));
return body.data;
}List recent orders with their external id metafield
Ask for orders created in the lookback window, and read back the order id, the name, and the external_sync.external_id metafield if one exists. A missing metafield simply comes back as null. We page through with a cursor so the job handles a large backlog.
ORDERS_QUERY = """
query($cursor: String, $q: String!) {
orders(first: 50, after: $cursor, query: $q) {
pageInfo { hasNextPage endCursor }
nodes {
id
name
metafield(namespace: "external_sync", key: "external_id") { id value }
}
}
}"""
def orders_missing_link():
q = "created_at:>-7d"
cursor = None
while True:
data = gql(ORDERS_QUERY, {"cursor": cursor, "q": q})["orders"]
for node in data["nodes"]:
yield node
if not data["pageInfo"]["hasNextPage"]:
return
cursor = data["pageInfo"]["endCursor"]const ORDERS_QUERY = `
query($cursor: String, $q: String!) {
orders(first: 50, after: $cursor, query: $q) {
pageInfo { hasNextPage endCursor }
nodes {
id
name
metafield(namespace: "external_sync", key: "external_id") { id value }
}
}
}`;
async function* ordersMissingLink() {
const q = "created_at:>-7d";
let cursor = null;
while (true) {
const data = (await gql(ORDERS_QUERY, { cursor, q })).orders;
for (const node of data.nodes) yield node;
if (!data.pageInfo.hasNextPage) return;
cursor = data.pageInfo.endCursor;
}
}Decide, with one pure function
Keep the decision in its own function that takes the order and the external id your own system says it should have, and returns true or false. A pure function like this is easy to read and easy to test, which we do later. The rule is deliberately narrow. If you have no expected id for this order yet, do nothing. If Shopify's metafield value already equals the expected id, do nothing. Only when the two disagree, whether the metafield is completely missing or holds a stale value, does the order need a write.
def needs_repair(order, expected_external_id):
if not expected_external_id:
return False
current = order.get("metafield")
current_value = current.get("value") if current else None
return current_value != expected_external_id
def plan_repairs(orders, external_id_lookup):
plan = []
for order in orders:
expected = external_id_lookup.get(order["name"])
if needs_repair(order, expected):
plan.append((order, expected))
return planexport function needsRepair(order, expectedExternalId) {
if (!expectedExternalId) return false;
const current = order.metafield ? order.metafield.value : null;
return current !== expectedExternalId;
}
export function planRepairs(orders, externalIdLookup) {
const plan = [];
for (const order of orders) {
const expected = externalIdLookup[order.name];
if (needsRepair(order, expected)) plan.push([order, expected]);
}
return plan;
}Write the metafield back with metafieldsSet
When an order needs a repair, call metafieldsSet with the order id as the owner, the namespace and key you configured, and the expected value. Always read back userErrors. If Shopify refuses, for example because the metafield definition expects a different type, the error tells you why, and the script should stop on it rather than pretend it worked.
METAFIELDS_SET = """
mutation($metafields: [MetafieldsSetInput!]!) {
metafieldsSet(metafields: $metafields) {
metafields { id key value }
userErrors { field message }
}
}"""
def set_external_id(order_id, external_id):
variables = {
"metafields": [{
"ownerId": order_id,
"namespace": METAFIELD_NAMESPACE,
"key": METAFIELD_KEY,
"type": "single_line_text_field",
"value": external_id,
}]
}
result = gql(METAFIELDS_SET, variables)["metafieldsSet"]
if result["userErrors"]:
raise RuntimeError(result["userErrors"])
return result["metafields"][0]["value"]const METAFIELDS_SET = `
mutation($metafields: [MetafieldsSetInput!]!) {
metafieldsSet(metafields: $metafields) {
metafields { id key value }
userErrors { field message }
}
}`;
async function setExternalId(orderId, externalId) {
const variables = {
metafields: [{
ownerId: orderId,
namespace: METAFIELD_NAMESPACE,
key: METAFIELD_KEY,
type: "single_line_text_field",
value: externalId,
}],
};
const result = (await gql(METAFIELDS_SET, variables)).metafieldsSet;
if (result.userErrors.length) throw new Error(JSON.stringify(result.userErrors));
return result.metafields[0].value;
}Wire it together with a dry run guard
The loop ties every piece together. Notice the dry run guard. On the first few runs, leave DRY_RUN on so the script only reports which orders it would repair. Read the output, agree with it, then switch it off to let it write. Run it on a schedule that matches how often orders are created outside your normal flow, for example once an hour, and plug in a real lookup against your own system of record where the placeholder is marked below.
Always start with DRY_RUN=true, and never invent a value for the external id. The script only ever copies the value from the system that already owns it, so a mistake there is your mistake to catch, not the script's to guess around.
The full code
Here is the complete script in one file for each language. It reads settings from the environment, logs what it does, respects the dry run flag, and is safe to run again and again because it only writes an order whose metafield is missing or does not match the value you already hold.
View this code on GitHub Full runnable folder with tests in the shopify-fixes repo.
"""Find orders whose external id metafield went missing on create, and set it back.
Some integrations write a linking key onto the order the moment it is created, usually
a metafield such as external_sync.external_id, so a warehouse system, a marketplace, or
an ERP can match the Shopify order back to its own record. When the order is created by
a flow that skips that write, such as a checkout that bypasses the app, a bulk import, or
a race between two systems creating the order at the same time, the metafield is never
set and the two systems can no longer find each other.
This script reads recent orders, compares the external_sync.external_id metafield against
the source of truth id you already have (for example your own order-to-external-id map),
and only repairs the ones that are missing or wrong, using metafieldsSet. It never touches
an order whose metafield already matches. Run on a schedule. Safe to run again and again.
"""
import os
import logging
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("repair_external_id_metafield")
SHOP = os.environ["SHOPIFY_SHOP"]
TOKEN = os.environ["SHOPIFY_ACCESS_TOKEN"]
API_VERSION = os.environ.get("SHOPIFY_API_VERSION", "2025-01")
ENDPOINT = f"https://{SHOP}/admin/api/{API_VERSION}/graphql.json"
DRY_RUN = os.environ.get("DRY_RUN", "true").lower() == "true"
METAFIELD_NAMESPACE = os.environ.get("METAFIELD_NAMESPACE", "external_sync")
METAFIELD_KEY = os.environ.get("METAFIELD_KEY", "external_id")
ORDERS_QUERY = """
query($cursor: String, $q: String!) {
orders(first: 50, after: $cursor, query: $q) {
pageInfo { hasNextPage endCursor }
nodes {
id
name
metafield(namespace: "external_sync", key: "external_id") { id value }
}
}
}"""
METAFIELDS_SET = """
mutation($metafields: [MetafieldsSetInput!]!) {
metafieldsSet(metafields: $metafields) {
metafields { id key value }
userErrors { field message }
}
}"""
def gql(query, variables=None):
r = requests.post(
ENDPOINT,
json={"query": query, "variables": variables or {}},
headers={"X-Shopify-Access-Token": TOKEN, "Content-Type": "application/json"},
timeout=30,
)
r.raise_for_status()
body = r.json()
if body.get("errors"):
raise RuntimeError(body["errors"])
return body["data"]
def needs_repair(order, expected_external_id):
"""Pure decision: does this order's external id metafield need to be (re)written?
order is a dict with an optional "metafield" (None, or {"value": "..."}).
expected_external_id is the id from the source of truth for this order, or None
if we have no external record for it yet (in which case there is nothing to do).
"""
if not expected_external_id:
return False
current = order.get("metafield")
current_value = current.get("value") if current else None
return current_value != expected_external_id
def plan_repairs(orders, external_id_lookup):
"""Pure: given orders and a name/id -> external id lookup, return the list of
(order, expected_external_id) pairs that need a metafieldsSet call."""
plan = []
for order in orders:
expected = external_id_lookup.get(order["name"])
if needs_repair(order, expected):
plan.append((order, expected))
return plan
def orders_missing_link():
"""Orders created in the lookback window, read with their external id metafield."""
q = "created_at:>-7d"
cursor = None
while True:
data = gql(ORDERS_QUERY, {"cursor": cursor, "q": q})["orders"]
for node in data["nodes"]:
yield node
if not data["pageInfo"]["hasNextPage"]:
return
cursor = data["pageInfo"]["endCursor"]
def set_external_id(order_id, external_id):
variables = {
"metafields": [{
"ownerId": order_id,
"namespace": METAFIELD_NAMESPACE,
"key": METAFIELD_KEY,
"type": "single_line_text_field",
"value": external_id,
}]
}
result = gql(METAFIELDS_SET, variables)["metafieldsSet"]
if result["userErrors"]:
raise RuntimeError(result["userErrors"])
return result["metafields"][0]["value"]
def run(external_id_lookup):
"""external_id_lookup maps order name (e.g. "#1001") to the external id it should
carry, coming from your own system of record. Wire this up to your database or API."""
repaired = 0
orders = list(orders_missing_link())
for order, expected in plan_repairs(orders, external_id_lookup):
log.warning(
"Order %s external id metafield %s. %s",
order["name"],
"missing" if not order.get("metafield") else "mismatched",
"would set" if DRY_RUN else "setting",
)
if not DRY_RUN:
set_external_id(order["id"], expected)
repaired += 1
log.info("Done. %d order(s) %s.", repaired, "to repair" if DRY_RUN else "repaired")
if __name__ == "__main__":
# Replace this with a real lookup, for example a query against your order system.
run(external_id_lookup={})/**
* Find orders whose external id metafield went missing on create, and set it back.
*
* Some integrations write a linking key onto the order the moment it is created, usually
* a metafield such as external_sync.external_id, so a warehouse system, a marketplace, or
* an ERP can match the Shopify order back to its own record. When the order is created by
* a flow that skips that write, such as a checkout that bypasses the app, a bulk import, or
* a race between two systems creating the order at the same time, the metafield is never
* set and the two systems can no longer find each other.
*
* This script reads recent orders, compares the external_sync.external_id metafield against
* the source of truth id you already have, and only repairs the ones that are missing or
* wrong, using metafieldsSet. It never touches an order whose metafield already matches.
* Run on a schedule. Safe to run again and again.
*/
import { pathToFileURL } from "node:url";
const SHOP = process.env.SHOPIFY_SHOP || "example.myshopify.com";
const TOKEN = process.env.SHOPIFY_ACCESS_TOKEN || "shpat_dummy";
const API_VERSION = process.env.SHOPIFY_API_VERSION || "2025-01";
const ENDPOINT = `https://${SHOP}/admin/api/${API_VERSION}/graphql.json`;
const DRY_RUN = (process.env.DRY_RUN || "true").toLowerCase() === "true";
const METAFIELD_NAMESPACE = process.env.METAFIELD_NAMESPACE || "external_sync";
const METAFIELD_KEY = process.env.METAFIELD_KEY || "external_id";
export function needsRepair(order, expectedExternalId) {
if (!expectedExternalId) return false;
const current = order.metafield ? order.metafield.value : null;
return current !== expectedExternalId;
}
export function planRepairs(orders, externalIdLookup) {
const plan = [];
for (const order of orders) {
const expected = externalIdLookup[order.name];
if (needsRepair(order, expected)) plan.push([order, expected]);
}
return plan;
}
async function gql(query, variables = {}) {
const res = await fetch(ENDPOINT, {
method: "POST",
headers: { "X-Shopify-Access-Token": TOKEN, "Content-Type": "application/json" },
body: JSON.stringify({ query, variables }),
});
if (!res.ok) throw new Error(`Shopify ${res.status}`);
const body = await res.json();
if (body.errors) throw new Error(JSON.stringify(body.errors));
return body.data;
}
const ORDERS_QUERY = `
query($cursor: String, $q: String!) {
orders(first: 50, after: $cursor, query: $q) {
pageInfo { hasNextPage endCursor }
nodes {
id
name
metafield(namespace: "external_sync", key: "external_id") { id value }
}
}
}`;
const METAFIELDS_SET = `
mutation($metafields: [MetafieldsSetInput!]!) {
metafieldsSet(metafields: $metafields) {
metafields { id key value }
userErrors { field message }
}
}`;
async function* ordersMissingLink() {
const q = "created_at:>-7d";
let cursor = null;
while (true) {
const data = (await gql(ORDERS_QUERY, { cursor, q })).orders;
for (const node of data.nodes) yield node;
if (!data.pageInfo.hasNextPage) return;
cursor = data.pageInfo.endCursor;
}
}
async function setExternalId(orderId, externalId) {
const variables = {
metafields: [{
ownerId: orderId,
namespace: METAFIELD_NAMESPACE,
key: METAFIELD_KEY,
type: "single_line_text_field",
value: externalId,
}],
};
const result = (await gql(METAFIELDS_SET, variables)).metafieldsSet;
if (result.userErrors.length) throw new Error(JSON.stringify(result.userErrors));
return result.metafields[0].value;
}
export async function run(externalIdLookup = {}) {
let repaired = 0;
const orders = [];
for await (const order of ordersMissingLink()) orders.push(order);
for (const [order, expected] of planRepairs(orders, externalIdLookup)) {
console.warn(
`Order ${order.name} external id metafield ${order.metafield ? "mismatched" : "missing"}. ${DRY_RUN ? "would set" : "setting"}`
);
if (!DRY_RUN) await setExternalId(order.id, expected);
repaired++;
}
console.log(`Done. ${repaired} order(s) ${DRY_RUN ? "to repair" : "repaired"}.`);
}
if (import.meta.url === pathToFileURL(process.argv[1]).href) {
// Replace this with a real lookup, for example a query against your order system.
run({}).catch((err) => { console.error(err); process.exit(1); });
}Add a test
The decision rule is the part most worth testing, because it decides which orders get written and which are left alone. Because we kept needs_repair and plan_repairs pure, the tests need no network and no Shopify account. They just feed in plain objects and check the answer.
from repair_external_id_metafield import needs_repair, plan_repairs
def order(name="#1001", value=None):
return {
"id": f"gid://shopify/Order/{name.strip('#')}",
"name": name,
"metafield": {"id": "gid://shopify/Metafield/1", "value": value} if value is not None else None,
}
def test_needs_repair_when_metafield_missing():
assert needs_repair(order(value=None), "ext-9001") is True
def test_needs_repair_when_value_does_not_match():
assert needs_repair(order(value="ext-old"), "ext-9001") is True
def test_no_repair_when_value_already_matches():
assert needs_repair(order(value="ext-9001"), "ext-9001") is False
def test_no_repair_when_no_expected_id_known():
assert needs_repair(order(value=None), None) is False
def test_plan_repairs_only_includes_mismatches():
orders = [
order(name="#1001", value=None),
order(name="#1002", value="ext-2002"),
order(name="#1003", value="ext-stale"),
]
lookup = {"#1001": "ext-1001", "#1002": "ext-2002", "#1003": "ext-3003"}
plan = plan_repairs(orders, lookup)
names = sorted(o["name"] for o, _ in plan)
assert names == ["#1001", "#1003"]import { test } from "node:test";
import assert from "node:assert/strict";
import { needsRepair, planRepairs } from "./repair-external-id-metafield.js";
const order = (name = "#1001", value = null) => ({
id: `gid://shopify/Order/${name.replace("#", "")}`,
name,
metafield: value !== null ? { id: "gid://shopify/Metafield/1", value } : null,
});
test("needs repair when metafield missing", () => {
assert.equal(needsRepair(order("#1001", null), "ext-9001"), true);
});
test("no repair when value already matches", () => {
assert.equal(needsRepair(order("#1001", "ext-9001"), "ext-9001"), false);
});
test("plan repairs only includes mismatches", () => {
const orders = [
order("#1001", null),
order("#1002", "ext-2002"),
order("#1003", "ext-stale"),
];
const lookup = { "#1001": "ext-1001", "#1002": "ext-2002", "#1003": "ext-3003" };
const plan = planRepairs(orders, lookup);
const names = plan.map(([o]) => o.name).sort();
assert.deepEqual(names, ["#1001", "#1003"]);
});Case studies
The 3PL that could not find the order
A homeware brand had a warehouse partner that pulled orders by their external id every hour. During a busy sale weekend, a handful of orders came in through a fallback checkout path the app did not watch, and their metafield was never set. The warehouse system quietly skipped those orders, and customers waited days past the promised ship date before support noticed.
Now the audit script runs every hour, compares Shopify's metafield against the order system's own record, and repairs the handful that drift. Nothing ships late because the two systems lost each other anymore.
The migration that skipped a hundred orders
A merchant migrated a year of historical orders into Shopify using a bulk import tool. The tool created the orders directly, bypassing the usual checkout webhook, so none of those orders got an external id metafield even though the source system already had one for each.
The team ran the script in dry run first, saw exactly which hundred orders needed a value and what that value should be, agreed the list was right, then let it write. The historical orders are now linked the same way as every order created since.
After this runs on a schedule, a dropped metafield is caught within an hour instead of surfacing as a support ticket days later. Every order the script touches is one whose value truly disagreed with your own system of record, so the audit is trustworthy, and orders that were always correct are never rewritten for no reason.
FAQ
Why does the external id metafield go missing on a new Shopify order?
The metafield is usually written by an app or a script right after the order is created, as a second step. If that order came from a path the app never watches, such as a bulk import, a draft order completed a different way, or a race where two systems create the order at nearly the same time, that second step never runs and the order is left without its linking key.
Is it safe to backfill the external id metafield with a script?
Yes, when the script only ever writes an order whose metafield is missing or does not match the value from your own system of record, and leaves alone any order where the metafield already agrees. Running in dry run first lets you see the exact list before anything is written.
What is metafieldsSet and why use it here?
metafieldsSet is the Admin GraphQL mutation for creating or updating a metafield on an owner such as an order, in one call, with the namespace, key, type, and value you choose. It reports userErrors, so a bad namespace, key, or type is caught immediately instead of silently failing.
Related field notes
Citations
On the problem:
- Shopify.dev: Metafields overview, including how apps attach custom data to resources like orders. shopify.dev/docs/apps/build/custom-data
- Shopify.dev: Admin webhooks, including orders/create and why a subscription can be missing or disabled. shopify.dev/docs/api/webhooks
- Shopify Community: apps missing metafield writes on orders created outside the usual checkout flow. community.shopify.com graphql admin api
On the solution:
- Shopify Admin GraphQL: the
metafieldsSetmutation. shopify.dev/docs/api/admin-graphql/latest/mutations/metafieldsSet - Shopify Admin GraphQL: the Order object, including the
metafieldfield and its namespace and key arguments. shopify.dev/docs/api/admin-graphql/latest/objects/Order - Shopify Admin GraphQL: the
ordersquery and its search syntax. shopify.dev/docs/api/admin-graphql/latest/queries/orders
Stuck on a tricky one?
If you have a problem in Shopify orders, payments, subscriptions, inventory, or fulfillment that you would rather hand off, this is the kind of work I do. Message me and we can work through it together.
Did this find your dropped links?
If this saved your integration from a pile of silently orphaned orders, you can buy me a coffee. It is the best way to keep these field notes free and growing.
Buy me a coffee on Ko-fi