allanninal.dev / bigcommerce

Field notes / Pricing / Price Lists

Repair Pricing / Price Lists

Concurrent price list bulk upserts fail the whole batch with 429

A nightly sync fans out one bulk upsert job per price list. Most of the time it works. Some nights, one or more of those jobs comes back with a 429 and none of its up to 1000 price records land, silently. BigCommerce allows only one in-flight bulk upsert job per store at a time on the price list records endpoint, no matter which price list each job targets, and a 429 from that lock drops the entire batch rather than applying it partially. Here is why that collision happens and a small script that serializes the writes and retries safely.

Python and Node.js BigCommerce V3 Price Lists API Safe by default (dry run)
A calculator sitting on top of a desk next to a laptop
Photo by Mehdi Mirzaie on Unsplash
The short answer

BigCommerce serializes writes to Price List records at the store level. The bulk upsert endpoint, PUT /v3/pricelists/{price_list_id}/records, allows only one in-flight bulk upsert job per store at a time, regardless of which price list is targeted. When multiple jobs, cron tasks, or app instances submit bulk PUT batches concurrently, the platform's price-list processing lock rejects every overlapping request with HTTP 429, and unlike a partial-batch validation error, the entire batch is dropped rather than partially applied. Fix it by acquiring a per-store lock (for example a Redis key pricelist:{store_hash}:bulk_lock) before every bulk PUT, queuing competing jobs instead of racing them, and on a 429 backing off with jitter and resubmitting the identical batch, since the upsert is idempotent on variant_id or sku plus price_list_id and currency. Full code, tests, and a dry run guard are below.

The problem in plain words

Price List records in BigCommerce are not just another catalog resource you can hammer with parallel writes. The bulk upsert endpoint processes each PUT as a job against the store's price list subsystem, and that subsystem only lets one such job run at a time, store-wide. It does not matter that job A is writing to price list 12 and job B is writing to price list 47. They still collide, because the lock is scoped to the store, not to the individual price list.

This becomes a problem the moment more than one thing can submit a bulk upsert: a nightly sync that fans out a batch per price list, a retry that fires while the original request is still being processed, or a second app instance running the same job. The second (and any later) overlapping request gets HTTP 429 back immediately. The catch is what that 429 actually means here. It is not the general API rate limit bucket telling you to slow down, and it is not a partial success where some records made it in. It is a flat rejection of the whole call, so none of the up to 1000 records in that batch are upserted. A caller that only checks for a 2xx and moves on, or that blindly retries without backing off, ends up quietly losing whole batches of price, sale_price, and map_price updates, with no partial-failure signal to notice by.

Job A: PUT records price_list_id 12 Job B: PUT records price_list_id 47 Store-level bulk upsert lock one job in flight per store, any price list ✓ Job A batch up to 1000 records upserted ✕ 429, lock held Job B batch whole batch dropped
The lock is store-wide, not per price list. Job B's records for price list 47 are not partially applied, they are dropped entirely, because the store already had job A's batch in flight.

Why it happens

A few concrete situations reliably trigger this collision:

See the citations at the end for the price list records API reference, a community thread on bulk price updates, and BigCommerce's own rate limit documentation.

The key insight

This 429 is not the same signal as running out of API rate limit. It is a concurrency lock on price list processing, scoped to the whole store, not to an individual price list. Confirm which one you are looking at by checking the response headers: general rate limit exhaustion carries X-Rate-Limit-Requests-Left: 0, while a concurrent bulk-upsert collision typically does not, and instead is best confirmed by cross-referencing your own job registry or lock table to see that a second job attempted a write while another job's lock was still held. Once you know it is the concurrency lock, the fix is serialization, not just generic rate-limit backoff.

The fix, as a flow

We do not change what gets written. We add a per-store lock that every bulk upsert job must acquire before calling the records endpoint, queue anything that arrives while the lock is held, and on a 429 back off and resubmit the identical batch rather than treating it as a failure to abandon.

Bulk upsert job ready to submit Acquire per-store bulk_lock (queue if held) PUT records one batch, one price list Status 200/201/207? yes 429, backoff, resubmit Release lock batch upserted
The lock forces one bulk upsert in flight per store. A 429 is not a failure to give up on, it is a signal to back off and resubmit the same idempotent batch once the lock clears.

Build it step by step

1

Get a store hash and an API access token

Create an API account in your BigCommerce control panel under Settings, API, or use the store's existing app credentials. Grant it Price Lists (modify) scope so it can list and write price list records. You need the store hash from your control panel URL and the access token, sent on every call as the X-Auth-Token header. Keep both in environment variables, never in the file.

setup (shell)
pip install requests

export BIGCOMMERCE_STORE_HASH="abc123"
export BIGCOMMERCE_ACCESS_TOKEN="..."
export MAX_ATTEMPTS="6"
export DRY_RUN="true"   # start safe, change to false to write
setup (shell)
// Node 18+ has fetch built in, no dependencies needed

export BIGCOMMERCE_STORE_HASH="abc123"
export BIGCOMMERCE_ACCESS_TOKEN="..."
export MAX_ATTEMPTS="6"
export DRY_RUN="true"   // start safe, change to false to write
2

Talk to the V3 Price Lists REST API

Every call goes to https://api.bigcommerce.com/stores/{store_hash}/v3/ with the token in the X-Auth-Token header. A small helper handles GET and PUT, returns the parsed JSON body alongside the status code and headers (we need those for the 429 case), and never raises on a 429 so the caller can decide what to do with it.

step2.py
import os, requests

STORE_HASH = os.environ["BIGCOMMERCE_STORE_HASH"]
ACCESS_TOKEN = os.environ["BIGCOMMERCE_ACCESS_TOKEN"]
API_BASE = f"https://api.bigcommerce.com/stores/{STORE_HASH}/v3"

HEADERS = {
    "X-Auth-Token": ACCESS_TOKEN,
    "Content-Type": "application/json",
    "Accept": "application/json",
}

def bc_get(path, params=None):
    r = requests.get(f"{API_BASE}{path}", headers=HEADERS, params=params or {}, timeout=30)
    r.raise_for_status()
    return r.json()

def bc_put_records(price_list_id, records):
    r = requests.put(
        f"{API_BASE}/pricelists/{price_list_id}/records",
        headers=HEADERS, json=records, timeout=60,
    )
    body = r.json() if r.text else {}
    return r.status_code, dict(r.headers), body
step2.js
const STORE_HASH = process.env.BIGCOMMERCE_STORE_HASH;
const ACCESS_TOKEN = process.env.BIGCOMMERCE_ACCESS_TOKEN;
const API_BASE = `https://api.bigcommerce.com/stores/${STORE_HASH}/v3`;

const HEADERS = {
  "X-Auth-Token": ACCESS_TOKEN,
  "Content-Type": "application/json",
  Accept: "application/json",
};

async function bcGet(path, params = {}) {
  const url = new URL(`${API_BASE}${path}`);
  for (const [key, value] of Object.entries(params)) url.searchParams.set(key, value);
  const res = await fetch(url, { headers: HEADERS });
  if (!res.ok) throw new Error(`BigCommerce ${res.status}`);
  return res.json();
}

async function bcPutRecords(priceListId, records) {
  const res = await fetch(`${API_BASE}/pricelists/${priceListId}/records`, {
    method: "PUT",
    headers: HEADERS,
    body: JSON.stringify(records),
  });
  const text = await res.text();
  const body = text ? JSON.parse(text) : {};
  return { status: res.status, headers: Object.fromEntries(res.headers.entries()), body };
}
3

Enumerate price lists and hold a per-store lock

Call GET /v3/pricelists?limit=250, paginated via meta.pagination, to get every price_list_id and its active/name state. Before any job issues a bulk PUT for one of those price lists, it must acquire a per-store lock, for example SETNX pricelist:{store_hash}:bulk_lock in Redis with a TTL, or an in-process semaphore of size 1 if everything runs in one process. Competing jobs queue instead of racing, and the lock is released only after the PUT call returns something other than 429.

step3.py
import threading

def all_price_lists():
    page = 1
    while True:
        payload = bc_get("/pricelists", {"limit": 250, "page": page})
        data = payload.get("data") or []
        if not data:
            return
        for price_list in data:
            yield price_list
        pagination = (payload.get("meta") or {}).get("pagination") or {}
        if page >= (pagination.get("total_pages") or page):
            return
        page += 1

# In-process serialization. Swap for a Redis SETNX lock across multiple hosts.
STORE_BULK_LOCK = threading.Lock()
step3.js
async function* allPriceLists() {
  let page = 1;
  while (true) {
    const payload = await bcGet("/pricelists", { limit: 250, page });
    const data = payload.data || [];
    if (!data.length) return;
    for (const priceList of data) yield priceList;
    const pagination = (payload.meta || {}).pagination || {};
    if (page >= (pagination.total_pages || page)) return;
    page += 1;
  }
}

// In-process serialization. Swap for a Redis SETNX lock across multiple hosts.
let storeBulkLockHeld = false;
async function withStoreBulkLock(fn) {
  while (storeBulkLockHeld) await new Promise((r) => setTimeout(r, 50));
  storeBulkLockHeld = true;
  try {
    return await fn();
  } finally {
    storeBulkLockHeld = false;
  }
}
4

Decide, with one pure function

Keep the retry decision in its own function that takes the response status, the attempt count, and the response headers, and returns exactly what to do next. Success statuses (200, 201, 207) mean the batch was accepted. A 429 means retry with a computed wait, honoring X-Rate-Limit-Time-Reset-Ms or Retry-After if BigCommerce sent one, otherwise capped exponential backoff. Any other 4xx gives up immediately since it will not resolve itself. A 5xx is treated as transient and retried the same as a 429, up to the attempt cap.

decide.py
def decide_retry(status_code, attempt, headers, max_attempts=6):
    if status_code in (200, 201, 207):
        return {"action": "success"}

    if status_code == 429:
        if attempt >= max_attempts:
            return {"action": "give_up", "reason": "max_attempts_exceeded"}
        wait_ms = _compute_wait_ms(attempt, headers)
        return {"action": "retry", "wait_ms": wait_ms, "reason": "concurrent_bulk_lock"}

    if 500 <= status_code < 600:
        if attempt >= max_attempts:
            return {"action": "give_up", "reason": "server_error_max_attempts"}
        wait_ms = _compute_wait_ms(attempt, headers)
        return {"action": "retry", "wait_ms": wait_ms, "reason": "server_error"}

    return {"action": "give_up", "reason": "client_error_non_retryable"}


def _compute_wait_ms(attempt, headers):
    headers = headers or {}
    reset_ms = headers.get("X-Rate-Limit-Time-Reset-Ms")
    if reset_ms is not None:
        try:
            return int(reset_ms)
        except (TypeError, ValueError):
            pass
    retry_after = headers.get("Retry-After")
    if retry_after is not None:
        try:
            return int(float(retry_after) * 1000)
        except (TypeError, ValueError):
            pass
    return min(60000, 2000 * (2 ** (attempt - 1)))
decide.js
export function decideRetry(statusCode, attempt, headers, maxAttempts = 6) {
  if ([200, 201, 207].includes(statusCode)) {
    return { action: "success" };
  }

  if (statusCode === 429) {
    if (attempt >= maxAttempts) {
      return { action: "give_up", reason: "max_attempts_exceeded" };
    }
    return { action: "retry", wait_ms: computeWaitMs(attempt, headers), reason: "concurrent_bulk_lock" };
  }

  if (statusCode >= 500 && statusCode < 600) {
    if (attempt >= maxAttempts) {
      return { action: "give_up", reason: "server_error_max_attempts" };
    }
    return { action: "retry", wait_ms: computeWaitMs(attempt, headers), reason: "server_error" };
  }

  return { action: "give_up", reason: "client_error_non_retryable" };
}

function computeWaitMs(attempt, headers) {
  headers = headers || {};
  const resetMs = headers["X-Rate-Limit-Time-Reset-Ms"] ?? headers["x-rate-limit-time-reset-ms"];
  if (resetMs !== undefined && resetMs !== null && !Number.isNaN(Number(resetMs))) {
    return Number(resetMs);
  }
  const retryAfter = headers["Retry-After"] ?? headers["retry-after"];
  if (retryAfter !== undefined && retryAfter !== null && !Number.isNaN(Number(retryAfter))) {
    return Number(retryAfter) * 1000;
  }
  return Math.min(60000, 2000 * 2 ** (attempt - 1));
}
5

Submit inside the lock, retry on the identical batch

When it is a job's turn to hold the lock, it submits the batch, feeds the response into decide_retry, and either releases the lock on success, waits and resubmits the same records array on retry, or releases the lock and gives up, logging the reason. Because the endpoint upserts on variant_id or sku plus price_list_id and currency, resubmitting the identical batch never double-applies anything.

apply.py
import time

def submit_batch_with_retry(price_list_id, records, max_attempts=6, dry_run=True):
    attempt = 1
    while True:
        if dry_run:
            return {"action": "success", "dry_run": True, "attempt": attempt}

        status_code, headers, body = bc_put_records(price_list_id, records)
        decision = decide_retry(status_code, attempt, headers, max_attempts)

        if decision["action"] == "success":
            return {"action": "success", "attempt": attempt, "body": body}
        if decision["action"] == "give_up":
            return decision

        time.sleep(decision["wait_ms"] / 1000)
        attempt += 1
apply.js
async function submitBatchWithRetry(priceListId, records, maxAttempts = 6, dryRun = true) {
  let attempt = 1;
  while (true) {
    if (dryRun) {
      return { action: "success", dry_run: true, attempt };
    }

    const { status, headers, body } = await bcPutRecords(priceListId, records);
    const decision = decideRetry(status, attempt, headers, maxAttempts);

    if (decision.action === "success") return { action: "success", attempt, body };
    if (decision.action === "give_up") return decision;

    await new Promise((r) => setTimeout(r, decision.wait_ms));
    attempt += 1;
  }
}
6

Wire it together with a dry run guard

The run loop takes each pending job, acquires the store lock, submits with retry, and releases the lock. Notice the dry run guard. Under DRY_RUN=true, the job only logs the planned serialized submission order and the lock or queue wait time per job, it never acquires a real lock or issues the PUT. Read the output, agree with it, then switch it off and let it serialize your real jobs.

Run it safe

Always start with DRY_RUN=true, never fan out bulk upsert jobs in parallel against the same store, and never treat a 429 on this endpoint as proof the batch is gone for good without first checking whether it should simply be retried after the lock clears.

The full code

Here is the complete script in one file for each language. It reads settings from the environment, serializes bulk upserts with a lock, retries a 429 with backoff on the identical idempotent batch, and respects the dry run flag so nothing writes until you are ready.

View this code on GitHub Full runnable folder with tests in the bigcommerce-fixes repo.

serialize_price_list_upserts.py
"""Serialize BigCommerce price list bulk upserts so concurrent jobs stop losing batches to 429.

BigCommerce serializes writes to Price List records at the store level. The bulk
upsert endpoint, PUT /v3/pricelists/{price_list_id}/records, allows only one
in-flight bulk upsert job per store at a time, regardless of which price list is
targeted. When multiple jobs, cron tasks, or app instances submit bulk PUT batches
concurrently, the platform's price-list processing lock rejects every overlapping
request with HTTP 429, and unlike a partial-batch validation error, the entire
batch is dropped rather than partially applied. This script acquires a per-store
lock before every bulk PUT, queues competing jobs instead of racing them, and on a
429 backs off with jitter and resubmits the identical batch, which is safe because
the endpoint upserts on variant_id or sku plus price_list_id and currency. Run one
instance per store. Safe to run again and again.

Guide: https://www.allanninal.dev/bigcommerce/concurrent-price-list-upserts-429/
"""
import os
import time
import random
import logging
import threading

import requests

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("serialize_price_list_upserts")

STORE_HASH = os.environ["BIGCOMMERCE_STORE_HASH"]
ACCESS_TOKEN = os.environ["BIGCOMMERCE_ACCESS_TOKEN"]
API_BASE = f"https://api.bigcommerce.com/stores/{STORE_HASH}/v3"
MAX_ATTEMPTS = int(os.environ.get("MAX_ATTEMPTS", "6"))
DRY_RUN = os.environ.get("DRY_RUN", "true").lower() == "true"

HEADERS = {
    "X-Auth-Token": ACCESS_TOKEN,
    "Content-Type": "application/json",
    "Accept": "application/json",
}

# In-process serialization for a single-host scheduler. For multiple hosts or
# processes, swap this for a Redis lock, e.g. SETNX pricelist:{store_hash}:bulk_lock
# with a TTL, released only after the PUT call returns a non-429 status.
_STORE_BULK_LOCK = threading.Lock()


def decide_retry(status_code, attempt, headers, max_attempts=6):
    """Pure decision logic (no I/O) for handling a price-list bulk-upsert response.

    status 200/201/207 -> success (batch accepted/upserted)
    status 429 and attempt < max_attempts -> retry with a computed wait_ms,
        reason "concurrent_bulk_lock"
    status 429 and attempt >= max_attempts -> give_up, "max_attempts_exceeded"
    status 4xx (not 429) -> give_up, "client_error_non_retryable"
    status 5xx -> retry (transient) up to max_attempts, else give_up
        "server_error_max_attempts"
    """
    if status_code in (200, 201, 207):
        return {"action": "success"}

    if status_code == 429:
        if attempt >= max_attempts:
            return {"action": "give_up", "reason": "max_attempts_exceeded"}
        return {
            "action": "retry",
            "wait_ms": _compute_wait_ms(attempt, headers),
            "reason": "concurrent_bulk_lock",
        }

    if 500 <= status_code < 600:
        if attempt >= max_attempts:
            return {"action": "give_up", "reason": "server_error_max_attempts"}
        return {
            "action": "retry",
            "wait_ms": _compute_wait_ms(attempt, headers),
            "reason": "server_error",
        }

    return {"action": "give_up", "reason": "client_error_non_retryable"}


def _compute_wait_ms(attempt, headers):
    headers = headers or {}
    reset_ms = headers.get("X-Rate-Limit-Time-Reset-Ms")
    if reset_ms is not None:
        try:
            return int(reset_ms)
        except (TypeError, ValueError):
            pass
    retry_after = headers.get("Retry-After")
    if retry_after is not None:
        try:
            return int(float(retry_after) * 1000)
        except (TypeError, ValueError):
            pass
    base = min(60000, 2000 * (2 ** (attempt - 1)))
    jitter = random.randint(0, 250)
    return base + jitter


def bc_get(path, params=None):
    r = requests.get(f"{API_BASE}{path}", headers=HEADERS, params=params or {}, timeout=30)
    r.raise_for_status()
    return r.json()


def bc_put_records(price_list_id, records):
    r = requests.put(
        f"{API_BASE}/pricelists/{price_list_id}/records",
        headers=HEADERS,
        json=records,
        timeout=60,
    )
    body = r.json() if r.text else {}
    return r.status_code, dict(r.headers), body


def all_price_lists():
    page = 1
    while True:
        payload = bc_get("/pricelists", {"limit": 250, "page": page})
        data = payload.get("data") or []
        if not data:
            return
        for price_list in data:
            yield price_list
        pagination = (payload.get("meta") or {}).get("pagination") or {}
        if page >= (pagination.get("total_pages") or page):
            return
        page += 1


def submit_batch_with_retry(price_list_id, records, max_attempts=MAX_ATTEMPTS, dry_run=DRY_RUN):
    attempt = 1
    while True:
        if dry_run:
            log.info(
                "DRY RUN: would submit batch price_list_id=%s records=%d attempt=%d",
                price_list_id, len(records), attempt,
            )
            return {"action": "success", "dry_run": True, "attempt": attempt}

        status_code, headers, body = bc_put_records(price_list_id, records)
        decision = decide_retry(status_code, attempt, headers, max_attempts)

        if decision["action"] == "success":
            log.info(
                "price_list_id=%s records=%d upserted on attempt %d",
                price_list_id, len(records), attempt,
            )
            return {"action": "success", "attempt": attempt, "body": body}

        if decision["action"] == "give_up":
            log.error(
                "price_list_id=%s gave up after attempt %d: %s",
                price_list_id, attempt, decision["reason"],
            )
            return decision

        log.warning(
            "price_list_id=%s got %s on attempt %d, retrying in %dms (%s)",
            price_list_id, status_code, attempt, decision["wait_ms"], decision["reason"],
        )
        time.sleep(decision["wait_ms"] / 1000)
        attempt += 1


def run_job(price_list_id, records):
    """Acquire the per-store lock, submit with retry, release the lock."""
    if DRY_RUN:
        log.info(
            "DRY RUN: job for price_list_id=%s queued, would wait for store lock",
            price_list_id,
        )
        return submit_batch_with_retry(price_list_id, records)

    with _STORE_BULK_LOCK:
        return submit_batch_with_retry(price_list_id, records)


def run(jobs):
    """jobs: iterable of (price_list_id, records) tuples to submit, one at a time."""
    results = []
    for price_list_id, records in jobs:
        results.append(run_job(price_list_id, records))
    succeeded = sum(1 for r in results if r["action"] == "success")
    log.info("Done. %d/%d job(s) succeeded.", succeeded, len(results))
    return results


if __name__ == "__main__":
    example_jobs = [
        (price_list["id"], [])
        for price_list in all_price_lists()
    ]
    run(example_jobs)
serialize-price-list-upserts.js
/**
 * Serialize BigCommerce price list bulk upserts so concurrent jobs stop losing batches to 429.
 *
 * BigCommerce serializes writes to Price List records at the store level. The bulk
 * upsert endpoint, PUT /v3/pricelists/{price_list_id}/records, allows only one
 * in-flight bulk upsert job per store at a time, regardless of which price list is
 * targeted. When multiple jobs, cron tasks, or app instances submit bulk PUT batches
 * concurrently, the platform's price-list processing lock rejects every overlapping
 * request with HTTP 429, and unlike a partial-batch validation error, the entire
 * batch is dropped rather than partially applied. This script acquires a per-store
 * lock before every bulk PUT, queues competing jobs instead of racing them, and on a
 * 429 backs off with jitter and resubmits the identical batch, which is safe because
 * the endpoint upserts on variant_id or sku plus price_list_id and currency. Run one
 * instance per store.
 *
 * Guide: https://www.allanninal.dev/bigcommerce/concurrent-price-list-upserts-429/
 */
import { pathToFileURL } from "node:url";

const STORE_HASH = process.env.BIGCOMMERCE_STORE_HASH || "example_hash";
const ACCESS_TOKEN = process.env.BIGCOMMERCE_ACCESS_TOKEN || "bc_dummy";
const API_BASE = `https://api.bigcommerce.com/stores/${STORE_HASH}/v3`;
const MAX_ATTEMPTS = Number(process.env.MAX_ATTEMPTS || 6);
const DRY_RUN = (process.env.DRY_RUN || "true").toLowerCase() === "true";

const HEADERS = {
  "X-Auth-Token": ACCESS_TOKEN,
  "Content-Type": "application/json",
  Accept: "application/json",
};

// In-process serialization for a single-host scheduler. For multiple hosts or
// processes, swap this for a Redis lock, e.g. SETNX pricelist:{store_hash}:bulk_lock
// with a TTL, released only after the PUT call returns a non-429 status.
let storeBulkLockHeld = false;
async function withStoreBulkLock(fn) {
  while (storeBulkLockHeld) await new Promise((r) => setTimeout(r, 25));
  storeBulkLockHeld = true;
  try {
    return await fn();
  } finally {
    storeBulkLockHeld = false;
  }
}

/**
 * Pure decision logic (no I/O) for handling a price-list bulk-upsert response.
 *
 * status 200/201/207 -> success (batch accepted/upserted)
 * status 429 and attempt < maxAttempts -> retry with a computed wait_ms,
 *     reason "concurrent_bulk_lock"
 * status 429 and attempt >= maxAttempts -> give_up, "max_attempts_exceeded"
 * status 4xx (not 429) -> give_up, "client_error_non_retryable"
 * status 5xx -> retry (transient) up to maxAttempts, else give_up
 *     "server_error_max_attempts"
 */
export function decideRetry(statusCode, attempt, headers, maxAttempts = 6) {
  if ([200, 201, 207].includes(statusCode)) {
    return { action: "success" };
  }

  if (statusCode === 429) {
    if (attempt >= maxAttempts) {
      return { action: "give_up", reason: "max_attempts_exceeded" };
    }
    return {
      action: "retry",
      wait_ms: computeWaitMs(attempt, headers),
      reason: "concurrent_bulk_lock",
    };
  }

  if (statusCode >= 500 && statusCode < 600) {
    if (attempt >= maxAttempts) {
      return { action: "give_up", reason: "server_error_max_attempts" };
    }
    return {
      action: "retry",
      wait_ms: computeWaitMs(attempt, headers),
      reason: "server_error",
    };
  }

  return { action: "give_up", reason: "client_error_non_retryable" };
}

function computeWaitMs(attempt, headers) {
  headers = headers || {};
  const resetMs = headers["X-Rate-Limit-Time-Reset-Ms"] ?? headers["x-rate-limit-time-reset-ms"];
  if (resetMs !== undefined && resetMs !== null && !Number.isNaN(Number(resetMs))) {
    return Number(resetMs);
  }
  const retryAfter = headers["Retry-After"] ?? headers["retry-after"];
  if (retryAfter !== undefined && retryAfter !== null && !Number.isNaN(Number(retryAfter))) {
    return Number(retryAfter) * 1000;
  }
  const base = Math.min(60000, 2000 * 2 ** (attempt - 1));
  const jitter = Math.floor(Math.random() * 250);
  return base + jitter;
}

async function bcGet(path, params = {}) {
  const url = new URL(`${API_BASE}${path}`);
  for (const [key, value] of Object.entries(params)) {
    if (value !== undefined && value !== null) url.searchParams.set(key, value);
  }
  const res = await fetch(url, { headers: HEADERS });
  if (!res.ok) throw new Error(`BigCommerce ${res.status}`);
  return res.json();
}

async function bcPutRecords(priceListId, records) {
  const res = await fetch(`${API_BASE}/pricelists/${priceListId}/records`, {
    method: "PUT",
    headers: HEADERS,
    body: JSON.stringify(records),
  });
  const text = await res.text();
  const body = text ? JSON.parse(text) : {};
  return { status: res.status, headers: Object.fromEntries(res.headers.entries()), body };
}

async function* allPriceLists() {
  let page = 1;
  while (true) {
    const payload = await bcGet("/pricelists", { limit: 250, page });
    const data = payload.data || [];
    if (!data.length) return;
    for (const priceList of data) yield priceList;
    const pagination = (payload.meta || {}).pagination || {};
    if (page >= (pagination.total_pages || page)) return;
    page += 1;
  }
}

async function submitBatchWithRetry(priceListId, records, maxAttempts = MAX_ATTEMPTS, dryRun = DRY_RUN) {
  let attempt = 1;
  while (true) {
    if (dryRun) {
      console.log(
        `DRY RUN: would submit batch price_list_id=${priceListId} records=${records.length} attempt=${attempt}`
      );
      return { action: "success", dry_run: true, attempt };
    }

    const { status, headers, body } = await bcPutRecords(priceListId, records);
    const decision = decideRetry(status, attempt, headers, maxAttempts);

    if (decision.action === "success") {
      console.log(`price_list_id=${priceListId} records=${records.length} upserted on attempt ${attempt}`);
      return { action: "success", attempt, body };
    }

    if (decision.action === "give_up") {
      console.error(`price_list_id=${priceListId} gave up after attempt ${attempt}: ${decision.reason}`);
      return decision;
    }

    console.warn(
      `price_list_id=${priceListId} got ${status} on attempt ${attempt}, retrying in ${decision.wait_ms}ms (${decision.reason})`
    );
    await new Promise((r) => setTimeout(r, decision.wait_ms));
    attempt += 1;
  }
}

async function runJob(priceListId, records) {
  if (DRY_RUN) {
    console.log(`DRY RUN: job for price_list_id=${priceListId} queued, would wait for store lock`);
    return submitBatchWithRetry(priceListId, records);
  }
  return withStoreBulkLock(() => submitBatchWithRetry(priceListId, records));
}

export async function run(jobs) {
  const results = [];
  for (const [priceListId, records] of jobs) {
    results.push(await runJob(priceListId, records));
  }
  const succeeded = results.filter((r) => r.action === "success").length;
  console.log(`Done. ${succeeded}/${results.length} job(s) succeeded.`);
  return results;
}

if (import.meta.url === pathToFileURL(process.argv[1]).href) {
  (async () => {
    const jobs = [];
    for await (const priceList of allPriceLists()) {
      jobs.push([priceList.id, []]);
    }
    await run(jobs);
  })().catch((err) => { console.error(err); process.exit(1); });
}

Add a test

The retry decision is the part most worth testing, because it decides whether a real batch gets resubmitted, abandoned, or wrongly treated as done. Because decide_retry takes only plain values and returns a plain object, the test needs no network and no BigCommerce store. It just feeds in status codes, attempt counts, and headers, and checks the answer.

test_concurrent_bulk_retry.py
from serialize_price_list_upserts import decide_retry


def test_success_on_200():
    assert decide_retry(200, 1, {}) == {"action": "success"}


def test_success_on_207_multi_status():
    assert decide_retry(207, 1, {}) == {"action": "success"}


def test_429_retries_with_reset_header_ms():
    result = decide_retry(429, 1, {"X-Rate-Limit-Time-Reset-Ms": "1500"}, max_attempts=6)
    assert result["action"] == "retry"
    assert result["wait_ms"] == 1500
    assert result["reason"] == "concurrent_bulk_lock"


def test_429_retries_with_retry_after_seconds():
    result = decide_retry(429, 1, {"Retry-After": "2"}, max_attempts=6)
    assert result["action"] == "retry"
    assert result["wait_ms"] == 2000


def test_429_falls_back_to_capped_exponential_backoff():
    result = decide_retry(429, 3, {}, max_attempts=6)
    assert result["action"] == "retry"
    # base 2000 * 2**(attempt-1) = 8000, plus up to 250ms of jitter
    assert 8000 <= result["wait_ms"] <= 8250


def test_429_gives_up_after_max_attempts():
    result = decide_retry(429, 6, {}, max_attempts=6)
    assert result == {"action": "give_up", "reason": "max_attempts_exceeded"}


def test_non_429_client_error_gives_up_immediately():
    result = decide_retry(422, 1, {}, max_attempts=6)
    assert result == {"action": "give_up", "reason": "client_error_non_retryable"}


def test_server_error_retries_then_gives_up():
    retry = decide_retry(503, 1, {}, max_attempts=2)
    assert retry["action"] == "retry"
    give_up = decide_retry(503, 2, {}, max_attempts=2)
    assert give_up == {"action": "give_up", "reason": "server_error_max_attempts"}
serialize-price-list-upserts.test.js
import { test } from "node:test";
import assert from "node:assert/strict";
import { decideRetry } from "./serialize-price-list-upserts.js";

test("success on 200", () => {
  assert.deepEqual(decideRetry(200, 1, {}), { action: "success" });
});

test("success on 207 multi-status", () => {
  assert.deepEqual(decideRetry(207, 1, {}), { action: "success" });
});

test("429 retries with reset header ms", () => {
  const result = decideRetry(429, 1, { "X-Rate-Limit-Time-Reset-Ms": "1500" }, 6);
  assert.equal(result.action, "retry");
  assert.equal(result.wait_ms, 1500);
  assert.equal(result.reason, "concurrent_bulk_lock");
});

test("429 retries with retry-after seconds", () => {
  const result = decideRetry(429, 1, { "Retry-After": "2" }, 6);
  assert.equal(result.action, "retry");
  assert.equal(result.wait_ms, 2000);
});

test("429 falls back to capped exponential backoff", () => {
  const result = decideRetry(429, 3, {}, 6);
  assert.equal(result.action, "retry");
  // base 2000 * 2**(attempt-1) = 8000, plus up to 250ms of jitter
  assert.ok(result.wait_ms >= 8000 && result.wait_ms <= 8250);
});

test("429 gives up after max attempts", () => {
  assert.deepEqual(decideRetry(429, 6, {}, 6), { action: "give_up", reason: "max_attempts_exceeded" });
});

test("non-429 client error gives up immediately", () => {
  assert.deepEqual(decideRetry(422, 1, {}, 6), { action: "give_up", reason: "client_error_non_retryable" });
});

test("server error retries then gives up", () => {
  const retry = decideRetry(503, 1, {}, 2);
  assert.equal(retry.action, "retry");
  const giveUp = decideRetry(503, 2, {}, 2);
  assert.deepEqual(giveUp, { action: "give_up", reason: "server_error_max_attempts" });
});

Case studies

Nightly sync fan-out

The store that synced every price list in parallel to save time

A merchant ran a nightly job that pulled fresh cost and margin data from an external system and pushed it into BigCommerce, one bulk upsert per price list, fired all at once to keep the sync fast. Every few nights, one or two price lists would simply not update, with no error surfaced anywhere except a 429 buried in a log no one was reading.

Switching the sync to acquire a single per-store lock before each bulk PUT, and queuing the rest, cost a few extra minutes of total run time but eliminated the silent drops entirely. Every price list updates every night now, in a predictable order.

Retry made it worse

The retry loop that collided with itself

A team had added a naive retry: on any non-2xx response, immediately resubmit. That "fix" made things worse, because the immediate resubmission fired while the original request's lock was often still held, generating a second 429 back to back, sometimes a third, before eventually giving up and logging a generic failure.

Feeding the response into decide_retry instead of retrying blindly fixed it. The wait_ms it returns, informed by X-Rate-Limit-Time-Reset-Ms when BigCommerce sends one, gives the lock time to actually clear before the resubmission, so the second attempt usually succeeds outright.

What good looks like

After this runs, every bulk price list upsert against a store goes through one lock, one at a time, so no job ever races another for the same store-level slot. A 429 is expected occasionally under load and handled by waiting and resubmitting the identical, safely idempotent batch, never by silently dropping price data or hammering the endpoint with an instant retry.

FAQ

Why does a bulk price list upsert fail with 429 when I have API rate limit left?

BigCommerce serializes writes to Price List records at the store level. The bulk upsert endpoint allows only one in-flight bulk upsert job per store at a time, regardless of which price list is targeted. A second overlapping PUT is rejected with 429 from the price-list processing lock, which is a different limit than the general API rate limit bucket, so you can see this 429 even when X-Rate-Limit-Requests-Left is well above zero.

When a bulk upsert batch gets a 429, did any of the records get applied?

No. Unlike a partial-batch validation error, a 429 from the concurrent-lock collision drops the entire batch. None of the up to 1000 records in that call are upserted, so you cannot assume partial progress and must resubmit the identical batch once the lock clears.

Is it safe to just resubmit the same batch after a 429?

Yes, once you back off and the lock has cleared. PUT /v3/pricelists/{id}/records is an idempotent upsert keyed on variant_id or sku plus price_list_id and currency, so resubmitting the identical batch does not create duplicates or double-apply anything. What is not safe is resubmitting immediately without a delay, or running multiple jobs in parallel against the same store, since that just recreates the collision.

Related field notes

BigCommerce orders stuck on Awaiting Payment after capture BigCommerce transactions total does not match the order Browse all BigCommerce field notes

Citations

On the problem:

  1. BigCommerce Developer Center: Price Lists Records reference. developer.bigcommerce.com price lists records
  2. BigCommerce Support: bulk price update based on SKU, community question and answer. support.bigcommerce.com bulk price update based on SKU
  3. BigCommerce Developer Center: API rate limits. developer.bigcommerce.com API rate limits

On the solution:

  1. BigCommerce API Reference: Create Batch of Price Lists Records. docs.bigcommerce.com create price lists records
  2. BigCommerce API Reference: Price Lists. docs.bigcommerce.com price lists
  3. BigCommerce Resource Hub: Navigating BigCommerce's API Rate Limits Update. developer.bigcommerce.com navigating API rate limits update

Stuck on a tricky one?

If you have a problem in BigCommerce orders, payments, webhooks, inventory, or pricing that you would rather hand off, this is the kind of work I do. Message me and we can work through it together.

Contact me on LinkedIn

Did this stop your nightly sync from losing batches?

If this saved you a pile of silently dropped price updates or a confusing 429 chase, you can buy me a coffee. It is the best way to keep these field notes free and growing.

Buy me a coffee on Ko-fi

Back to all BigCommerce field notes