---
title: "Cloning a Shopify store you can run twice — Hypermetron"
description: "Copying a Shopify store once is easy. Copying it again without duplicates, broken references or wiped data is the real work. How HyperSync does it, and the bugs that taught us."
canonical: "https://hypermetron.com/blog/cloning-a-shopify-store-you-can-run-twice"
lang: "en"
updated: "2026-11-06"
---

# Cloning a Shopify store you can run twice

Published 2026-11-06 · Shopify, Engineering

Copying a Shopify store once is easy. Copying it again without duplicates, broken references or wiped data is the real work. How HyperSync does it, and the bugs that taught us.

Shopify has no built-in way to clone a store. Teams that want a staging copy of production, or a second store per market, end up copying data with an app or a script. The first run is rarely the problem. The second run is: it either creates a second copy of everything, or quietly overwrites things it shouldn’t.

[HyperSync](https://hypersync.hypermetron.com/clone-shopify-store) is the Shopify app we built for this job. Its main promise is that a copy can be run again at any time and converge instead of duplicating. This post is about what that took, including the bugs that showed us where our first ideas were wrong.

## A mapping table, written after every resource

The core is a table that remembers which object in the destination store came from which object in the source. Its key is the two stores, the kind of resource and the source ID, so copying A to B and B to A are separate rows. Every write records its mapping straight away, as an upsert:

```ts
await db.insert(idMappings).values({
  sourceShop, targetShop, resourceKind, sourceId, targetId, pairId, firstSeenRunId,
}).onConflictDoUpdate({
  target: [idMappings.sourceShop, idMappings.targetShop, idMappings.resourceKind, idMappings.sourceId],
  // Upsert, so a re-run refreshes rather than failing.
  set: { targetId: input.targetId, updatedAt: new Date() },
});
```

Recording after each write, rather than at the end of a batch, matters more than it looks. If the worker dies between a write and its mapping, the next run doesn’t know the object exists and creates it again. The window can’t be closed completely, so we narrowed it to one resource.

Two details came from mistakes. The table used to be keyed on the store pair, which meant a file import (no pair) had nowhere to record anything. And the pair reference used to cascade on delete, so unpairing two stores wiped the history, and the next copy re-created the whole catalogue.

## When the bookkeeping is wrong

A mapping table is a cache, and caches go stale. Someone resets the staging store, or the product already existed before HyperSync was installed. So every kind of resource also has a natural key in the destination: the handle for products, collections, pages, blogs, articles and menus, the path for redirects, type and handle for metaobjects, the SKU for inventory.

For products, Shopify supports this directly. `productSet` accepts a handle as its identifier, so when no mapping exists the write itself finds the existing product instead of creating a second one. When a natural key matches something already in the destination, HyperSync updates it and says so in the report.

That last part became a rule after a first run planned “will create 1” and then overwrote a merchant’s existing `about` page. The preview now counts these matches separately, before anything runs, so nobody finds out from the result.

Two smaller lessons from the same area. The `menus` query silently ignores a `handle:` filter, so menus are listed in full and matched in memory; before that fix, the footer’s links were written into the main menu. And when a mapped ID no longer exists, we retry once without it, but only for kinds that have a natural key, and we recognise “not found” by Shopify’s error code, not its message. `FILE_DOES_NOT_EXIST` and `REFERENCE_TARGET_DOES_NOT_EXIST` read almost the same in English, and confusing them would replace a live file.

## References: never write a source ID

Store data is a graph. Metafields point at products and files, collections contain products, menus link to pages, metaobjects point at each other. Each resource type declares what it depends on, and the plan is a topological sort of those declarations. If someone adds a cycle, planning refuses instead of guessing an order.

Order isn’t enough, though, because some links point at objects that aren’t copied yet, or never will be. The rule we ended up with: a reference is only written if it was positively rewritten to a destination ID. A source ID and a destination ID have exactly the same shape, so “looks like a valid ID” can never be the test.

We learned that one the hard way. Until September, reference metafields were copied with the source store’s IDs in them. They looked fine and pointed nowhere, and it took a merchant’s support question to show it. When a target is missing now, the owner decides what happens:

- A metafield is left out, so the destination keeps its own value, and the report says why.
- A collection is created with the products that resolved, and only adds members until its list is known to be complete.
- A menu with any unresolved link is skipped whole. A menu with links pointing nowhere is worse than an unchanged one.

Running the copy again fills in what was missing, because by then the targets exist. That is also why the honest answer to “does it copy everything in one go?” is sometimes no: a metaobject that links to a product can need a second run.

## Lists replace, so never send half of one

`productSet` has replace semantics: leave a list out and the field is untouched, send an empty list and it is cleared. That makes a truncated read dangerous. If a nested list came back with more pages than we read, writing it would delete the rows we didn’t see. Any read that comes back truncated is now refused outright, and archive entries missing a required list are skipped rather than written.

Writes have limits that reads don’t. A collection with 280 products failed because a mutation takes at most 250 at a time; reads had been paginated, writes hadn’t. A build test now fails on any list sent in a mutation that isn’t classified as chunked or bounded.

## Long runs on a plain Postgres queue

A large copy runs for a long time, so it can’t live in a web request. Runs are split into units in a Postgres table, and separate worker processes claim them with `FOR UPDATE SKIP LOCKED` and hold a lease they renew with a heartbeat. A unit that keeps killing its worker is set aside instead of crashing the next one. No message broker was needed.

Each unit saves a checkpoint after every resource it writes, along with its report so far. A retry continues where the last attempt stopped, and the skips and failures from the earlier attempt aren’t lost. Pause uses the same checkpoint. An audit found that pause used to mark the unit as done and drop its checkpoint, which is exactly the kind of bug that never shows up on a small test store.

Rate limits are handled when Shopify pushes back, not budgeted in advance. A throttled response is retried with exponential backoff and full jitter, so workers don’t all retry at the same instant, and Shopify’s own retry hint wins when it sends one:

```ts
export function computeBackoffMs(attempts: number, random: () => number, serverHintMs?: number): number {
  if (typeof serverHintMs === "number" && serverHintMs > 0) {
    return Math.min(serverHintMs, MAX_DELAY_MS);
  }
  const ceiling = Math.min(BASE_DELAY_MS * 2 ** Math.max(0, attempts - 1), MAX_DELAY_MS);
  // Full jitter: retries from different workers land at different instants.
  return Math.max(1, Math.round(ceiling * random()));
}
```

To be straightforward about it: a test run of 5,498 resources was never throttled, so this path has been tested but not yet exercised by real throttling. What did bite was classification. Shopify reports throttling as a GraphQL error code inside an HTTP 200, and for a while we treated that as a permanent failure. Two lanes per worker and a cap of four units in flight per store keep the pressure down in the meantime.

## Three outcomes, not two

A copy isn’t “succeeded” or “failed”. Every resource ends up in one of three lists, and the run report shows all of them in plain language: not copied (it should have been, and wasn’t), skipped and why (nothing written, on purpose), and copied with a caveat (written, with something the merchant should know). One bad resource is recorded and the batch carries on. That rule exists because Shopify’s built-in `frontpage` collection once failed on the first real copy and took every other collection in the batch with it.

## What we’d tell anyone building this

- **Make re-running the default, not a feature.** If the second run isn’t safe, nobody will trust the first.
- **Never write a value you didn’t positively translate.** A plausible-looking ID is the most dangerous kind of wrong.
- **Report matches and skips before the run, not only after.** The preview is where trust is won or lost.
- **A fix is only verified for the shapes your fixture had.** Several of our “works end to end” claims held only for small stores until a larger test store proved otherwise.
- **Measure before you warn.** We once published a caveat that media duplicates on re-run. It was inferred, not tested, and it was wrong.

Some things can’t be copied by any app: inventory history, customer passwords, gift card codes, original order dates. HyperSync lists those up front. Everything else is designed to be copied, checked, and copied again. The step-by-step version for merchants is on [the HyperSync clone guide](https://hypersync.hypermetron.com/clone-shopify-store).

Working on something with the same kind of problem? Start a project: https://hypermetron.com/#contact