Describe the bug

A Worker exposes HTTP endpoint that creates a sandbox per request. When several requests hit it concurrently, some of the creations fail and return HTTP 500 with:

RPC session was shut down by disposing the main stub

This happens with the account at very few running containers (10 instances), well below the instance cap. After the batch, the Containers dashboard shows container instances still in the Running state that do not correspond to any successfully created sandbox. They stay Running on their own and were only removed by an explicit destroy call using their id.

To Reproduce

1. Setup

Start with the account at 0 running container instances. Deploy the Worker below.

2. Worker code

One HTTP RPC endpoint. The create_workspace case mints a fresh id, gets a sandbox, and runs the create:

// apps/sandbox-proxy/src/rpc/handler.ts
case 'create_workspace': {
  const workspaceId = crypto.randomUUID();
  const sandbox = getSandbox(env.Sandbox, workspaceId);
  try {
    const result = await workspaceOps.createWorkspace(env.BACKUP_BUCKET, sandbox, workspaceId, request.params);
    return { workspaceId, status: 'ready', ...result };
  } catch (error) {
    const cleanupSandbox = getSandbox(env.Sandbox, workspaceId);
    await workspaceOps
      .destroyWorkspace(env.BACKUP_BUCKET, cleanupSandbox, workspaceId)
      .catch((cleanupError) =>
        console.error(`Failed to clean up after a failed create_workspace (${workspaceId})`, cleanupError),
      );
    throw error;
  }
}

createWorkspace then makes the first container-touching SDK calls — mkdir (which starts the container), then restoreBackup. The failure surfaces on one of these:

// apps/sandbox-proxy/src/workspace.ts
export const createWorkspace = async (bucket, sandbox, workspaceId, options) => {
  await sandbox.mkdir(WORKSPACE_DIR, { recursive: true });
  const snapshot = await resolveSnapshotById(bucket, options.snapshotId);
  await sandbox.restoreBackup(snapshot);
  // ...
};

3. Test code

Sign the control-plane JWT once, then fire 10 POSTs of { method: 'create_workspace', params: { snapshotId } } at the same time. Building all fetches before the first await keeps them concurrent:

const requests = Array.from({ length: 10 }, () =>
  fetch(endpoint, {
    method: 'POST',
    headers: { 'content-type': 'application/json', authorization: `Bearer ${token}` },
    body: JSON.stringify({ method: 'create_workspace', params: { snapshotId } }),
  }).then(async (res) => ({ status: res.status, body: await res.text() })),
);
const results = await Promise.all(requests);

4. Logs

One run (each line is one of the 10 concurrent requests; start/end are ms from batch start):

[00] FAIL http=500 start=+0ms   end=+7544ms   RPC session was shut down by disposing the main stub
[01] OK   http=200 start=+13ms  end=+4005ms   workspaceId=4119ca0c-... status=ready
[02] FAIL http=500 start=+14ms  end=+5270ms   RPC session was shut down by disposing the main stub
[03] OK   http=200 start=+14ms  end=+5684ms   workspaceId=216890c6-... status=ready
[04] OK   http=200 start=+14ms  end=+4912ms   workspaceId=1014ec88-... status=ready
[05] OK   http=200 start=+14ms  end=+5614ms   workspaceId=84233253-... status=ready
[06] OK   http=200 start=+14ms  end=+4882ms   workspaceId=ffc8fe21-... status=ready
[07] OK   http=200 start=+14ms  end=+7545ms   workspaceId=19c6cab9-... status=ready
[08] FAIL http=500 start=+15ms  end=+10975ms  RPC session was shut down by disposing the main stub
[09] OK   http=200 start=+15ms  end=+5267ms   workspaceId=b97eaeef-... status=ready
---- summary: 7/10 ok · 3/10 failed ----

Failed responses return quickly (~2–5s); successful ones take longer (~5–10s).

Counts across the batches run so far (only a few batches were run):

Running the 10 creations sequentially (one at a time) produces no failures.

5. Dashboard

After the batch, one or two instances remain in the Running state with no matching successfully created sandbox.

Expected behavior

A concurrent batch of 10 creations, with the account well below the instance cap, should create all 10. A creation that cannot complete should not leave a container instance in the Running state.

Environment

• @cloudflare/sandbox 0.12.2 (also observed on 0.11.0)
• container image cloudflare/sandbox:0.12.2
• instance type basic