Sandbox lifecycle
The states a sandbox moves through, and what the control plane and host do at each step.
A sandbox moves between states as the control plane queues commands and host agents report results. Stop keeps the disk and, when it can, the memory. Delete removes both.
Read the states
These are every state a sandbox row is written with (apps/api/src/sandboxState.ts).
| State | Meaning |
|---|---|
provisioning | The row exists and a create command is queued. The host has not reported yet. |
starting | A start of a stopped sandbox is in flight. |
running | The in-VM agent answered on vsock. |
stopping | A stop or delete is in flight. The guest runs until the host confirms. |
stopped | No VM process. The disk, and usually the memory, are saved. |
resuming | A clone, fork or snapshot restore is booting. |
restoring | The sandbox is being rebuilt from a backup. |
migrating | The sandbox is moving to another host. |
archiving | A stopped sandbox is uploading to object storage. |
archived | The sandbox is stored only in object storage. |
unarchiving | An archived sandbox is coming back to a host before it starts. |
failed-start | A boot or restore failed. A sandbox that once ran can be started again. |
deleted | A tombstone. The row stays until it is purged. |
There is no paused state. POST /v1/sandboxes/:id/pause is an alias for stop, and the sandbox settles as stopped.
The public API enum also lists pending, failed-provision and lost. The control plane never writes them. They stay for SDK compatibility.
Create a sandbox
- Request.
POST /v1/sandboxesvalidates the body, the template and the organization's quotas. - Placement. One transaction picks a
readyhost with a recent heartbeat and enough headroom, reserves capacity, and inserts the sandbox row and itssandbox.createcommand. - Wake. The control plane releases the host's open poll, so the host picks up the command at once.
- Boot. The host reflink-clones the template disk, creates the VM's TAP and launches
cloud-hypervisor. - Ready. The host pings the in-VM agent on vsock port
18047and reportsrunningwith the sandbox's internal IP.
If the host never reports, the reconciler fails the command and the sandbox becomes failed-start. Its capacity is released.
Stop a sandbox
POST /v1/sandboxes/:id/stop queues a sandbox.stop command. The host pauses the VM, writes a memory snapshot into the sandbox's stop-snap/ directory, stops the process and keeps the disk.
When a memory snapshot is not possible, the host does a disk-only stop. The next start cold-boots the saved disk, and running processes are lost.
Auto-stop stops a running sandbox after auto_stop_minutes with no activity.
Start a sandbox
POST /v1/sandboxes/:id/start (or /resume) restores the stop snapshot. Guest memory is loaded on demand as the guest touches it, and processes continue where they stopped.
If the original host is gone or full, the sandbox can be placed on another host, and its snapshot is fetched from object storage. An archived sandbox is unarchived first.
A preview URL request resumes a stopped sandbox unless it was created with auto_resume: false.
Delete a sandbox
DELETE /v1/sandboxes/:id sets deleted_at and queues sandbox.delete. The host kills the VM and removes its TAP, firewall rules and files.
Snapshot, clone and fork
| Call | Result |
|---|---|
POST /v1/sandboxes/:id/snapshot | A disk snapshot on the same host. The VM pauses briefly. |
POST /v1/sandboxes/:id/restore | The sandbox boots from one of its snapshots. |
POST /v1/sandboxes/:id/clone | A new sandbox booted from a snapshot. |
POST /v1/sandboxes/:id/fork | Snapshot plus clone in one call. |
What each snapshot holds is in Storage.
Archive, back up and recover from host loss
These need object storage.
- Archive.
POST /v1/sandboxes/:id/archiveuploads a stopped sandbox and frees its local disk.auto_archive_daysdoes this automatically. - Backup. A running sandbox is backed up every
backup_interval_minminutes.POST /v1/sandboxes/:id/backuptakes one now. - Restore from backup.
POST /v1/sandboxes/:id/restore-from-backupcold-boots the backed-up disk on a new placement. Memory is not restored. - Host loss. A host silent for 5 minutes becomes
lost. After a further 10 minutes, each running sandbox on it that has a backup is restored on another host. The rest becomefailed-start.
Let the reconciler converge
The leader runs the reconciler every 10 seconds (apps/api/src/reconciler.ts). Each pass is idempotent. It:
- marks silent hosts
lost, - fails commands whose host never answered,
- marks a
runningsandboxfailed-startwhen its guest stops answering, - fails over sandboxes on lost hosts,
- applies auto-stop, auto-archive and auto-delete.
When every in-service host goes silent at the same moment, the reconciler does nothing. That pattern points at the control plane's own database or network, not at the hosts.
Know the guarantees
- Placement, the sandbox row and its command commit together or not at all.
- A late completion from a host cannot bring a deleted sandbox back.
- A memory restore keeps processes and open files. A disk-only stop or a backup restore does not.
See also: Storage · Architecture overview