Shared Volume recovery validation

Acceptance boundary

This work is not a claim that the full live-writer lifecycle passes. It combines reproduced recovery fixes with a guarded two-host acceptance runner. Transparent live-RW checkpointing remains unproven on the tested virtio-fs stack; a rejected checkpoint and successful cancellation are different outcomes from a successful checkpoint. The pinned patched client binary is installed on both Development hosts, but their mounted clients still run the older build; no real-client E2E flush proof is claimed. It is therefore not part of the tested runtime evidence below.

Real Development rig observations

The local Development control plane used its existing isolated PostgreSQL database, two Linux hosts, real JuiceFS metadata service, and configured object storage. No bucket-wide cleanup or direct database fence clearing was used.

The two hosts ran Cloud Hypervisor 52.0.0, virtiofsd 1.13.2, and JuiceFS 1.4.1+2026-07-30.0b90c7d. Candidate installation was checked against the executable of the running agent process on each host, not just the file on disk.

Current cleanup state: the two historical Jang attachment records are absent after the cross-host recovery bridge, and 40 original delete operations have succeeded. Three task volumes and five explicitly approved old volumes remain write-locked by the unresolved global recovery; they are not claimed deleted.

For the failed-create and four-detach results, both running host agents matched source 764be15d22fa386dd79862d20a0d191071e12d21, executable SHA-256 3090a69c99573a949b82498165726451b9891f1b7c60477643ae927a6f184736. The automatic legacy-detach replay additionally used control-plane source bfe6a41acb0d6f6b6a1106a6d0d7fd8464adb9c9. Later code changes require fresh candidate validation; these observations do not implicitly attest every later commit.

The final two quota-quiesce observations used source 24079bc198642953fbed1e847b1bef727a8d3b8d on the control plane and both running host agents; both executable hashes were 42d5c52ad9ef372fee38f8dd923f6929807420b6d35bdc5581fa829b5636a157. The current TypeScript SDK, Python SDK, and CLI also read the isolated org's three retained Shared Volumes successfully through the real local API. Those volumes remain write-locked by the unresolved global recovery, not deleted. The retained fixture vol_01M1SGRRHKD75C5TTNJBN9WXT8 produced aggregate SHA-256 38f608137fa38b88e6a9e8f49bf57eb1cafab83e9cd7dd7ba80575b0621b1ae1 on both hosts (sorted relative filenames and file SHA-256 records). Its two additional files are the original checksum manifests, not extra data records.

Deleted volume data is not recoverable through the volume API. Durable operation receipts and sandbox-deletion authority records are retained deliberately for delayed-command safety; they are not live attachments. Object-store trash/GC retention is independent of logical volume deletion.

Automated evidence and source provenance

The SHAs below identify historical tested snapshots, not the current PR head. History consolidation preserved the implementation tree at the recut boundary; it did not make 6a2b8008 an ancestor of the recut branch. Subsequent changes need their own validation, and neither content equivalence nor CI proves the remaining live acceptance matrix.

• Admission-journal startup correction 0b720448: isolated Linux go test -race -count=1 -p 1 ./cmd/host-agent -run ClosureAdmission passed. The regressions cover excess uncommitted temporary files, preserved active owner fencing after reload, exact owner release and subsequent acquisition. Committed-looking corrupt records remain strict; fence loading no longer prevents rescue modes or terminates the agent before registration.

• Terminal legacy cleanup bridge 771ee854: integrated API TypeScript check and 22 tests / 154 assertions across the owner-bridge and checkpoint cancellation suites passed against an explicitly named disposable database. Controls cover the actual fingerprint-absent legacy shape, conflicting identities, exact original-owner retries, authenticated absence, completed replay after lease release, and unchanged host capacity accounting.

• The exact published 0b720448 passed all required GitHub/Vercel checks. This result does not attest later unpushed changes or real-host installation.

• Historical source 6a2b8008896375c3d0820eaeae9d6d9395f92444 independently passed the assembled database validation: 172 tests and 896 assertions across 22 suites, with zero failures or skips. Separate guarded checks passed for the collector (1 test, 58 assertions), authenticated host route (3 tests, 21 assertions), generic routes (6 tests, 35 assertions), and HTTP wrapper and migration checks (12 tests, 24 assertions). Full API TypeScript typecheck passed. This is database-backed local behavior, not mounted-client flush, crash/outage durability, or two-host lifecycle acceptance; the pinned patched client was not installed for this evidence.

• Historical exact-source Linux host-agent race run for 6a2b800 passed all 11 packages (go test -race -count=1 -p 1 ./...); the host-agent package completed in 64.973 seconds. This remains host-agent test evidence, not proof of the full real-host lifecycle or live-client durability matrix.

Earlier source-specific validation

• Final selected Shared Volume/JuiceFS API run: 439 tests across 53 files passed in separate Bun processes on an isolated database. API typecheck passed.
• Final combined Linux host-agent go test -race -count=1 ./... passed (host package 104.700 seconds; all internal packages passed). Vet and Linux build passed as well.
• The local acceptance runner's seven contract tests passed, including declared file-content checksums and rejection of empty attachment inventories.
• Repeated component suites passed: TypeScript SDK 132, Python SDK 132, CLI 125, MCP 209. Two live SDK tests were skipped, not counted as live acceptance.
• The eight additional template-pin and deleted-sandbox-capacity integration tests from updated main (8a738a21) passed on the combined branch.
• Environment isolation checks passed; the complete integration commit range passed the credential scanner. Local public health returned 200, the manifest reported the exact running source, and unauthenticated volume access returned 401.

These counts are intermediate checkpoints, not a substitute for running the final combined candidate or the full real-host lifecycle and fault matrix.

The temporary no-volume probe sandbox was deleted through the API and its host recorded durable deletion completion. No test sandbox remains running. Three test Shared Volumes are retained for diagnosis because the global recovery fence still prevents their safe lifecycle cleanup. The snapshot feature flag remains at its original disabled setting.

Live checkpoint durability limitation

Guest FIFREEZE is unsupported on the tested stack. The recovery fix preserves that failure instead of silently snapshotting an active writer or replacing writer processes. Experiments with freezing the host export daemon are isolated from the shipping path.

A two-client private-container experiment with the exact JuiceFS version found that generic syncfs and a newly opened read-only file's fsync could return success while another client still observed old bytes. Reopening a retained writable inode and calling fsync made the new bytes visible in that bounded experiment, but this is not a complete drain proof: interrupted/failed closes can leave background chunk commits after the originating file descriptor is gone. A successful transparent checkpoint needs an audited all-writer fence and a reliable JuiceFS userspace-buffer drain, including those orphan commits.

The full create → concurrent RW → live checkpoint → stop/start → detach → sandbox delete → volume delete sequence, with every restart/outage boundary, must remain marked incomplete until it passes without weakening writer survival, checksum, or physical-cleanup assertions.