Preview Before Commit¶

Verified — minor divergences from the code — 1 finding(s) · 19d408be · 2026-08-25

The invariant's guarantee, enforcement mechanism, and single-use semantics are accurately described and verified against backend/persistence/mutation_store.py (write_preview, consume_preview), backend/domain/mutation_engine.py (build_preview, derive_entity_uuid, _canonicalize), backend/api/mutations_router.py (commit endpoint PREVIEW_REQUIRED gate ordering), and frontend/app.js:9042 (I-2 Confirm-button disable logic matches exactly). One minor factual inaccuracy found regarding where cleanup_stale_previews is invoked at startup.

Divergences from the code — details

Sev	Where	Doc says	Code does	Evidence
LOW	Preview TTL / cleanup_stale_previews	"cleanup_stale_previews is called at server startup inside ensure_dirs()." (Preview TTL section)	cleanup_stale_previews() is NOT called from inside ensure_dirs(); it is a separate, sequential call made right after ensure_dirs() returns, in backend/server.py. The function's own docstring in mutation_store.py even says "Called at startup after ensure_dirs()", contradicting the doc's "inside" phrasing.	`backend/server.py:859-860`

The Preview Before Commit invariant guarantees that no mutation reaches the overlay filesystem without a validated, server-processed preview step having been completed first. Every command that would write a new node or relation to disk must pass through /preview before /commit accepts it.

The Guarantee¶

Two properties are enforced together:

No direct commits. A caller cannot POST to a commit endpoint with a raw command and expect it to succeed. The commit endpoint requires a previewId. If the previewId is absent or unknown, the request is rejected with HTTP 400 before any disk write occurs.

Single-use previewId. A previewId is consumed at the moment of commit. Once consumed, the same previewId cannot be submitted again. Replaying a commit request with a previously used previewId returns HTTP 400. There is no path through the code that allows the same preview to drive two separate writes.

These two properties together mean the overlay directories (mutations/nodes/, mutations/relations/) can only be modified by a commit that had a live, unconsumed preview token at the moment it ran.

Where It Is Enforced¶

`write_preview`: storing the preview token¶

When the /preview endpoint completes successfully, the server calls:

def write_preview(self, preview_id: str, cmd: dict) -> None:
    """Persist a pending preview command so any worker can commit it."""
    self.write_atomic({"cmd": cmd}, self.mutations_dir / "pending" / f"{preview_id}.json")

Param	Type	Description
`preview_id`	`str`	Opaque identifier for this preview. Generated by the server at preview time.
`cmd`	`dict`	The validated command dict that was previewed (e.g., the `AddNode` command).

The preview file is written to mutations/pending/{preview_id}.json using write_atomic, which means temp file creation, fsync, then os.replace. The file is safe against partial writes — the directory will never contain a half-written preview file (CCR Rule 5).

The file stores exactly {"cmd": <command dict>}. The cmd dict is the original request payload that was already validated at preview time. At commit time the server re-uses this stored cmd rather than trusting the client to re-send the payload, which means the payload that was previewed is exactly the payload that is committed — a client cannot silently alter the command between preview and commit.

The pending/ directory is shared across all uvicorn worker processes because it lives on the filesystem, not in process memory. This is the explicit motivation noted in the source: with multiple workers, the preview could be stored by worker A and the commit routed to worker B. Writing to mutations/pending/ makes the store shared for all workers.

`consume_preview`: the single-use gate¶

def consume_preview(self, preview_id: str) -> dict | None:
    """Read and atomically delete a pending preview.
    Returns the stored cmd dict, or None if not found (already consumed or unknown).
    """
    path = self.mutations_dir / "pending" / f"{preview_id}.json"
    with self._write_lock:
        if not path.exists():
            return None
        try:
            data = json.loads(path.read_text(encoding="utf-8"))
        except Exception:
            return None
        try:
            path.unlink()
        except OSError:
            pass
    return data.get("cmd")

Param	Type	Description
`preview_id`	`str`	The preview token submitted with the commit request.

Return type: dict | None. Returns the cmd dict if the preview existed, or None if the preview file was not present (not found, already consumed, or the file was corrupted).

What it reads and mutates: Reads mutations/pending/{preview_id}.json and, on success, deletes it. Both operations happen inside self._write_lock (a threading.Lock).

Atomicity — best-effort, not transactional. The read and delete are serialised by the in-process threading.Lock, which prevents two coroutines in the same worker process from racing. However, the read and unlink are two separate syscalls. They are not wrapped in a single atomic filesystem operation. This means:

Within a single process, the lock guarantees only one goroutine reads-then-deletes at a time. The sequence is: check existence, read, unlink. A second caller inside the same process will block at with self._write_lock and will then see path.exists() == False after the first caller returns, so it returns None.
Across worker processes, there is no cross-process lock on the pending/ file specifically. (The write-lock flock covers MUTATIONS_DIR/_write.lock for the mutation commit phase, but consume_preview acquires only the threading lock.) In a worst-case concurrent scenario with two worker processes both receiving a commit for the same preview_id at the same time, both could pass the path.exists() check before either calls unlink. This is the race condition discussed in the Failure Modes section below.

Error handling: If the file exists but is unreadable or corrupt, consume_preview returns None (treating corruption as "already consumed"). If unlink fails with OSError, the error is suppressed. The file may remain on disk after a failed unlink, meaning that preview token could be consumed again by a subsequent commit. In practice this is rare but represents a known best-effort bound on the guarantee.

The commit endpoint: calling consume_preview before any disk write¶

The commit endpoint calls consume_preview at the start of its handler, before any call to write_atomic, build_overlay_node, or cache invalidation. The code flow is:

Receive POST request with previewId.
Call consume_preview(preview_id) — this is the gate.
If the return value is None, return HTTP 400 with error code PREVIEW_REQUIRED.
If the return value is a cmd dict, continue with building the overlay document and committing it to disk.

No overlay write happens before step 2 completes with a non-None result. The commit path uses the cmd dict returned from consume_preview as the authoritative payload — not any payload fields the client sent in the commit request body.

The Single-Use Property¶

What happens on a duplicate commit¶

If the same previewId is submitted twice to the commit endpoint:

First commit: consume_preview finds the file, reads it, deletes it, returns the cmd dict. Commit proceeds.
Second commit: consume_preview calls path.exists() — the file is gone. Returns None. The commit endpoint returns HTTP 400.

The error response for a missing or consumed previewId carries the error code PREVIEW_REQUIRED. The response body indicates that the preview was not found, without distinguishing between "never existed" and "already consumed". From the API caller's perspective these are equivalent: in both cases the client must call /preview again to get a fresh previewId before retrying the commit.

The UI enforces this at the frontend layer (invariant I-2): the Confirm button is disabled when previewId is falsy or when the preview response carried errors. This means that under normal operation a user cannot submit a commit without a live previewId. The server-side gate exists as defence-in-depth against stale clients, direct API calls, and scripted integrations.

Why Preview Exists¶

User-facing diff before confirmation¶

The preview step returns a diff to the UI: what the new node will look like, where it will appear in the tree, its display path, and the scope of the subtree that will be affected. The user sees this information before clicking Confirm. The preview step is the mechanism that makes a "show me what will change, then confirm" UX possible.

Stable UUID derivation at preview time¶

The entity UUID is derived deterministically at preview time inside build_preview:

def build_preview(cmd: dict, eff: dict) -> tuple[dict, dict]:
    ...
    node_uuid = derive_entity_uuid(cmd, parent_context=parent_id)
    ...
    node_diff = {
        "id": node_uuid,
        ...
    }
    return node_diff, scope

derive_entity_uuid produces a UUID v5 from the canonicalized command:

def derive_entity_uuid(cmd: dict, parent_context: str | None = None) -> str:
    return str(uuid.uuid5(NAMESPACE_EIDOS, _canonicalize(cmd, parent_context)))

_canonicalize builds a deterministic JSON string from commandType, filtered payload fields (excluding transient fields timestamp, actor, idempotencyKey), parentContext, and idempotencyKey. Because the same input always produces the same UUID v5, the UUID shown in the preview diff is identical to the UUID that will be written to disk at commit time. This is CCR Rule 16: preview UUID equals commit UUID.

The consequence is that the UI can display the exact future id of the node before the user confirms. It also means that idempotency is fully deterministic: replaying the same command (same idempotencyKey, same payload, same parent) always targets the same UUID, so a crashed-and-retried commit does not create a duplicate node.

Preview TTL¶

`cleanup_stale_previews`¶

def cleanup_stale_previews(self, ttl_seconds: int = 86400) -> int:
    """Delete pending/*.json files older than ttl_seconds. Returns count deleted."""
    import time as _time
    pending_dir = self.mutations_dir / "pending"
    if not pending_dir.exists():
        return 0
    cutoff = _time.time() - ttl_seconds
    deleted = 0
    for f in pending_dir.glob("*.json"):
        try:
            if f.stat().st_mtime < cutoff:
                f.unlink()
                deleted += 1
        except Exception as exc:
            _log.warning("cleanup_stale_previews_error", file=f.name, exc=str(exc))
    if deleted:
        _log.info("cleanup_stale_previews_done", deleted=deleted)
    return deleted

Param	Type	Description
`ttl_seconds`	`int`	Maximum age of a pending preview file before it is deleted. Default: 86400 (24 hours).

Return type: int — count of files deleted.

What it mutates: Calls f.unlink() on each pending/*.json file whose st_mtime is older than cutoff. Files newer than the cutoff are not touched.

When it runs: cleanup_stale_previews is called at server startup inside ensure_dirs(). It runs once per worker startup, not on a recurring timer. There is no background thread or scheduled task — cleanup happens opportunistically when the server restarts.

The TTL value of 86400 seconds (24 hours): This is long enough that a user who starts a preview dialog and walks away for several hours can still return and commit. It is short enough that the pending/ directory does not accumulate unbounded files if users repeatedly open and abandon dialogs without committing. After 24 hours without a commit, the preview token is silently gone and the user must start a new preview.

The 24-hour window is intentional: it covers realistic human use (overnight, different time zones, long review cycles) without requiring the server to maintain preview state indefinitely. There is no mechanism to extend a preview TTL in place; the only path forward is a fresh /preview call.

Failure Modes¶

Preview succeeds, commit fails (partial state)¶

If /preview completes and the file is written to pending/, but the commit subsequently fails — for example, because the write_atomic call for the overlay node raises an exception — the preview file remains in pending/. No node has been written to mutations/nodes/. The overlay is clean; no partial state is visible to readers.

The previewId is still live in pending/ unless consume_preview was called before the failure. If consume_preview was called first (which it is, at the start of the commit handler), the preview file has already been deleted. The commit failure then leaves the previewId unusable and the node unwritten. The caller must call /preview again to retry.

If the failure happens before consume_preview is called — which is not possible in the current code path since consume_preview is the first operation in the commit handler — the preview file would still be present and reusable. In any case, no disk state is corrupted because write_atomic guarantees that either the full file is written and renamed, or nothing is changed.

Network timeout between preview and commit¶

If the client calls /preview, receives a previewId, and then loses network connectivity before calling /commit, the preview file sits in mutations/pending/ until the TTL expires (86400 seconds from when it was written). After expiry, cleanup_stale_previews removes it on the next server startup. The client, upon reconnecting, must call /preview again — there is no way to recover the original previewId.

This is consistent with the invariant: a previewId is a short-lived intent token. The server does not guarantee that it will be held indefinitely. The client is responsible for completing the preview-to-commit sequence within the TTL window.

Two concurrent commits with the same previewId (race condition)¶

This is the most important failure mode to understand. consume_preview acquires self._write_lock (a threading.Lock) which is per-process. If two HTTP requests arrive for the same previewId on two different uvicorn worker processes simultaneously, both processes have their own _write_lock instance and neither blocks on the other.

The sequence that can produce a double-commit:

Worker A receives commit for previewId=X. Acquires its own _write_lock. Calls path.exists() — returns True. Reads the file.
Worker B receives commit for previewId=X. Acquires its own _write_lock (different object, no contention). Calls path.exists() — the file is still present because Worker A has not yet called unlink(). Returns True. Reads the file.
Worker A calls path.unlink(). Releases lock.
Worker B calls path.unlink(). Gets OSError (file already gone). Suppresses it. Releases lock.
Both Worker A and Worker B return the same cmd dict and proceed to commit.

Both commits will write to mutations/nodes/{same_uuid}.jsonld with an os.replace call. Because os.replace is atomic at the POSIX level and both workers are writing the same deterministic content (same UUID, same payload), the second replace simply overwrites the first with an identical file. The end state is correct — one node file exists with the right content — but the double-write did occur and both audit log entries will be written.

The deterministic UUID (CCR Rule 16) is the property that makes this race safe in terms of data correctness: both workers compute the same node_uuid from the same cmd, so even if the race fires, the filesystem ends up with one file containing the correct node document. The audit log will contain two entries for the same logical mutation, which is a known imprecision but not a data loss event.

To eliminate this race entirely would require a cross-process lock on the pending/ file read-unlink pair, analogous to the _write.lock flock used for the broader mutation commit. This is the current architectural bound: the single-use property is guaranteed within a single worker process and is best-effort across multiple workers.

Summary of Invariant Invariants¶

Property	Enforced by	Strength
No commit without prior preview	`consume_preview` returns `None` → HTTP 400	Hard within a process; best-effort across workers
PreviewId is single-use	`path.unlink()` inside `_write_lock` before returning `cmd`	Hard within a process; best-effort across workers
Preview payload equals commit payload	`cmd` stored at preview time, re-used at commit time	Hard — client payload is not trusted at commit
Preview UUID equals commit UUID	`derive_entity_uuid` called with same inputs in both `build_preview` and the commit handler	Hard — deterministic UUID v5
Stale previews are eventually removed	`cleanup_stale_previews` at startup with 86400s TTL	Best-effort — only runs at server restart