Deterministic UUID

Verified against the code · 4996d1ceb770fb4545edebc5fd140792bfb23830 · 2026-10-20

956 D1b-4 (2026-10-20): 5 finding(s) fixed — from_iri raises InvalidIRIError on a bare uuid and only GET /resolve falls back to the raw id; since WP4b the relation target is the single on-disk eidos:target field (ExternalRef/UnresolvedRef node or node IRI); derive_overlay_relation_uuid hashes only {commandType, idempotencyKey} and duplicates are caught by _find_existing_triple; payload_hash uses json.dumps default separators (not compact); build_preview calls derive_entity_uuid at mutation_engine.py:272. #1016 (2026-10-20, earlier): the HTTP AddNode preview/commit UUID finding was closed in code (preview and graph_crud_engine.create_node both build cmd with mutation_engine.normalize_add_node_cmd). Re-verified against main 4996d1ce. #989 WP9 S3 (2026-10-22): the frontend sends the IRI on every parsed public node-id input it calls (not /me/can-edit, which it does not call). Checked against frontend/app.js and detail-panel.js on the #989 branch (base 3c69a14f).

Every entity written to the EIDOS overlay — every node, every relation — carries a UUID that is computed, not generated at random. Given the same mutation command, the same UUID is always produced. This is not a convenience feature; it is a load-bearing invariant that makes the entire overlay safe to replay, import multiple times, and retry without accumulating duplicate records.

The Guarantee

For any given AddNode or CreateRelation mutation command, the UUID derived from that command is always the same. Re-submitting the identical command tomorrow, next week, or after a server crash produces the exact same UUID as the first submission. This property is called deterministic identity derivation and it is encoded in CCR Rules 10 and 16.

The practical consequence is that mutations are idempotent at the identity level: the second submission of a command will attempt to write a node file whose name is identical to the one already on disk. Combined with the idempotency record system (CCR Rule 13), the second write is detected and harmlessly rejected before any file is touched.

The Excel importer depends on this guarantee directly. When an Excel file is re-imported after a partial failure, the importer re-derives commands from the file's content hash (CCR 16). Because the UUIDs are deterministic, previously committed nodes receive the same ID they already have on disk. Only genuinely new rows produce new UUIDs, and the commit for those rows can proceed without risk of creating ghost duplicates of what was already written.

Where It Is Enforced

The invariant is enforced in two functions inside backend/infra/key_derivation.py — the canonical identity owner since G2 (#929). backend/domain/mutation_engine.py re-exports both symbols so legacy imports (from domain.mutation_engine import derive_entity_uuid) keep working.

_canonicalize

def _canonicalize(cmd: dict, parent_context: str | None = None) -> str:

Param	Type	Description
cmd	dict	The raw mutation command dictionary
parent_context	str \| None	UUID of the parent node; required for child nodes to prevent cross-subtree collisions

Returns: a compact JSON string with no whitespace, keys sorted alphabetically.

_canonicalize produces the name string that will be fed into UUID v5. It does three things:

1. Strips transient fields. The set _TRANSIENT_FIELDS = {"timestamp", "actor", "idempotencyKey"} is removed from the payload before it is serialised. These fields change between submissions (timestamps differ, actor may differ on retry) but they do not define the identity of the entity. Including them would make the UUID non-deterministic across retries.

2. Builds a fixed-structure envelope. The output JSON always contains exactly four top-level keys — commandType, payload, parentContext, idempotencyKey — in that sort order. The payload value is the filtered command dict (transient fields removed). parentContext is the parent_context argument, or an empty string if None. idempotencyKey is taken from the original command.

3. Serialises with sort_keys=True and compact separators. The separators (",", ":") eliminate all optional whitespace so the string is byte-for-byte identical regardless of how the original dict was constructed or which Python version is running.

The result is a deterministic, human-readable string that uniquely identifies the intent of the mutation command.

derive_entity_uuid

def derive_entity_uuid(cmd: dict, parent_context: str | None = None) -> str:

Param	Type	Description
cmd	dict	The raw mutation command dictionary
parent_context	str \| None	UUID of the parent node; passed through to _canonicalize

Returns: a UUID v5 string in canonical hyphenated form, e.g. "3d6f4d2a-...".

Reads: NAMESPACE_EIDOS from infra.key_derivation.

Mutates: nothing. Pure function with no side effects.

Error conditions: none raised explicitly. If cmd is missing expected keys, dict.get returns empty strings rather than raising — the canonical string will still be valid but may produce an unintended UUID if required fields are absent. Schema validation in validate_add_node must run before this function is called in any commit path.

This function is the single point of UUID derivation for all overlay nodes and overlay relations. It is called in build_preview (mutation_engine.py:272) and must be called by the commit handler with exactly the same cmd and parent_context arguments. Because the canonical string is identical in both calls, the preview UUID is guaranteed to equal the commit UUID (CCR Rule 16: "preview UUID == commit UUID").

derive_relation_uuid_for_unresolved_target

def derive_relation_uuid_for_unresolved_target(
    source_id: str, ref_key: str, relation_type: str
) -> str:

Param	Type	Description
source_id	str	UUID of the source node
ref_key	str	The external or unresolved target path/ref (e.g. "+FEM.CTP01.TA15")
relation_type	str	Relation predicate

Returns: a UUID v5 string in canonical hyphenated form.

Reads: NAMESPACE_EIDOS from infra.key_derivation.

Mutates: nothing. Pure function with no side effects.

Added in G4 (#931), this function retires the former random UUID placeholder that resolve_relation_rows previously minted for the EXTERNAL target UUID (tgtUuid). It supplies a stable UUID v5 placeholder for the TARGET endpoint only — it is NOT the owning relation's UUID. The relation's own UUID is derived from the CreateRelation command's idempotency key via derive_overlay_relation_uuid (WP4 #953). The derivation payload uses the internal key "targetRef" (not "targetId"), which is kept distinct from the shape of any resolved-target derivation so the two namespaces never collide in NAMESPACE_EIDOS.

The two cases are handled differently (G4 round 2, PR #957):

Since WP4b (#965), build_overlay_relation (mutation_engine.py:737-783) writes the target as one on-disk field, eidos:target; there are no eidos:toNodeId / eidos:externalTargetRef keys on new files.

• EXTERNAL rows: this function supplies a stable UUID v5 for the cross-domain target UUID — the "best-effort target identity until resolved" value. On disk eidos:target is an eidos:ExternalRef node (domain + ref). The placeholder itself is not written; the reader re-derives it from (source, ref, relationType). An eidos:node IRI is embedded only when a real targetId is known and differs from the derived placeholder (or there is no ref to derive from). The relation file's own uid is derived from the CreateRelation idem key.
• UNRESOLVED rows: derive_relation_uuid_for_unresolved_target is NOT used. On disk eidos:target is an eidos:UnresolvedRef node carrying only the deferred ref, so the read-side target id is empty (consumers that test the target for truthiness continue to see "no target yet"); the relation's OWN uid is derived from the CreateRelation command's idem key via derive_overlay_relation_uuid. The command's unresolvedTargetRef is what the dup check keys on: graph_crud_engine.create_relation's triple-dup check keys UNRESOLVED rows on (source, unresolvedTargetRef, predicate) so two unresolved rows from the same source with different refs do not collide.

Because the UUID is derived from (source_id, ref_key, relation_type) for EXTERNAL targets, re-importing the same Excel file always produces the same EXTERNAL tgtUuid placeholders — safe overwrite on retry (CCR Rule 16).

On disk the ref lives inside the eidos:target ExternalRef node, not under a "targetRef" key. The "targetRef" label is used only within the derivation payload.

payload_hash

def payload_hash(cmd: dict) -> str:

Param	Type	Description
cmd	dict	The raw mutation command dictionary

Returns: a 64-character lowercase hex string (SHA-256 digest).

Reads: all fields in cmd except idempotencyKey.

payload_hash is a companion function, not the UUID derivation itself. It is used for idempotency key integrity checks (CCR Rule 13). When a client submits a mutation, it includes an idempotencyKey in the command. The server uses payload_hash to verify that the payload has not changed between the submission that created the idempotency record and the current retry. If the hash differs, the server rejects the request — the same idempotencyKey cannot be reused for a different payload.

The serialisation in payload_hash is simpler than _canonicalize: it sorts the remaining keys alphabetically and calls json.dumps(sort_keys=True) with the default separators (", " and ": "), so the JSON is not compact, then hashes the UTF-8 bytes with SHA-256. It does not impose the same four-key envelope that _canonicalize uses, so the two functions are not interchangeable.

UUID v5 vs UUID v4

Why UUID v4 is wrong here

UUID v4 is generated by drawing 122 bits from a cryptographically random source. No two calls return the same value. If the mutation engine used uuid.uuid4() for entity IDs, every invocation of derive_entity_uuid — even with identical inputs — would produce a different UUID. Re-importing the same Excel file would create duplicate nodes. Retrying a failed HTTP request would create a second copy of the same node. The overlay would accumulate garbage with no way to distinguish legitimate new nodes from accidental duplicates.

This failure mode is not hypothetical. The violated_by entry in deterministic-uuid.yaml documents exactly this: the retired write-engine used uuid.uuid4() and its IDs were irreconcilable with the mutation engine. Finding 11 in ARCHITECTURE_REVIEW.md describes the consequences.

How UUID v5 works

UUID v5 is defined in RFC 4122. It takes two inputs: a namespace UUID and a name string. It computes SHA-1(namespace_bytes + name_utf8_bytes), then formats the first 128 bits of that digest as a UUID with version bits 0101 and variant bits set. The result is fully deterministic: the same namespace and name always produce the same UUID.

The Python call in derive_entity_uuid is:

str(uuid.uuid5(NAMESPACE_EIDOS, _canonicalize(cmd, parent_context)))

NAMESPACE_EIDOS is the fixed constant f47ac10b-58cc-4372-a567-0e02b2c3d479, defined in backend/infra/key_derivation.py and imported at module load time. The module docstring in key_derivation.py states explicitly: "Namespaces are fixed forever once data has been written using them. Do not change a namespace value without a full data migration." Changing NAMESPACE_EIDOS would cause every node ever written to receive a different UUID on the next derivation — every existing file would become orphaned and every idempotency record would be invalidated.

NAMESPACE_EIDOS is reserved exclusively for overlay node and relation UUIDs. Other namespaces serve other purposes: NAMESPACE_SEEDED_RELATION is used by the .eidos archive reader convert uses (tools/migration/eidos_archive.py), NAMESPACE_BOOTSTRAP by the bootstrap and deploy scripts. No other module may define its own namespace UUID constants; key_derivation.py is the single source of truth.

The parent_context Parameter

Why child nodes need it

Consider two separate subtrees, each containing a node with the label "Pump-01" under a parent called "Hydraulics". If UUID derivation depended only on commandType, label, and parentNodeId, these two nodes would receive identical UUIDs — a collision that would cause the second node to overwrite the first on disk.

The parent_context parameter breaks this symmetry. For AddNode commands, the caller passes parent_context=parent_id (the UUID of the parent node). Because parent_id differs across subtrees, the canonical string differs, and the derived UUID differs.

In build_preview the call is:

node_uuid = derive_entity_uuid(cmd, parent_context=parent_id)

The commit handler must use the same call pattern. Using parent_context=None in one path and parent_context=parent_id in the other would break the "preview UUID == commit UUID" guarantee. The same holds for the cmd key set: for AddNode, the preview and graph_crud_engine.create_node both build cmd with mutation_engine.normalize_add_node_cmd, so the keys and defaults are identical on both sides (#1016).

What happens without it

If parent_context is omitted (None), _canonicalize substitutes an empty string for parentContext. Two nodes with the same label under different parents would collide. The collision would be silent: the second commit would overwrite the first node's file on disk with the new node's content, corrupting the graph without raising an exception. The only observable symptom would be that the first node's data disappears and its children become orphaned.

For CreateRelation commands, parent_context is None. Since WP4 (#953) derive_overlay_relation_uuid(idem_key) hashes only {commandType: "CreateRelation", idempotencyKey}; the endpoints are not inputs. Uniqueness of the (source, target, predicate) triple is enforced separately by graph_crud_engine._find_existing_triple under the write lock (see the WP4 section below).

Idempotency Guarantee This Enables

Same mutation, same UUID

When a client submits an AddNode command and the network drops before the response arrives, the client retries. The retry carries the same commandType, parentNodeId, label, and idempotencyKey fields. _canonicalize produces the same string. uuid.uuid5 produces the same UUID. The commit handler attempts to write a node file whose name is <uuid>.jsonld. That file already exists from the first (successful) write.

Without an idempotency record system, the second write would silently overwrite the first — harmless for identical content but dangerous if any mutable field (like eidos:created) differs between attempts. The idempotency record system (CCR Rule 13) detects the duplicate before the write: it looks up the idempotencyKey in the idempotency store, finds a record from the first submission, and returns the stored success response without touching the file system.

The deterministic UUID is what makes this system coherent: the client can always recompute the UUID of the entity it tried to create, and the server can always find the correct file on disk using that UUID even if the idempotency store entry has expired.

Double-submission rejection

If a user clicks "Save" twice in quick succession, two AddNode requests arrive at the server. The first acquires a lock on the idempotencyKey, writes the node file, writes the idempotency record, and releases the lock. The second request, upon acquiring the lock, finds the idempotency record and returns the first request's success response. The node file is written exactly once. The overlay contains exactly one copy of the node.

The Derived UUID Is the Entity's IRI

The uuid this invariant derives is the entity's identity. On the wire it is the IRI https://id.ontoteq.com/{uuid} (WP1 #950): infra.iri.node_iri, relation_iri and audit_iri wrap the uuid into one flat IRI space shared by nodes, relations and audit events, and rdf:type tells them apart. Since #954 WP5 S7 every node @id and uri in an API response is this IRI. infra.iri.from_iri parses it, the legacy urn:eidos:audit: and /node/{id} forms back to the same uuid, for stored data. It raises InvalidIRIError on a bare uuid, and its subclass LegacyFormRefusedError (naming tools.migration) on a legacy urn:eidos:node: / urn:eidos:relation: id (#1102 S8b).

The IRI is identity only. It is not dereferenceable: nothing serves id.ontoteq.com, and neither the app nor the backend fetches it. The location is a separate lookup: GET /resolve?id={iri|uuid} (backend/server.py, #954 S2) returns {iri, uuid, domain}, and the domain names the server that holds the entity.

Public node-id inputs (WP9 S1, #989)

A node id that a caller sends in a query or body field is parsed by one function, infra.iri.parse_public_node_id (parse_public_node_ids for a list). It is built on canonical_node_uuid and returns the bare uuid, which every store stays keyed by (overlay files, in-memory maps, audit, notes).

Input	Result
https://id.ontoteq.com/{uuid} (canonical uuid)	the uuid, no log
{uuid} (canonical, lowercase, hyphenated)	the uuid, plus one bare_uuid_deprecated warning with endpoint, caller and count per request — legacy input during the WP9 deprecation window
another host, urn:… (including the legacy urn:eidos:node:), uppercase, braces, 32-hex, a path	typed 422 V-INVALID_NODE_ID (api/node_id.py), nothing logged
a value shaped like no node id (no : or /, not uuid-shaped — e.g. a cross-domain placeholder)	passed through unchanged, and treated as an unknown id as before, on every route below except GET /resolve

The parsed inputs are GET /resolve id, GET /nodes/meta and GET /metrics/batch ids, GET /me/can-edit node_id, GET /nodes-referencing-external uuid, POST /notifications/node node_id, and node on GET/POST/PUT/DELETE /me/node-notes. An IRI therefore gives the same answer as its uuid, including the same 404 or omission for a node the caller cannot read. A note saved through the IRI is the note saved through the uuid. The notification deep link is node_deeplink(base, uuid) for either form. Path segments (/node/{id} and the like) are locators, not identity inputs, and keep the uuid. The mutation commands and heal-uuid are not parsed here yet. Since WP9 S3 the frontend sends the IRI on every one of these inputs except /me/can-edit, which it does not call, so the frontend's own requests log no bare_uuid_deprecated line.

The deprecation window is log-driven (owner decision on #989): it closes when DEV and prod have logged no bare_uuid_deprecated line for 14 days, and lasts 30 days at the least. WP9 S6, which answers a bare uuid on each of these inputs with the same typed 422, has not started; until it ships, a bare uuid keeps working here. Hash links already sent (#<uuid>, #{BASE}/node/<uuid>) stay readable after S6 too.

See the Canonical Deep Link invariant for how a share link differs from this identity IRI, and the Domain Routing Key invariant for how the resolved domain becomes a fetch base.

Testing

Core test pattern

A test for the deterministic UUID invariant has exactly this shape:

cmd = {
    "commandType": "AddNode",
    "parentNodeId": "some-parent-uuid",
    "label": "Pump-01",
    "idempotencyKey": "key-abc",
}
uuid_first  = derive_entity_uuid(cmd, parent_context="some-parent-uuid")
uuid_second = derive_entity_uuid(cmd, parent_context="some-parent-uuid")
assert uuid_first == uuid_second

A second test confirms that different inputs produce different UUIDs:

cmd_b = {**cmd, "label": "Pump-02"}
assert derive_entity_uuid(cmd, parent_context="some-parent-uuid") != \
       derive_entity_uuid(cmd_b, parent_context="some-parent-uuid")

The edge case: same label, different parents

The most important edge case is two nodes with the same label under different parent UUIDs. Without parent_context propagation, these would collide:

cmd = {
    "commandType": "AddNode",
    "label": "Pump-01",
    "idempotencyKey": "key-abc",
    "parentNodeId": "parent-A",
}
uuid_under_A = derive_entity_uuid(cmd, parent_context="parent-A")

cmd_b = {**cmd, "parentNodeId": "parent-B"}
uuid_under_B = derive_entity_uuid(cmd_b, parent_context="parent-B")

assert uuid_under_A != uuid_under_B  # must differ — different subtrees

This test verifies that parent_context is actually flowing into _canonicalize. If someone refactors derive_entity_uuid and forgets to pass parent_context through, this test will catch the regression before any data is written.

Testing transient field exclusion

A third category of test verifies that timestamp and actor do not affect the UUID:

cmd_with_timestamp = {**cmd, "timestamp": "2026-01-01T00:00:00Z", "actor": "alice"}
cmd_without = {k: v for k, v in cmd.items() if k not in ("timestamp", "actor")}
assert derive_entity_uuid(cmd_with_timestamp, parent_context="parent-A") == \
       derive_entity_uuid(cmd_without, parent_context="parent-A")

If this test fails, a retry that includes a fresh timestamp in the command would produce a different UUID than the original submission — breaking idempotency entirely.

Regression guard: namespace must not change

The namespace constant NAMESPACE_EIDOS = uuid.UUID("f47ac10b-58cc-4372-a567-0e02b2c3d479") should be pinned in a test:

from infra.key_derivation import NAMESPACE_EIDOS
import uuid
assert NAMESPACE_EIDOS == uuid.UUID("f47ac10b-58cc-4372-a567-0e02b2c3d479")

This test prevents a well-intentioned developer from changing the namespace constant without realising they are invalidating every UUID ever written to the overlay.

Slug Segment (WP2 #951)

Alongside the UUID, mutation_engine now stores a ref-path segment on every overlay node as skos:notation. The segment is derived by infra.key_derivation.derive_slug_segment(label, is_root=...) — the same stripping rule that _build_ref applies when assembling the dotted ref path. Root nodes keep their full label (including the sigil prefix such as +FFL); non-root nodes use _strip_prefix(label). The full ref path is assembled at read time from the chain of stored segments, falling back to deriving a segment from the label whenever a pre-WP2 node lacks the field — so existing ref paths are byte-stable with no file rewrite.

Full-path uniqueness within the domain is enforced inside graph_crud_engine.create_node, update_node, and move_node under the per-domain write lock. A collision returns MutationResult(ok=False, error_code="V-SLUG_COLLISION") and records an audit rejection — no write occurs, and there is no silent rename or suffix disambiguation.

Lookup from a full slug path to a node UUID routes through infra.key_derivation.resolve_slug(full_path, ref_index) — the public façade over resolve_ref used by /node-by-ref, the admin rulebook, the unresolved node metric, and the export service. In WP2 the two are equivalent; later slices may replace the resolver without touching call sites.

Path-Free Domain Import (WP3 #952)

Before WP3, derive_domain_import_uuid(domain, path) tied node identity to the node's dotted path. The same concept at a renamed path got a different UUID — a re-import after rename or move created a duplicate instead of recognising the existing node. The export surface therefore treated the ref path as the join key, which conflicted with the stated rule that ref_index and ref_by_uid are lookup indexes, not identity sources.

WP3 removes the path from the identity input. The new signature is:

def derive_domain_import_uuid(
    domain: str, idempotency_key: str, row_identity: str,
) -> str:

The canonical input string is f"domain-import-v2:{domain}:{idempotency_key}:{row_identity}". The "domain-import-v2:" prefix distinguishes new-formula ids from the pre-WP3 "domain-import:" form so the two namespaces never collide in NAMESPACE_DOMAIN_IMPORT. Existing stored node ids are intentionally NOT re-derived — there is no backfill (WP3 rule 3).

row_identity is the row ordinal (round 2)

Callers pass the row's ORDINAL — str(idx) where idx is the row's 0-based position in eidx["nodes"]. The dotted path is content, not identity, so passing the path would reintroduce the pre-WP3 problem (rename ↔ different uid). Two rows at the same ordinal in different import files get different uids because the idempotency_key (_file_hash) differs; two rows at the same ordinal in the SAME file get the same uid because the derivation is deterministic.

The idempotency_key in production is the sha-256 of the import file, filled by parse_eidx into eidx["_file_hash"] so neither preview_import nor commit_import has to grow a third argument. Round-7 fail-closes when _file_hash is missing or empty — an eidx dict handed to preview_import without an idempotency key would make every ordinal uid identical across unrelated files, silently colliding on commit. Tests that build eidx dicts inline must therefore set _file_hash to a stable non-empty string (any sentinel works, since the derivation is deterministic over that value).

The eidx row id field

A row serialised by serialize_domain carries an id ONLY when the stored uid is a canonical lowercase UUID (round-4 roundtrip rule). Rows whose uid is some pre-WP3 non-canonical form OMIT the field, and the importer recognises them by path on re-import (strategy 2 below). The asymmetry keeps the file loss-less for every UUID-shaped id while guaranteeing that every id ever written to disk round-trips through parse_eidx without being rejected as non-canonical. When the id IS written, it is the primary recognition key (strategy 1 below) and makes rename / move / move-to-another-parent lossless. parse_eidx is STRICT about this field:

• Only a canonical lowercase hyphenated UUID is accepted (aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee).
• Hostile shapes — a path (../etc/passwd), uppercase hex, braces, urn:uuid: prefix, non-string types — are rejected with a typed DomainExportError. Preview and commit therefore see a single, byte-stable id verdict and the raw string can never flow into a filename or a UUID derivation.
• The same strictness applies to relation from_id / to_id, and every relation row's type must be a non-empty string (a missing type previously surfaced as a 500 on the API boundary).

IRI form (#954 WP5 S6). serialize_domain writes id, from_id and to_id as the canonical identity IRI infra.iri.node_iri(uid) (https://id.ontoteq.com/{uuid}), and the file carries a top-level @context (infra.jsonld_context.overlay_context()). parse_eidx accepts either a bare canonical UUID or exactly that IRI around a canonical UUID, and writes the bare UUID back into the parsed row, so preview_import / commit_import see one form and a file with uuid ids imports to the same result as one with IRI ids. Any other IRI — a foreign host, a legacy urn:eidos:* or {base}/node/{id} form, a non-UUID or non-canonical UUID inside the IRI — is rejected with the same typed DomainExportError.

Because the ids changed form, eidos_export is bumped from "1.0" to "1.1". Readers older than this change (pre-S6, eidos_export 1.0) reject IRI ids: their parse_eidx accepts only a bare canonical UUID, so a 1.1 file exported from an upgraded instance cannot be imported into an instance that has not been upgraded. The reverse direction is safe — a 1.0 file with bare UUID ids still imports, to the same result as the equivalent 1.1 file with IRI ids.

Legacy (pre-round-2) eidx files that omit id on nodes and relations still parse and import — they just skip the id-based strategy and fall back to path-based lookup. The same path fallback catches rows written by this version whose uid is non-canonical and therefore has no id.

Fail-closed on missing domain root

Both preview_import and commit_import (via its internal preview_import call) refuse the import with a typed DomainExportError when the domain has no resolvable root node, or when the root carries no treeId. The in-domain scope check depends on the root's treeId; without it every row would collapse to "added" and silently duplicate the entire domain on disk. serialize_domain raises the same typed DomainExportError on the export side ONLY when the domain root exists without a treeId — the case where it would otherwise produce an export whose rows no re-import could place in the right tree. When the domain has no root at all, serialize_domain returns an empty-domain export rather than raising, since the result is unambiguous and safe to re-import. At the API boundary DomainExportError reaches the global BackendDomainError handler and surfaces as HTTP 422 — the caller sees a typed error, never a corrupt import.

Recognition order in preview_import

For every eidx row, preview_import tries ALL THREE strategies before deciding; nothing short-circuits on an id that is unknown to the target graph.

1. Row id (domain-scoped) — file_node.get("id") matched against the current graph AND required to live in the import domain's tree. An id that resolves to a node in a DIFFERENT domain's tree is a cross-domain conflict (round 3 rule 2), never an update that would overwrite the other domain's node.
2. Full-path lookup — ref_index[path], with the resolved uid also required to belong to the import domain's tree. Standard case: the node has not been renamed or moved since export, OR the row has no id (legacy export).
3. Notation lookup — the row's leaf segment, passed through derive_slug_segment(leaf, is_root=False), is matched against the domain-scoped notation index AND the match's stored parent is required to equal the row's RESOLVED parent uid (round 3 rule 1 — leaf notations are unique only among siblings, so an unscoped match would silently bind a new row to an unrelated node under a different parent). The notation index is built ONCE per preview and is limited to the import domain. Returns a uid only when exactly one node under the correct parent matches; ambiguous (0 or >1 match) falls through to "added" so no silent collision occurs.

Conflict and "added" branches:

• Cross-domain id conflict — the row id resolves to a node in a different domain's tree. Status is conflict; commit skips the row.
• id/path conflict — id resolves to node A, path resolves to node B, A ≠ B. Two different nodes claim the same row identity; status is conflict and commit skips the row.
• Duplicate-uid conflict (round 6) — two rows in one file resolve to the SAME existing node. Classic shape: row A carries X's stored id but X's old path (matched by id); row B has no id but uses X's current path (matched by path). Both land on X. BOTH rows are marked conflict, naming each other and the shared uid, so neither write silently overwrites the other.
• Duplicate-path conflict (round 7) — two rows in one file both land at the same path within this import (checked via pending_path_uid BEFORE deriving a new uid). Classic shape: two id-less rows with the same NEW path — without this guard both would mint different derived uids and commit would write two nodes at one path. BOTH rows are marked conflict, naming each other and the shared path. The earlier row — whether added, updated, or unchanged — is promoted to conflict so neither write lands.
• Derived-uid-exists conflict (round 7) — a row that fails every recognition strategy (id / path / notation) and would otherwise be added carries a computed uid (derived from (domain, _file_hash, ordinal), or the explicit id field) that is already in the graph. Commit would otherwise silently overwrite that existing node. Classic shape: an id-less file re-imported after the matching node has been renamed — the derived uid lands on the renamed node and would revert the rename. Status is conflict, message is derived-uid-exists with the computed uid and the existing node's path.
• Unresolved-parent (round 4) — the row has no dotted path (so there is no parent segment to resolve), OR the row's parent path does not resolve to a node in the import domain. Status is unresolved-parent and summary["unresolved_parent"] is incremented. The row carries uid=None and never enters pending_path_uid, id_to_resolved, or will_exist (the universe a relation endpoint may land on), so a later relation referencing this row is reported as unresolved rather than forming a dangling endpoint on disk. commit_import skips the row.
• Added with explicit id — nothing resolves AND the row carries an id (lossless cross-instance migration into an empty slot). Commit writes the node file at the row's explicit id.
• Added with derived id — nothing resolves AND the row has no id. The uid is derived from (domain, _file_hash, str(idx)) — the row ORDINAL is the identity input, not the path (path is content, not identity).

A foreign id on an existing path matches via strategy 2, NOT "added", so a duplicate cannot be created.

The summary returned by preview_import always contains added, updated, unchanged, and conflicts; it ALSO contains unresolved_parent whenever at least one row hits the status (the key is omitted when zero to keep the common-case summary small).

commit_import reads the preview, never re-resolves

preview_import writes the resolved uid AND the resolved parent_uid onto every non-conflict row of its result (rule 4). commit_import reads both for added rows and the uid for updated rows — it never recomputes identity and never touches ref_index itself. For a brand-new node that is also a parent of a later row in the same import, preview populates parent_uid from its own pending map so the child's parent lookup succeeds without any commit-side ref_index access. The preview-id-equals-commit-id guarantee is pinned by tests/test_rdf_wp3_domain_import.py::test_wp3_preview_uid_equals_commit_uid.

Relations: endpoints by id, remapped through the node rows

serialize_domain writes from_id and to_id on every relation row alongside from_path / to_path, with a per-endpoint canonical-UUID guard: each *_id is written ONLY when the endpoint uid is a canonical lowercase UUID (round 7). parse_eidx rejects any non-canonical endpoint id, so writing one would make the whole file unimportable; omitting the field per endpoint keeps the export lossless for every UUID-shaped id and lets the importer fall back to the path lookup for every pre-UUID uid. preview_import resolves each endpoint in this order (round 3 rule 3):

1. Through the node rows' resolution map (id_to_resolved) — a foreign from_id or to_id that matched a local node by path lands on the LOCAL uid, not the foreign id. Previously the relation was written verbatim with the foreign id and dangled on disk.
2. Direct in-domain id — the relation references an existing domain node that had no row of its own.
3. Legacy ref_index[path] fallback for pre-round-2 eidx files (or round-7 export rows whose endpoint uid is not canonical and therefore has no *_id), followed by the within-import pending map for intra-import relations between two newly-added nodes.

Every row id AND every relation endpoint id is re-validated as a canonical lowercase UUID inside preview_import itself (round 7 defence at the sink), so no caller can bypass parse_eidx and reach a filename or UUID derivation with a hostile string (../etc/passwd, uppercase hex, braces, urn:uuid:…, non-string types).

A relation is only marked added when BOTH endpoints resolve to a uid that will exist in the import domain after commit (either already present or being committed in this run). Otherwise the row is reported with status unresolved and commit_import skips it — a dangling endpoint never lands on disk. unresolved is the only non-{added, unchanged} relation status; it fires both for a truly missing endpoint and for a relation that references an unresolved-parent node row.

Document nodes keep their old formula

derive_document_import_uuid(source_type, key) is a separate function whose output is byte-identical to the pre-WP3 call derive_domain_import_uuid("documents", f"{source_type}:{key}"). Document node ids therefore stay valid across the signature change — documents_engine.derive_document_uuid routes through it, and tests/test_rdf_g2_uuid_freeze.py::test_freeze_document_import_uuid pins the byte-identity.

Ratchet: ref_index path-join sites

tests/test_rdf_wp3_path_key_ratchet.py scans the import/export + IO scope (backend/persistence/{domain_export,project_import,project_export, export_service}.py and backend/routes/{domain_io,project_io}.py) for ref_index[<path>] and ref_index.get(<path>) call sites — the shapes that use the ref index as a path-to-uid identity join. The baseline is 3 (all in domain_export.py, each sanctioned site named in the test file; round 3 collapsed the two legacy relation-endpoint lookups into a single shared closure, dropping the baseline from 4). The two-sided ratchet prevents any new path-join site from landing without the reviewer naming it first.

Relation Identity by Idempotency Key (WP4 #953)

Before WP4, every overlay relation's UUID was derived from the triple (sourceId, targetId, relationType). The side effect was that the same physical relation could never live under two commands, and moving an endpoint later re-keyed the whole relation — the "node moves" branch of cross-domain healing therefore had to tombstone and rewrite. WP4 moves relation identity off endpoint hashing: a relation's UUID now derives from the CreateRelation command's idempotency key instead. Same command → same UUID; moving an endpoint later does not change the relation's identity.

There is no stored-data rewrite. Existing relation files keep the UUID they already have on disk; IRI = https://id.ontoteq.com/<stored_id> remains valid for every pre-WP4 overlay. Only new overlay relations and new import-committed relations use the new derivation.

derive_overlay_relation_uuid — new signature

def derive_overlay_relation_uuid(idem_key: str) -> str:

Param	Type	Description
idem_key	str	The CreateRelation command's idempotency key — the client's previewId for the HTTP path, the per-row key for the import router (relation_import_idem_key(file_hash, anchor, row_idx))

Returns: a UUID v5 string. Routes through derive_entity_uuid with the canonical envelope {commandType: "CreateRelation", idempotencyKey: idem_key}, so the engine, the HTTP router, and the agent path all land on the same UUID when they share an idem_key.

The pre-WP4 3-argument form (source, target, type) raises TypeError at the Python call layer — the function now takes a single positional argument, so passing extra positional args fails before any derivation runs. No in-function type check is needed; the arity mismatch is enough to stop a stale call site from silently hashing a tuple of endpoints.

The owner-authoritative rules the derivation protects:

1. Deterministic. Same idem_key → same UUID across preview and commit; retries land on the same file.
2. Endpoint-invariant. A MoveRelation can later edit eidos:source or eidos:target without changing the stored UUID.
3. Explicit duplicate check. The old scheme coalesced duplicates automatically (same triple → same UUID → file-exists guard). Under WP4 that invariant moves into graph_crud_engine.create_relation via _find_existing_triple, called under the write lock. The check matches three channels against the effective graph, keyed by the relation's active channel:
4. in-model rows match on (source, target, predicate);
5. external-domain rows (command carries externalDomain or externalTargetRef) match on (source, externalTargetRef, predicate);
6. unresolved rows (command carries unresolvedTargetRef) match on (source, unresolvedTargetRef, predicate).

A hit returns the existing relation's uid with ok=True, changed=False. The engine itself (_write_dup_idem_record) persists an idempotency record pointing at that uid before returning, so a retry with the same per-row idem_key resolves through the idempotency store even when nothing new was written. The router carries no parallel idem write.

derive_domain_import_relation_uuid and derive_document_import_relation_uuid — batch-import relations

def derive_domain_import_relation_uuid(
    file_hash: str, row_ordinal: str,
) -> str:

def derive_document_import_relation_uuid(
    fingerprint: str, row_ordinal: str,
) -> str:

Param	Type	Description
file_hash	str	sha-256 of the .eidx domain-import file (eidx["_file_hash"])
fingerprint	str	DocuGraph repo fingerprint on the parsed ImportedRepo
row_ordinal	str	0-based position of the relation row in the preview's relations list — the same stable list commit/commit_import iterates

Returns: a UUID v5 string. The derivations are byte-equal to derive_overlay_relation_uuid(f"domain-import-rel:{file_hash}:{row_ordinal}") and derive_overlay_relation_uuid(f"document-import-rel:{fingerprint}:{row_ordinal}") respectively — the single-key canonicalisation path drives both the overlay flow and the import flow.

Round-9 (sendback 2026-10-14 item 5) splits the two importers into distinct prefixes. Earlier revisions used a single "domain-import-rel:" prefix for both .eidx file hashes and DocuGraph fingerprints; because the derivation feeds UUIDv5 with a single opaque string, any accidental hash/fingerprint collision could have produced the same relation uid for two unrelated imports. The split guarantees the two id spaces are disjoint by construction, even if a file hash and a fingerprint ever happen to match.

persistence.domain_export.commit_import calls derive_domain_import_relation_uuid with the .eidx file hash; domain.documents_import.commit calls derive_document_import_relation_uuid with the DocuGraph fingerprint. Same (key, ordinal) → same UUID across preview and commit for either importer; moving an endpoint never changes the import-relation identity.

Both importers also run a (from, to, predicate) triple-level de-dup check before writing a relation file. The set is seeded from the live graph (eff["relations"]) at the start of the loop AND grown by every triple as it is written — so a single eidx or FME file that lists the same triple twice writes it exactly once, on top of the usual guard against re-importing a triple that already exists in the graph.

Seeded relation ids — retired from the backend (#935 S5a)

derive_seeded_relation_uuid and eidos_loader.seeded_rel_id — the endpoint-hash identity uuid5(NAMESPACE_SEEDED_RELATION, f"{from_id}::{predicate}::{to_id}") of a .eidos base relation — are deleted: the backend no longer builds a base graph from a .eidos snapshot. The formula survives verbatim, frozen, only in tools/migration/eidos_archive.seeded_rel_id, where convert derives an archive's relation ids to check for tombstones on base relations.

Rule 5 ratchet — test_rdf_wp4_endpoint_ratchet.py

A two-sided baseline=0 scanner counts BOTH seeded_rel_id( AND derive_seeded_relation_uuid( call sites in backend/. Its exempt set is empty since #935 S5a (it held eidos_loader.py and key_derivation.py, the two sites S5a deleted), so any backend module that calls EITHER form is a regression. New callers must route through derive_overlay_relation_uuid(idem_key) or derive_domain_import_relation_uuid(import_key, ordinal) instead.

MISSING_FINGERPRINT — DocuGraph FME import rejection

domain.documents_import.parse_and_validate rejects any FME payload whose eidos:fingerprint is absent or empty with ImportValidationError(code="MISSING_FINGERPRINT", ...). The rejection is a hard precondition for WP4 rule 4: the relation UUID is derive_document_import_relation_uuid(fingerprint, ordinal), so a missing fingerprint collapses the whole file into a single ordinal→UUID namespace and lets two unrelated repos land their relations at the SAME UUIDs. The parser is the single gate — every caller downstream (preview, commit, risk assessment) sees a validated ImportedRepo with a non-empty fingerprint.

Explicit-id import tombstone protection (WP4 rule 7)

persistence.domain_export.preview_import enforces the tombstone check by default: when the caller omits the optional node_tombstone_ids: frozenset[str] | None parameter, the function loads the overlay's current node_tombstones set itself (via _resolve_tombstone_ids → mutation_store.load_overlay). An eidx row whose explicit id sits in that set is reported as a conflict with the tombstoned: message instead of silently taking the lossless cross-instance-migration path. The HTTP preview/commit endpoints and commit_import therefore inherit the guard without having to pre-load the tombstone set themselves — a resurrection via import is impossible by default.

Callers that already own the tombstone set (project import, tests) can still pass it explicitly to avoid a redundant overlay read. Only an explicit frozenset() (empty set) opts out — the None sentinel means "load the current set".

The engine owns the duplicate-triple idempotency record. When graph_crud_engine.create_relation short-circuits a duplicate triple with ok=True, changed=False, it writes an idempotency record keyed on the per-row idem_key and pointing at the existing relation's uid, using key_derivation.canonical_payload_hash(cmd) for the hash field. The bulk-import router does not keep a parallel copy of this write — a retry with the same idem_key resolves through the idempotency store even when nothing new was persisted.

verified_at_commit

The claims on this page are held against the verified_at_commit pin in deterministic-uuid.yaml. It was re-pinned in #956 K1 to f37a1e648a0335bb272ec2f5ac0dc43b8373dc37 (main after #954 WP5 S7), where the WP4 derivation claims and the identity-IRI section above were checked against backend/infra/key_derivation.py, backend/infra/iri.py and backend/server.py.