Export Engine¶
Verified — minor divergences from the code — 2 finding(s) · 19d408be · 2026-08-25
The compiled doc page (docs/manual/dev/export-engine.md) is exhaustively accurate against the code — all ~35 canonical_operations symbols (frontend plugin functions, EidosExportRegistry, backend routes, export_service functions) were confirmed present with matching names/signatures/behavior across frontend/plugins/*.js, frontend/app.js, backend/server.py, backend/persistence/export_service.py, backend/api/mutations_io_router.py, backend/api/api_reports_router.py, and backend/routes/project_io.py. Two minor divergences found.
Divergences from the code — details
| Sev | Where | Doc says | Code does | Evidence |
|---|---|---|---|---|
| MEDIUM | responsibility (formatVersion) | KM YAML responsibility field: 'produces full-model JSON-LD exports (FME) via build_export_doc with formatVersion "1.0.0"' | export_service.py defines FORMAT_VERSION = "1.3.0" (module-level constant used throughout build_export_doc); the compiled .md page itself correctly states "1.3.0" everywhere, so only the KM entity YAML's responsibility prose is stale. | backend/persistence/export_service.py:24 |
| LOW | _render_export_bytes | Doc's Data Staging Endpoints section gives the internal function signature as _render_export_bytes(headers: list, rows: list, fmt: str) -> tuple[bytes, str] |
The actual function signature is _render_export_bytes(headers: list, rows: list, fmt: str, column_meta: list = None) -> tuple[bytes, str] — an undocumented column_meta parameter drives per-column styling/alignment/numeric formatting in the xlsx branch, and /export/stage passes payload.get('column_meta') into it. |
backend/server.py:3782 |
Layer: api
Overview — Export Architecture¶
The Export Engine is responsible for all file downloads that EIDOS Explorer produces: structured data tables (CSV/XLSX) and rendered diagrams (PDF). It is built around a plugin-per-format model: every export format lives in a dedicated JavaScript file under frontend/plugins/. Each plugin registers itself with a central in-process registry (EidosExportRegistry) at page load time. The application core (app.js) does not know about any specific format; it only knows how to open the registry, present the user with a choice, and dispatch to the chosen plugin.
The split between "data exports" and "PDF reports" is fundamental to the architecture:
- Data exports run entirely in the browser for data assembly (calling backend REST endpoints to fetch raw graph data), then POST the rendered rows to a backend staging endpoint (
POST /export/stage) which returns a short-lived token. The browser immediately follows up withGET /export/d/{token}to download the file. - PDF report exports (marked
kind: 'pdf-report') follow a three-stage pipeline mandated by ADR-007 Approach C: the browser GETs a Mermaid diagram source from the backend, renders the SVG client-side using the Mermaid library, then POSTs the SVG back to the server for PDF assembly using ReportLab/svglib.
All 11 Export Formats¶
| Registry ID | Plugin File | Category | Description |
|---|---|---|---|
boq |
export-boq.js |
data | Bill of Quantities: one row per hasType-grouped component type with count and paths |
component_list |
export-electrical.js |
data | All Component-typed nodes with type, description, location |
signal_list |
export-electrical.js |
data | Terminals carrying hasSignal relations, grouped by signal |
cable_list |
export-electrical.js |
data | IEC-prefixed cable nodes with from/to system and component detail |
relation_list |
export-relations.js |
model | All effective outgoing relations for the primary domain, import-compatible |
property_list |
export-properties.js |
model | Engineering properties (direct and inherited) for every node |
product_nodes |
export-products.js |
model | Complete product-node tree snapshot with root, import-compatible |
product_nodes_template |
export-products.js |
model | Product nodes without the root row, for re-import under any anchor |
system_list |
export-products.js |
data | Functional and/or Technical systems filtered by node type |
location_nodes |
export-locations.js |
model | Complete location-domain node tree with root |
location_nodes_template |
export-locations.js |
model | Location nodes without root, for re-import under any anchor |
reference-whitelist |
export-reference-whitelist.js |
model | Styled XLSX per DS/ISO 81346-12:2018 Annex A, whole-project only |
fsb-depth |
export-system-breakdown.js |
model | Functional System Breakdown PDF, depth-gradient palette |
fsb-category |
export-system-breakdown.js |
model | Functional System Breakdown PDF, category-hue palette |
system-context |
export-system-context.js |
model | System Context & Interface Diagram PDF anchored at selected node |
Note: report-ref-arch.js also loads as a plugin but does not register with EidosExportRegistry; it exposes window.triggerRefArchDownload() and is invoked from the Insights menu independently.
Export Plugin Interface¶
Registry Contract¶
Every export plugin must obtain references to the two global singletons and call registry.register() before returning:
const registry = window.EidosExportRegistry;
const core = window.EidosCore;
if (!registry || !core) {
console.warn('[my-plugin] EidosExportRegistry or EidosCore not found — plugin skipped');
return;
}
registry.register('my-export-id', { /* definition */ });
Data Export Definition¶
For formats that produce tabular rows (CSV/XLSX), the definition object must include:
registry.register('my-export-id', {
label: 'Human-readable name shown in the dropdown',
headers: ['Column A', 'Column B'], // header row for the file
keys: ['col_a', 'col_b'], // property names on each row object
build: buildMyRows, // async function (scope, setMsg, options?) → row[]
summarize: summarizeMyRows, // optional: (rows) → [{label, count}]
category: 'model' | 'data', // 'model' = schema exports; 'data' = derived listings
options: [ /* radio option defs */ ], // optional: triggers configure card before preview
});
build(scope, setMsg, options) is the core contract:
async function buildMyRows(scope = 'global', setMsg = () => {}, options = {}) {
// scope: 'global' | 'subtree'
// setMsg: (message: string) => void — call to update the loading indicator
// options: object with values from any declared options (radio choices)
// Return: array of plain objects whose keys match the keys array
}
| Param | Type | Description |
|---|---|---|
scope |
'global' \| 'subtree' |
Whether to include all nodes or only the selected subtree |
setMsg |
(msg: string) => void |
Callback to update progress text in the export preview panel |
options |
object |
Key/value pairs from any declared option radios (e.g. {systemType: 'functional'}) |
PDF Report Definition¶
For PDF report exports, the definition uses kind: 'pdf-report' and replaces build/headers/keys with a trigger function:
registry.register('my-pdf-id', {
kind: 'pdf-report',
category: 'model',
label: 'My PDF Report',
description: 'Description shown on the configure card.',
wholeProjectOnly: false, // true → disables subtree scope radio
actionLabel: 'Generate PDF', // button text on the card
options: [ /* radio option defs */ ],
trigger: myTriggerFn,
});
trigger(opts, setMsg) is the PDF plugin contract:
async function trigger(opts, setMsg) {
// opts: merged object of {scope: 'global'|'subtree', ...optionValues}
// setMsg: (message: string) => void
// Must throw on error — the caller surfaces err.message to the user
}
| Param | Type | Description |
|---|---|---|
opts |
object |
Includes scope from the dialog plus values of any declared option radios |
setMsg |
(msg: string) => void |
Progress indicator callback |
How the Application Activates Plugins¶
Plugins are not statically imported. At boot, loadConfig() fetches /config from the backend, which returns the list of plugin IDs enabled for this instance. _activatePlugins(ids) then injects a <script> tag into the document head for each enabled plugin:
function _activatePlugins(ids) {
return Promise.all((ids || []).map(id => {
const path = _KNOWN_PLUGINS[id];
if (!path) return Promise.resolve();
return new Promise(resolve => {
const s = document.createElement('script');
s.src = path + '?v=' + _DEPLOY_VER; // cache-busted per deploy
s.onload = resolve;
s.onerror = () => {
console.warn(`[EIDOS] Plugin failed to load: ${id}`);
resolve(); // non-fatal: other plugins still load
};
document.head.appendChild(s);
});
}));
}
Each plugin script executes as it loads and calls registry.register() during that execution. After all plugin promises settle, _pluginsLoaded is set to true. openExportModal() is a no-op if called before this flag is set (BL-IE-015).
The mapping of plugin ID to file path lives in _KNOWN_PLUGINS in app.js. A plugin ID absent from this map is silently ignored, which means new plugins require a corresponding entry in _KNOWN_PLUGINS in addition to their file.
Backend Export Endpoints¶
Data Staging Endpoints¶
POST /export/stage — accepts a pre-built data payload and returns a one-time download token.
Request body:
{
"headers": ["Col A", "Col B"],
"rows": [["val1", "val2"]],
"format": "xlsx",
"filename": "my-export"
}
| Field | Type | Description |
|---|---|---|
headers |
list[str] |
Column header row |
rows |
list[list] |
Data rows; each inner list must align with headers |
format |
"xlsx" \| "csv" |
Output format; CSV is UTF-8 BOM-encoded for Excel compat |
filename |
str |
Base filename (extension is appended by the server) |
Response:
{ "url": "/export/d/<token>" }
The token is a 16-byte URL-safe random string with a 5-minute TTL. Maximum 100,000 rows. The staged file is written to the EXPORT_STAGE_DIR directory (default /tmp/eidos-export-stage).
GET /export/d/{token} — streams the staged file with Content-Disposition: attachment. Bearer auth required, and the caller must be the user who staged the token (meta.owner, recorded by /export/stage) — an unguessable token is not sufficient on its own; a leaked URL is not replayable by anyone else (hardening 0.4, #514). The ownership check runs before the file is consumed, so a foreign caller (403) cannot burn the single-use token. The file is deleted from disk immediately after a successful owner download.
_render_export_bytes(headers, rows, fmt) — internal function called by the staging endpoint:
def _render_export_bytes(headers: list, rows: list, fmt: str) -> tuple[bytes, str]:
# Returns (file_bytes, content_type)
# fmt == 'xlsx': uses openpyxl Workbook
# fmt == 'csv': uses csv.writer with ';' delimiter, UTF-8 BOM prefix
Raw Data Endpoints (called by plugins)¶
GET /mutations/products/export
Query parameters:
include_root: bool (default false) — include the tree root row
Returns one row per product node in depth-first order. Row shape: {node_path, reference, description, nodeType, note}. Powered by build_product_export_rows() in backend/api/excel_export.py. IAM-filtered: nodes the caller cannot read never appear.
GET /mutations/relations/export
Returns all effective outgoing relations for the primary domain. Row shape: {source_path, target_path, relation_type, inherit_source, is_inherited}. Includes inherited relations computed by _resolve_effective_outgoing(). IAM-filtered: targets outside the caller's read scope are replaced with [restricted].
GET /mutations/properties/export
Returns all engineering properties (direct and inherited) for every node in the primary domain. Row shape: {node_path, property, value, unit, group, inherited, source_path}. IAM-filtered at both the node and the inheritance source ancestor.
GET /api/proxy/{server}/mutations/products/export
Proxy endpoint that surfaces a secondary domain's node export (e.g. the Location domain) through the primary server. Used by export-locations.js so the location plugin can call apiFetch('/mutations/products/export', locationBase()) without CORS issues.
PDF Report Endpoints¶
These are owned by the Reports Engine but are called directly by PDF export plugins:
| Endpoint | Method | Description |
|---|---|---|
GET /api/reports/system-breakdown |
GET | Returns {mermaid_src, anchor_description} for the breakdown diagram |
POST /api/reports/system-breakdown/pdf |
POST | Accepts {svg, palette, scope, ...}, returns application/pdf |
GET /api/reports/system-context |
GET | Returns {mermaid_src, anchor_label, anchor_description} for context diagram |
POST /api/reports/system-context/pdf |
POST | Accepts {svg, lang, anchor_label, anchor_description}, returns application/pdf |
GET /api/reports/reference-whitelist.xlsx |
GET | Returns a styled multi-sheet XLSX directly |
The PDF POST endpoints set response headers X-Render-Scale, X-Render-Pt, and X-Render-Legible so the frontend can warn the user when body text falls below 9pt.
A Complete Export Flow¶
Example: Bill of Quantities (BOQ) Export¶
1. User opens the Export dialog
The user clicks the Export button. openExportModal() calls EidosExportRegistry.getAll(), groups the registered types by category, and populates the <select> element. The user selects "Bill of Quantities" and chooses scope "All nodes", then clicks "Preview".
2. openExportPreview() dispatches to _runDataExportPreview()
async function openExportPreview() {
const def = EidosExportRegistry.get('boq');
// def.kind is undefined (not 'pdf-report'), def.options is undefined
// → goes directly to _runDataExportPreview
await _runDataExportPreview(def, 'boq', 'global');
}
3. buildBoqRows(scope, setMsg) runs in the browser
async function buildBoqRows(scope = 'global', setMsg = () => {}) {
// Step A: fetch the full node list
const probe = await apiFetch('/nodes?limit=1&offset=0');
const total = probe.total || 0;
const data = await apiFetch(`/nodes?limit=${total}&offset=0`);
// Step B: in batches of 20, fetch /node/{id} for each candidate
// and resolve hasType relations to type nodes
const BATCH = 20;
for (let i = 0; i < candidates.length; i += BATCH) {
setMsg(`Resolving types… ${i} / ${candidates.length} nodes`);
await Promise.allSettled(batch.map(async c => {
const nd = await apiFetch(`/node/${c.id}`);
const rel = outgoing.find(r => r.predicate === 'hasType');
// group by type target URI → {typeName, count, paths[]}
}));
}
// Step C: sort, number rows, return
return rows; // [{linjeId, typeName, typeDesc, docLink, quantity, components}]
}
4. Preview table renders in the browser
_runDataExportPreview stores the rows in _exportRows, calls _repaintExportWrap() to render the table, and enables the Download buttons.
5. User clicks "Download XLSX"
The download handler collects _exportRows, maps each row to an array using the plugin's keys definition, and POSTs to /export/stage:
const body = {
headers: def.headers, // ['No.', 'Type', 'Description', ...]
rows: _exportRows.map(r => def.keys.map(k => r[k] ?? '')),
format: 'xlsx',
filename: 'bill-of-quantities-2026-06-23',
};
const { url } = await fetch('/export/stage', { method: 'POST', body: JSON.stringify(body), ... }).then(r => r.json());
// The download is owner-gated bearer-auth (hardening 0.4, #514) — a plain
// navigation cannot carry the Authorization header, so fetch WITH authHeaders()
// and save the returned blob under the real filename:
const dl = await fetch(`${BASE_URL}${url}`, { headers: authHeaders() });
_downloadBlob(await dl.blob(), filename); // deferred-revoke helper in app.js
6. Server stages and streams the file
export_stage renders the XLSX using openpyxl, writes it to the staging directory with a random token, records the issuing user as meta.owner, and returns {"url": "/export/d/<token>"}. The browser fetches that URL with its bearer header; export_download verifies the session and that the caller equals meta.owner (before consuming — a foreign 403 does not burn the token), then reads the file, deletes it from disk, and returns it as Content-Disposition: attachment.
Example: System Context PDF Export¶
1. User selects a node in the tree, then opens the Export dialog and selects "System Context & Interface Diagram".
2. openExportPreview() routes to _renderPdfReportCard(def, scope)
The configure card appears showing a language radio (English/Dansk) and an action button.
3. User clicks "Generate PDF"
The card's click handler calls def.trigger({lang: 'en', scope: 'global'}, setMsg).
4. trigger() in export-system-context.js runs the three-stage pipeline
async function trigger(opts, setMsg) {
const anchorId = core.selectedNodeId;
if (!anchorId) throw new Error('No node selected...');
// Stage 1: Fetch Mermaid source from backend (IAM-filtered)
setMsg('Building diagram…');
const srcResp = await fetch(
BASE_URL + '/api/reports/system-context?anchor_node_id=' + anchorId + '&lang=en',
{ headers: authHeaders() }
);
const srcBody = await srcResp.json();
// srcBody: { mermaid_src: '...', anchor_label: '...', anchor_description: '...' }
// Stage 2: Render SVG client-side (ADR-007 Approach C)
setMsg('Rendering diagram…');
const mermaid = await loadMermaid(); // lazy CDN load, singleton
const { svg } = await mermaid.render('eidosContext_' + Date.now(), srcBody.mermaid_src);
// Stage 3: POST SVG to backend for PDF assembly
setMsg('Building PDF…');
const pdfResp = await fetch(
BASE_URL + '/api/reports/system-context/pdf',
{
method: 'POST',
headers: { ...authHeaders(), 'Content-Type': 'application/json' },
body: JSON.stringify({ svg, lang: 'en', anchor_label: srcBody.anchor_label, anchor_description: srcBody.anchor_description }),
}
);
// Stage 4: Trigger download
const blob = await pdfResp.blob();
const a = document.createElement('a');
a.href = URL.createObjectURL(blob);
a.download = 'system-context-2026-06-23.pdf';
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
URL.revokeObjectURL(a.href);
}
The backend /api/reports/system-context applies IAM filtering before building the Mermaid source. The backend /api/reports/system-context/pdf trusts the SVG from the browser and assembles the PDF using svglib + ReportLab. Legibility is checked post-render and surfaced via X-Render-Legible / X-Render-Pt headers.
Data Structures¶
Plugin Definition Object¶
{
// Required for all plugins:
label: string, // shown in the export type dropdown
category: 'model' | 'data',
// For data exports (kind absent or not 'pdf-report'):
headers: string[], // column header row
keys: string[], // property names matching headers, same order
build: async (scope, setMsg, options?) => object[],
summarize: (rows) => {label: string, count: number}[], // optional
options: OptionDef[], // optional, triggers configure card
cellRender: { [key]: (val) => string }, // optional HTML renderer per column
// For PDF reports (kind === 'pdf-report'):
kind: 'pdf-report',
description: string,
wholeProjectOnly: boolean,
actionLabel: string,
trigger: async (opts, setMsg) => void,
options: OptionDef[],
}
OptionDef (Radio Option)¶
{
id: string, // key in the options bag passed to build()/trigger()
label: string, // displayed label in the configure card
type: 'radio',
default: string, // value of the initially selected choice
choices: { value: string, label: string }[],
}
Export Row (data exports)¶
Each row is a plain object. Keys are defined by the plugin's keys array. Values are always strings (the staging endpoint serialises each cell as a string). For example, the BOQ row:
{
linjeId: '1',
typeDesc: 'Centrifugal process pump',
typeName: 'Pump',
quantity: '4',
}
Staged Export Token¶
// POST /export/stage response:
{ "url": "/export/d/<token>" }
// Filesystem layout on server:
// <EXPORT_STAGE_DIR>/<token>.bin — rendered file bytes
// <EXPORT_STAGE_DIR>/<token>.meta.json — { filename, format, content_type, owner }
// (owner = user_id of the staging caller; the download requires owner match — #514)
Error Handling¶
Plugin-Level Guards¶
Every plugin begins with a guard that bails out silently when global singletons are unavailable. This prevents a failed plugin from blocking the page load:
if (!registry || !core) {
console.warn('[my-plugin] EidosExportRegistry or EidosCore not found — plugin skipped');
return;
}
Load errors are caught by _activatePlugins's s.onerror handler, which resolves the plugin's promise without rejecting, so a network error fetching one plugin file does not prevent others from loading.
PDF Export Error Mapping¶
Both export-system-breakdown.js and export-system-context.js implement a _friendlyExportError(rawText, status, stage) helper that parses the server's {detail: "..."} JSON envelope and maps known error cases to user-actionable copy:
| Condition | User-facing message |
|---|---|
| "components cannot anchor" in detail | Direct user to select a non-Component node |
| HTTP 403 / "forbidden" in detail | Permission denied, contact admin |
| HTTP 404 / "not found" in detail | Node deleted or moved, refresh tree |
| HTTP 5xx | Server error, try again |
| Unknown | Raw detail string from server |
Staging Endpoint Validation¶
POST /export/stage enforces:
- format must be "xlsx" or "csv" — returns HTTP 400 otherwise
- headers and rows must be lists — returns HTTP 400 otherwise
- rows length must not exceed 100,000 — returns HTTP 413 otherwise
GET /export/d/{token} requires a valid bearer session (401 without) and validates the token against a strict [A-Za-z0-9_\-]{8,64} pattern before any filesystem access, returning HTTP 400 on mismatch. Expired or missing tokens return HTTP 404. A caller who is not the staging owner (meta.owner) receives HTTP 403 — without consuming the single-use token (#514).
Data Export Batch Errors¶
Data plugins fetch node detail in batches using Promise.allSettled(), so a single failed /node/{id} request does not abort the whole export — the failing node is simply skipped. The export-electrical.js cable list explicitly handles missing from/to sides by returning an EMPTY_SIDE sentinel, so a cable with no connections still appears in the output with blank endpoint columns.
Data Freshness Invariants¶
Per-run cache (cross-domain nodes): Every build function that resolves cross-domain targets (location, type) creates a fresh const runCache = new Map() at the top of the function. This Map deduplicates identical target URLs within a single export run while guaranteeing that a new export always fetches live data — no stale cross-domain node data can bleed in from a previous run. A module-level Map is forbidden for node data; only _domainsJsonCache (domain config, which does not change during a session) is permitted at module scope.
From/To component nodes in cable list: buildCableListRows must fetch the component nodes at each cable end via apiFetch('/node/{id}'), not via fetchNode() / core._fetchNodeByTarget(). _fetchNodeByTarget reads the module-level nodeCache, which holds a snapshot from the last time the user navigated to that node. After an in-session location edit the cached copy still carries the old isLocatedIn relation, making the export show stale location data. Direct apiFetch bypasses the cache and always returns the current backend state. (Regression: BL-IE-036)
Integration with Other Engines¶
Mutation Store / Effective Graph: The three raw data endpoints (/mutations/products/export, /mutations/relations/export, /mutations/properties/export) all call mutation_store.get_effective_graph() to obtain the live merged graph (seed + overlays + rule-derived edges). This means exports always reflect the current state including unsynchronised overlay changes.
IAM Engine: All export endpoints depend on the canonical auth surface (api/deps.py — CurrentUser for read exports, AdminOrService for admin/service exports) and pass the resolved user identity through is_node_accessible(). Nodes or relation targets outside the caller's read scope are excluded (products/properties) or masked as [restricted] (relations) before rows are returned.
Reports Engine: PDF export plugins directly call the Reports Engine's endpoints (/api/reports/system-breakdown, /api/reports/system-context). The Export Engine owns only the trigger-and-download half of the pipeline; diagram source generation and PDF assembly are fully owned by the Reports Engine (backend/reports/).
Excel Importer/Exporter: build_product_export_rows() in backend/api/excel_export.py is the single canonical owner of the product node row shape. It is imported by /mutations/products/export and mirrors excel_importer.parse_product_node_rows() column for column, guaranteeing that exported files round-trip through the importer without data loss.
Domain Registry: export-locations.js reads core.DOMAIN_REGISTRY.domains.location.baseURI at call time to resolve the Location domain's base URL. If no Location domain is registered, the plugin throws a descriptive error rather than silently producing empty output.
Adding a New Export Format¶
Follow these exact steps to add a new export type to EIDOS Explorer.
Step 1: Create the plugin file¶
Create frontend/plugins/export-myformat.js. Use an IIFE to avoid polluting the global scope:
(function () {
'use strict';
const registry = window.EidosExportRegistry;
const core = window.EidosCore;
if (!registry || !core) {
console.warn('[export-myformat] EidosExportRegistry or EidosCore not found — plugin skipped');
return;
}
const { apiFetch } = core;
async function buildMyRows(scope = 'global', setMsg = () => {}) {
setMsg('Fetching data…');
const rows = await apiFetch('/mutations/myformat/export');
return rows;
}
registry.register('myformat', {
label: 'My Format',
headers: ['Column A', 'Column B'],
keys: ['col_a', 'col_b'],
build: buildMyRows,
category: 'data', // or 'model'
});
}());
Step 2: Register the plugin ID in app.js¶
Add an entry to _KNOWN_PLUGINS in frontend/app.js:
const _KNOWN_PLUGINS = {
// ... existing entries ...
'export-myformat': 'plugins/export-myformat.js',
};
Without this entry, the backend config can list the plugin ID but _activatePlugins will silently skip it because _KNOWN_PLUGINS[id] returns undefined.
Step 3: Add a backend data endpoint (if needed)¶
If the format requires new data that no existing endpoint provides, add a FastAPI route to backend/server.py:
from api.deps import CurrentUser
@app.get("/mutations/myformat/export")
async def mutations_myformat_export(auth: CurrentUser):
"""Return rows for My Format export."""
user_id = auth.user_id
_require_mutations()
eff = mutation_store.get_effective_graph()
# Build and return rows as a list of dicts
return rows
Apply IAM filtering before returning rows (see /mutations/products/export as the reference implementation).
Step 4: Enable the plugin in the instance configuration¶
The backend /config endpoint returns the list of plugin IDs for the current instance. Add 'export-myformat' to the plugins list in the instance configuration file (domains.json or the equivalent deployment config). The exact mechanism is instance-specific; consult backend/routes/setup_router.py and the deployment documentation for the target instance.
Step 5: Write tests¶
Add a test file tests/test_myformat_export.py (backend endpoint) and frontend/plugins/__tests__/export-myformat.test.js (plugin build function). The existing test files tests/test_bl_ie_011_product_export.py and frontend/plugins/__tests__/export-boq.test.js are the canonical references.
Full-Model Export (FME)¶
export_service.build_export_doc() (in backend/persistence/export_service.py) assembles the full-model JSON-LD dump from pre-fetched domain data, producing a document with:
| Field | Value |
|---|---|
@type |
eidos:FullModelExport |
eidos:formatVersion |
"1.3.0" (additive-only within the major version — a major bump requires a migration procedure) |
eidos:mode |
"view" (default) or "archive" |
eidos:delivery |
{"source": "live"} — always present; signals the export was assembled from the live server |
eidos:fingerprint |
SHA-256 of canonical JSON of domains + auditLog + relationCatalog |
eidos:auditLog |
Included only when audit_log is not None |
build_export_doc(name, domains, audit_log, mode="view", interface_lookup=None)
# SOURCE: backend/persistence/export_service.py
doc = {
"@context": EIDOS_CONTEXT,
"@type": "eidos:FullModelExport",
"eidos:formatVersion": FORMAT_VERSION, # "1.3.0"
"eidos:mode": mode, # "view" | "archive"
"eidos:delivery": {"source": "live"},
"eidos:fingerprint": fingerprint,
...
}
FORMAT_VERSION = "1.3.0" is a module-level constant. Evolution is additive-only within the major version; consumers parse eidos:formatVersion and reject exports whose major does not match. A top-level eidos:metrics field carrying the Metrics Engine catalog's archive-mode values arrives in that engine's Wave 2.
The WORLD node is extracted from every domain's node list by _extract_world() and placed once at eidos:world. hasDomain edges from WORLD to each domain root are materialized by _has_domain_edges() — they are implicit in per-domain files but explicit in the merged export. _resolve_external_refs() runs after archive-mode stamping so that lookup keys match the stored uuids the IIS detector indexed.
webURI (view mode) — canonical node weblinks¶
In mode: view exports, each eidos:hasDomain edge carries a webURI — the canonical deep-link origin: the primary public origin (BASE_URL), the same value for every domain. A node deep link is domain-agnostic — {origin}/node/{uuid} — because the consumer resolves the owning domain itself (see the canonical-deep-link invariant; Contract 02.01: route by identity, never transport). So webURI must not be a per-domain routing publicURI — for a secondary that is the /api/proxy/{domain} path, which would leak transport into the link. Consumers build a direct link to a node as:
webURI + "/node/" + <node id>
and wrap it like the Explorer share link:
<explorer origin><path>#<url-encoded node URI>
The value is built by export_service.canonical_web_uris() from the primary's BASE_URL and is identical on every hasDomain edge; the per-domain publicURI is intentionally not used (it stays the internal routing key). If BASE_URL is empty the map is empty and no webURI is stamped. webURI is omitted in mode: archive and in reconstruction / as_of exports (host-neutral). It lives only on hasDomain edges — never on the domains payload or per node — so it does not affect eidos:fingerprint. Added in formatVersion 1.3.0 (additive, non-breaking).
filename_for(name) produces a sanitised <name>-<YYYY-MM-DD>.jsonld filename for the export payload.
Tombstones and Deletion History¶
load_tombstones(mutations_dir) reads deletion history for the archive export (FMDE D3). It scans two directories:
MUTATIONS_DIR/node_tombstones/*.jsonld— node tombstones withdeletedAt,deletedBy,auditRefMUTATIONS_DIR/tombstones/*.jsonld— relation tombstones (creation time = deletion moment; exposed asdeleted_at)
Malformed files are skipped with a warning log, never fatal — same tolerance as mutation_store.load_overlay.
GET /tombstones — served from backend/server.py. Returns the full tombstone payload ({"nodes": [...], "relations": [...]}) for admin and archive consumers. No pagination; the tombstone set grows with each deletion event. IAM-protected: requires a valid session.
Admin Data Exchange Tab¶
The admin panel exposes a single Data Exchange tab (#admin-content-dataexchange in frontend/index.html, activated by switchAdminTab('dataexchange') in frontend/app.js). It consolidates every "get data in or out of the project" affordance the admin surface offers, replacing the historical Snapshots + Export split that arrived in three separate feature waves (April 2026 FME, May 2026 domain snapshot, June 2026 project exchange, July 2026 FMDE). The consolidation is pure UI-IA — every endpoint, request/response shape, server filename, auth model, capability model, and file format is unchanged (see issue #425 §5 for the full compatibility statement).
Section order and vocabulary¶
The tab is grouped by scope first, purpose second, in this fixed order:
- Project (round-trip) —
GET /admin/project/exportandPOST /admin/project/import/{preview,commit}. Portable.eidx-projectZIP bundling every domain plus a manifest. Promoted to the top because it is the newest and highest-fanout surface (it internally callsserialize_domain()per reachable domain and zips the result). - Domain (round-trip) —
GET /domain-exportandPOST /domain-import/{preview,commit}. Single-domain.eidx.json. A convenience surface over the sameserialize_domain()primitive that Project Export bundles. - Full Model — analysis snapshot —
GET /admin/export/full-model/{info,download}. Merged, UUID-addressed.jsonldfor downstream consumers. No import counterpart — it is a frozen analysis snapshot, not a round-trip file. Both facets live behind one endpoint (mode=viewandmode=archive&as_of=YYYY-MM-DD). - Audit Log (provenance) —
GET /admin/audit/export. Append-only.jsonlprovenance stream, one record per mutation. No import counterpart by design. - Mutation checkpoints —
GET /snapshotsandGET /snapshots/{id}/download. Subtreeeidos:Snapshotfiles written before every mutation commit; tier-gated bysnapshot_service.compute_and_write(silent no-op below thedomain_snapshotcapability). Kept as the last section because it is a read-only recovery aid, not a user-triggered exchange.
The word "snapshot" is reserved for one meaning across the tab: the FMDE archive facet (mode=archive&as_of=…). The former "Export/Import snapshot" microcopy on the domain surface is now "Export/Import domain" (matches serialize_domain() and the .eidx.json file convention). "Auto-snapshots created before each mutation commit" is now "Mutation checkpoints" (describes what the file is — a pre-mutation safety net — rather than how it was implemented).
Surfaced capabilities¶
Two capabilities that were delivered on the server side but never reached the admin UI are now visible on the tab:
- FMDE historical
as_of— the Full Model section renders a date picker whosemin/maxare read from/admin/export/full-model/info.historyat load time. Picking a date and clicking "Download archive snapshot (.jsonld)" issuesGET /admin/export/full-model/download?mode=archive&as_of=YYYY-MM-DD— the same endpoint the eidosdash collector already consumes via service token. The server's existing error envelope (before_history_floor,invalid_as_of,reconstruction_failed) is surfaced verbatim. - Project Export options — an "Options" disclosure below the Export/Import buttons exposes two checkboxes that map to server-side query params the endpoint has always accepted:
include_configandinclude_provenance. Both default to unchecked, which produces a request byte-for-byte identical to the pre-#425 call (server defaults hold).
Filename handling¶
Every admin download prefers the server's Content-Disposition filename over any frontend-side guess. The three shared helpers in frontend/app.js are:
_buildFullModelDownloadUrl(baseUrl, {name, audit, mode, asOf})— backs both the "Download current" button (mode=view, the default) and the "Download archive snapshot" button (mode=archive); emits only the query params that were actually set, so the URL stays minimal._buildProjectExportUrl(baseUrl, {includeConfig, includeProvenance})— same shape; unchecked options are omitted, preserving the endpoint's default behaviour._filenameFromContentDisposition(cd, fallback)— pullsfilename="…"out of the header, or returns the caller's fallback. Used by every admin download path (Full Model current, Full Model archive, Domain export, Project export, Audit Log) so the server's date-stamped filenames survive the browser round-trip. This closes a long-standing inconsistency (G7 in the pre-implementation report) where the domain export handler used to override the server's{safe_domain}-{YYYY-MM-DD}.eidx.jsonwith${domain}-snapshot.eidx.json, both discarding the date-stamp and re-injecting the overloaded word "snapshot" into filenames.
Deep linking¶
Admin tabs are switched entirely inside switchAdminTab() — no location.hash, no location.search, no external URL points into a specific admin tab. The rename from ?snapshots|export to ?dataexchange therefore has no deep-link impact. (The canonical deep-link contract at docs/superpowers/specs/2026-07-27-canonical-deep-link-contract.md governs /node/{uuid} and is unrelated to admin surfaces.)
Archive Mode and IIS Criticality¶
When mode="archive" and interface_lookup is provided, build_export_doc stamps interface relations with IIS criticality before external-ref resolution:
# SOURCE: backend/persistence/export_service.py build_export_doc()
iface_preds = {p["id"] for p in catalog if p.get("creates_interface")}
for d in domains:
for rel in d.get("relations", []):
if rel.get("predicate") not in iface_preds:
continue
hit = interface_lookup(rel.get("from"), rel.get("to"), rel.get("predicate"))
if hit and hit.get("criticality_class") is not None and hit.get("status") is not None:
rel["eidos:iis"] = {
"criticality_class": hit["criticality_class"],
"status": hit["status"],
}
Only predicates with creates_interface: true in the relation catalog qualify. The IIS stamp (criticality_class, status) is injected directly onto the relation object, so downstream consumers compute IIS criticality without calling back to the live server. D1 invariant: engine results are materialized in the file; consumers apply no logic after export.