Files
pages/docs/design/projection-adapters.md
T
Anya Tokugawa c1175d389f Implement Knowledge City platform, organizer and DAV integration
Capture accumulated application, UI, CLI, regression tests and design documentation changes, including task project mapping and default CalDAV list naming.

Keep deployment data, backups, local credentials and build artifacts out of Git. Move the Compose enrollment token to an ignored local environment file.
2026-09-27 14:44:43 +09:00

20 KiB

Projection Adapter Design

Purpose

A document address identifies content. It does not select its renderer. Viewer chooses a projection adapter from document metadata and runtime capabilities:

internal address
  -> revision metadata
  -> effective MIME detection
  -> projection adapter selection
  -> bounded data delivery
  -> Viewer component

The adapter boundary must remain independent from Semantic Path, Page UUID, Revision UUID, and raw payload delivery.

Current Adapter Contract

The server registry in internal/server/content_adapter.go describes:

Field Meaning
ID Stable adapter identifier exposed as Viewer metadata
Kind Template component used by the current Viewer
Delivery HTML, decoded text, raw media, raw frame, or download
NeedsText Whether safe character decoding is required
WindowableText Whether large input must move to bounded retrieval
Match MIME and filename matching predicate

Selection is ordered. Specific structured formats must precede generic JSON, text, and download adapters.

The initial registry reserves these stable IDs:

Adapter Current projection Intended projection
html rendered HTML rendered HTML
text CodeMirror text windowed text
csv bounded table with sticky header paginated virtualized table
excalidraw validated interactive canvas + source fallback canvas + editor action
nwdiag bounded network diagram projection static SVG preview + hosted/editor action
archive ZIP/RAR/7z/TAR entry table + comic mode archive directory
eml decoded headers and body multipart mail viewer
debian-package ar/control metadata package metadata
pe PE/COFF headers and sections executable metadata
image raw media raw media
audio raw media raw media
video raw media raw media
pdf raw frame PDF viewer
download download download

These adapters are built in and versioned with Pages Wiki. They are always listed by GET /_/api/v1/adapter; they are not configuration records and cannot be deleted by an administrator.

Integration Adapter Plugins

Additional hosted viewers and editors are Domain-scoped integration adapter plugins. They may run at another FQDN or in a separately deployed container. They supplement the built-in projection and do not replace document identity, Raw delivery, or immutable Revision storage.

Each plugin declares:

Field Meaning
id Stable lowercase plugin identifier
name User-facing name in “Open with”
mode hosted viewer or hosted editor
media_types Exact MIME types or suffix-wildcards such as image/*
extensions Lowercase filename extensions
url_template Absolute HTTP(S) URL for the hosted service
enabled Whether it is offered at runtime

Templates can use {url}, {path}, and {filename}. Substitutions are URL-encoded. {url} is an immutable Revision Raw URL; it never carries a third-party token. Only http: and https: templates are accepted.

Native application launch is not implemented. vim.exe, code.exe, /usr/bin/vim, custom URL schemes, a local workspace cache, and a Box Edit style helper remain a future TODO. Legacy native records remain visible but are disabled and are never returned as runtime actions.

Management and runtime discovery use:

GET    /_/api/v1/adapter
POST   /_/api/v1/adapter
POST   /_/api/v1/adapter/delete
GET    /_/api/v1/adapter?path=/semantic/path
GET    /v/open-with?adapter=<id>&path=/semantic/path

The server revalidates MIME/extension compatibility when /v/open-with is called. A context-menu URL is therefore not an authorization bypass.

Hosted Deployment

A hosted adapter can be deployed beside Pages Wiki without being part of the Pages Wiki process:

pages.example.test       -> Pages Wiki
code.pages.example.test  -> code-server + Pages Wiki adapter gateway
vim.pages.example.test   -> vim.wasm adapter

The current integrated deployment uses one Pages Wiki origin and an internal Gateway service instead of publishing the Gateway port:

pages.anya.jp/_/adaptor/code/* -> adaptor-gateway:3000 -> code-server:8080
pages.anya.jp/_/adaptor/vim/*  -> adaptor-gateway:3000 -> vim.wasm assets/API

adaptor-gateway:3000 is available only on the Compose network. Pages Wiki owns the public /_/adaptor/ route, forwards the prefix to the Gateway, and retains the prefix in redirects and cookie paths.

The first implementation opens the configured service in a top-level browser navigation. Cross-origin iframe embedding is not assumed; a service may opt in later with an explicit CSP frame-ancestors policy.

vim.wasm or a purpose-built Web editor can consume the immutable {url} directly for read-only use. A general code-server container cannot edit an HTTP Raw URL as a local workspace by itself. It requires an adapter gateway that:

  1. exchanges a short-lived, one-use launch ticket;
  2. materializes the selected Revision inside a per-session workspace;
  3. opens that workspace in code-server;
  4. submits changed bytes as a new Revision with optimistic conflict checking;
  5. destroys or expires the workspace.

This launch and write-back contract is implemented by the reference adaptor-gateway container. An editor URL template must contain {ticket}; the ticket expires after 60 seconds and can be exchanged only once. The gateway stores the resulting eight-hour session bearer server-side and sends it in an Authorization header. Saving creates a new Revision. If the base Revision is no longer current, Pages Wiki returns HTTP 409 and preserves both versions without silently overwriting either one.

The reference gateway materializes the selected file and its descendant documents in an isolated session directory on a dedicated workspace volume and opens that directory through an internal code-server instance. code-server is not published on a host port and runs with authentication disabled only inside the Compose network; the Gateway requires its HttpOnly adapter session before proxying /code/, including WebSocket traffic. A watcher submits changed target bytes as a new Revision. Relative hierarchy and a separate optimistic base Revision are retained for every file. Hosted plugin settings can request a new browser tab and an HTML/Markdown round trip: HTML becomes Markdown in the workspace and is rendered back to canonical HTML when saved. vim.wasm integrates behind this same Gateway contract rather than receiving Pages Wiki database or Document Store access. The Gateway loads the session workspace into vim.wasm MEMFS. A BufWritePost hook exports the written file, and the browser sends those bytes through the scoped session API to create a new Revision.

vim.wasm, its Worker, Wasm binary, and data archive are served with Cross-Origin-Opener-Policy: same-origin, Cross-Origin-Embedder-Policy: require-corp, and Cross-Origin-Resource-Policy: same-origin. The deployment must also use a secure context. A trusted HTTPS adapter URL is preferred; loopback development may use http://localhost, which browsers treat as potentially trustworthy.

POST /_/api/v1/adapter/session/exchange
GET  /_/api/v1/adapter/session/workspace
GET  /_/api/v1/adapter/session/content
PUT  /_/api/v1/adapter/session/content

Recommended container boundaries are:

  • a dedicated origin and cookie namespace;
  • an allowlist of adapter origins in Pages Wiki;
  • no direct mount of the Pages Wiki Document Store or SQLite database;
  • CPU, memory, process, and workspace quotas per adapter container;
  • immutable Revision download through Pages Wiki APIs only;
  • write-back through the narrowly scoped adapter session capability only.
  • code-server terminal and extension privileges confined to its container;
  • only the adapter workspace volume shared with the Gateway.

File Context Menu

Directory, search, and file-manager projections expose the same contextual operations: open, rename, copy, paste, move, recursive delete, and “Open with”. Copy/paste uses a browser-local clipboard containing Semantic Paths; the server performs the actual Domain-scoped copy through POST /_/api/v1/path/copy.

A copied Page receives new Object and Revision UUIDs. Its immutable Document hashes are reused, and its Revision history and metadata are preserved. Copy therefore means an independent Page, not a new Path link to the source. “Open with” is shown only for adapters matching the resolved latest Revision.

Fallback rendering is intentional. Recognizing a format and providing an interactive renderer are separate capabilities.

PDF page counts and non-ZIP archive listings now use the persistent, bounded projection.metadata.generate cache. Viewer requests enqueue missing metadata and render explicit pending/retry states; decoder subprocesses execute only in the scheduler. Cached entry names remain plain data and receive authorized, immutable-Revision links at response time.

Metadata UI

Viewer must not reserve a full horizontal row for MIME information. Document metadata is exposed from the information control at the right of the breadcrumb bar. It contains:

  • filename
  • effective MIME type
  • selected adapter
  • payload size

The control must work with both pointer hover and keyboard focus.

Web Upload

/v/upload accepts a destination Semantic Path and one file. A destination ending in / appends the uploaded filename. An exact existing Path creates a new Revision of the existing Page; it never destroys history. Upload stores the original filename and detected MIME type as Revision metadata.

The request is bounded to 256 MiB. Specialized resumable upload is a separate future adapter and must not silently raise this synchronous request limit.

Media Interaction

Image projection opens a modal full-viewport lightbox from the inline preview. Audio and video projections retain native controls and add a common toolbar for elapsed time and playback rate. Video additionally exposes Picture-in-Picture when supported and full-screen projection. Raw media remains Range-capable.

ZIP entries are exposed through a scoped archive-entry projection. It resolves the archive by Domain and semantic Path and never accepts a host filesystem path. Comic mode includes recognized image entries and lazy-loads them in archive order.

The topbar search form sends the current semantic Path as its scope. SQLite searches effective Paths and latest Revision filenames inside that segment boundary and returns at most 100 results. The scope parameter is retained so a later Domain-wide FTS index can add “all” search without changing the topbar contract.

Children Macro Views

Macro processing has explicit modes:

Mode children behavior
Authoring HTML → Markdown Preserve the macro source verbatim
Markdown editor preview Render Markdown to HTML, then expand the macro
Saved Viewer projection Expand from canonical stored HTML
Markdown source projection Preserve authoring source

Expansion must never run before an authoring round trip: generated async projection markup is runtime output and must not replace the macro in the editor or a subsequent Revision.

The macro only selects the child Path. Presentation is viewer state rather than document content:

{{children}}
{{children:/gallery}}
{{children:/downloads exclude=/downloads/private}}

The breadcrumb action area exposes thumbnail, detail, and normal-list views. The selected view is stored locally and applies to every children projection on the page. Legacy view=, mode=, and compact arguments are ignored. exclude= remains part of the macro because it changes the projected set.

Ctrl/Command-click toggles selection without following the link. Right-clicking a selected item opens bulk move and recursive-delete actions. Moving a virtual folder renames its Path prefix, preserving Object and Revision identity.

The macro initially emits a lightweight placeholder. Viewer fetches the Domain-scoped /v/children projection asynchronously, so Page HTML and other macros are not blocked by metadata or thumbnail discovery.

Children are delivered in bounded pages. There is no explicit “More” action: an IntersectionObserver requests the next page when its sentinel approaches the viewport, including touch/drag scrolling. The item grid and labels are painted before any preview payload is requested.

Icon projection uses generated, immutable thumbnails for images, first-page PDF previews, video frames, and Excalidraw scenes. Folders, ZIP archives, Debian packages, audio, plain text, and source code use format-specific SVG icons. Cache checks are viewport-prioritized. Mutable path: raw projections use a short private cache; immutable rev: projections use a one-year immutable cache and ETag.

Preview I/O is a background workload. The browser:

  • paints item frames and labels before scheduling preview work;
  • starts work during an idle period;
  • allows at most two concurrent previews, or one on data-saver/2G links;
  • observes only items within 250 px of the viewport;
  • cancels queued fetches and active Excalidraw requests on page departure.

Preview Raw URLs carry preview=1. The server isolates those requests behind a single global preview slot. Normal Viewer navigation, APIs, and non-preview Raw delivery never wait for this slot.

File-manager thumbnails no longer use Raw PDF frames or browser video decoders. They consume only immutable files produced by the persistent event scheduler. Until generation succeeds, the icon projection remains a lightweight format icon.

Raw and Derived Projection URLs

Adapters never construct an alternative identity URL. Raw bytes continue to use an explicit internal address:

/v/raw/path:/reports/annual.pdf
/v/raw/uuid:<page-uuid>
/v/raw/rev:<revision-uuid>

Structured viewers use derived endpoints below /v/<adapter>/ and carry the same address forms. Proposed examples:

/v/text/path:/logs/server.log?offset=0&limit=65536
/v/csv/rev:<revision-uuid>?row=0&limit=200
/v/archive/rev:<revision-uuid>?path=/usr/bin
/v/deb/rev:<revision-uuid>
/v/excalidraw/rev:<revision-uuid>

Printable projections use the address-model route /_/view/print/{path:|uuid:|rev:} and pin every Raw dependency to the selected immutable Revision. Text and CSV print output is bounded to 2 MiB; media is printed from metadata plus its immutable Raw URL without hydrating its body. Structured formats above the synchronous hydration ceiling degrade to a download link instead of blocking foreground navigation.

A mutable path: request may resolve to a revision and return its UUID in the response. Paging requests after the first response must use rev: so a changing current document cannot mix two revisions in one Viewer session.

Large Text and JSON

The current inline text threshold is 2 MiB. Crossing the threshold is recorded as a requirement for windowed delivery. The complete scalable implementation must not hydrate or embed the full document merely to render its first screen.

The text adapter will use:

  1. metadata-only revision lookup;
  2. a seekable Document Store reader;
  3. byte-window retrieval with a fixed server maximum;
  4. UTF boundary repair and explicit byte offsets;
  5. line-index checkpoints built asynchronously or lazily;
  6. client virtualization for visible lines and minimap summaries.

JSON uses bounded structural streaming at /v/json/rev:<revision-uuid>. Expansion fetches a capped direct-child range selected by JSON Pointer. The request handler streams tokens and never unmarshals the complete payload; nested containers are revisited lazily from the immutable Document. A future persistent checkpoint index may accelerate deep pointers without changing the API or Revision identity.

Responses should include:

revision_uuid
document_hash
content_type
total_bytes
offset
next_offset
line_start
line_end
truncated

Range limits, parse depth, row count, archive entries, and decompressed bytes must be capped server-side.

CSV Viewer

CSV detection uses MIME type or .csv. The derived adapter must:

  • sniff delimiter and quoting from a bounded prefix;
  • expose encoding and dialect in metadata;
  • paginate rows;
  • virtualize columns and rows in the browser;
  • preserve the immutable Revision during paging;
  • fall back to the text adapter on malformed input.

nwdiag network diagrams

nwdiag is a source/document format, not an instruction to execute arbitrary code. The adapter parses a bounded, version-pinned grammar and produces a static SVG (or PNG) projection. It must reject oversized graphs, recursive includes, external image/font imports, unsafe SVG attributes, and unknown directives rather than forwarding them to a renderer. The source Revision is immutable and remains available through the plain/source view.

An optional hosted editor is a separate viewer/editor adapter. It receives only an immutable Revision URL and a one-use save ticket, never the browser session token. Save uses optimistic Revision checks and creates a new source Revision; rendering is queued and cached by (document hash, parser version, theme, renderer version). If the renderer or editor is unavailable, the Viewer shows escaped source and the last verified static projection. A future Kroki provider may render nwdiag, but network access, origin allowlists, timeout, resource quotas, and provider version must be explicit.

ZIP Viewer

ZIP projection lists entries without extracting them into the data directory. Entry paths must be normalized and zip-slip paths rejected. Limits apply to entry count, nested archives, compression ratio, total expanded size, and individual previews. Entry download is a derived projection of the containing Revision and entry path.

Debian Package Viewer

The DEB adapter reads the outer ar members and bounded control archive only. It exposes package name, version, architecture, dependencies, description, installed size, checksums, and a bounded file list. Maintainer scripts render as escaped text. No package script is executed.

Excalidraw

.excalidraw files are recognized before generic JSON. The renderer validates a bounded scene graph and produces an isolated canvas projection.

The preferred projection is the bundled org.pageswiki.excalidraw-viewer plugin. It runs @excalidraw/excalidraw in read-only view mode in a separate container and is routed through /_/adaptor/excalidraw/. Pages Wiki passes only an immutable same-origin rev: Raw URL. The plugin rejects mutable, cross-origin, and over-2-MiB inputs, is framed with a restrictive CSP, and reports readiness through postMessage. A timeout, load failure, or invalid scene activates the dependency-free built-in Canvas renderer.

Configuration may provide an editor URL template. The action receives a read-capable document URL and, only when explicitly authorized, a save callback or capability. Tokens must not be placed in a third-party URL query string. Without a configured editor, Viewer offers the internal canvas and source fallback.

Security and Failure Rules

  • Normal HTML displays use the common controlled-v2 allowlist and application CSP; saved source remains unchanged. Raw HTML attachment, raw/diagram SVG sandbox, and registered application frames retain their separate response contracts. There is no executable authored-HTML mode. See HTML projection trust and compatibility; public shares and anonymous viewing still require their own authorization gates.
  • Archive and package adapters never execute contained files.
  • Derived adapters operate on immutable revisions after initial resolution.
  • MIME detection is advisory; parser validation is authoritative.
  • An unavailable or failed specialized adapter falls back to safe text or download behavior.
  • Adapter errors do not change document identity or stored content.