Extract Grammaratur, Dregarnuhr, Geduldh, Mestionora, Ordoschnellic, Schutzaria, Kunstzeal and Seheweit behind compatible application contracts. Include accumulated editor resilience, navigation and theme UX, remote CLI, storage/schema and DR groundwork, tests and design/operations documentation. Verified: uncached full Go suite; go vet ./...; race tests for all eight components; 11 targeted Chromium tests; Node suites; 19 Python CLI tests; reproducible editor build. No push, release tag or deployment in this checkpoint.
477 lines
20 KiB
Markdown
477 lines
20 KiB
Markdown
# Projection Adapter Design
|
|
|
|
## Purpose
|
|
|
|
A document address identifies content. It does not select its renderer.
|
|
Viewer chooses a projection adapter from document metadata and runtime
|
|
capabilities:
|
|
|
|
```text
|
|
internal address
|
|
-> revision metadata
|
|
-> effective MIME detection
|
|
-> projection adapter selection
|
|
-> bounded data delivery
|
|
-> Viewer component
|
|
```
|
|
|
|
The adapter boundary must remain independent from Semantic Path, Page UUID,
|
|
Revision UUID, and raw payload delivery.
|
|
|
|
## Current Adapter Contract
|
|
|
|
The registry in `internal/kunstzeal/adapters.go`, accessed through compatible
|
|
server types and selection wrappers, describes:
|
|
|
|
| Field | Meaning |
|
|
| --- | --- |
|
|
| `ID` | Stable adapter identifier exposed as Viewer metadata |
|
|
| `Kind` | Template component used by the current Viewer |
|
|
| `Delivery` | HTML, decoded text, raw media, raw frame, or download |
|
|
| `NeedsText` | Whether safe character decoding is required |
|
|
| `WindowableText` | Whether large input must move to bounded retrieval |
|
|
| `Match` | MIME and filename matching predicate |
|
|
|
|
Selection is ordered. Specific structured formats must precede generic JSON,
|
|
text, and download adapters.
|
|
|
|
The initial registry reserves these stable IDs:
|
|
|
|
| Adapter | Current projection | Intended projection |
|
|
| --- | --- | --- |
|
|
| `html` | rendered HTML | rendered HTML |
|
|
| `text` | CodeMirror text | windowed text |
|
|
| `csv` | bounded table with sticky header | paginated virtualized table |
|
|
| `excalidraw` | validated interactive canvas + source fallback | canvas + editor action |
|
|
| `json` | bounded JSON structure/text projection | indexed structural retrieval |
|
|
| `archive` | ZIP/RAR/7z/TAR entry table + comic mode | archive directory |
|
|
| `eml` | decoded headers and body | multipart mail viewer |
|
|
| `debian-package` | ar/control metadata | package metadata |
|
|
| `pe` | PE/COFF headers and sections | executable metadata |
|
|
| `image` | raw media | raw media |
|
|
| `audio` | raw media | raw media |
|
|
| `video` | raw media | raw media |
|
|
| `pdf` | raw frame | PDF viewer |
|
|
| `download` | download | download |
|
|
|
|
These adapters are built in and versioned with Pages Wiki. They are always
|
|
listed by `GET /_/api/v1/adapter`; they are not configuration records and
|
|
cannot be deleted by an administrator.
|
|
|
|
2026-11-01: adapter selection and existing Kroki execution were initially
|
|
extracted into [Kunstzeal](ja/kunstzeal.md). Stable IDs/order and source authorization
|
|
remain unchanged. nwdiag and slide providers are still TODO, not entries in the
|
|
implemented registry. Kroki redirects are now refused; derived SVG cache writes
|
|
use atomic replacement. Provider versioning and unified resource controls remain pending.
|
|
|
|
## Integration Adapter Plugins
|
|
|
|
Additional hosted viewers and editors are Domain-scoped integration adapter
|
|
plugins. They may run at another FQDN or in a separately deployed container.
|
|
They supplement the built-in projection and do not replace document identity,
|
|
Raw delivery, or immutable Revision storage.
|
|
|
|
Each plugin declares:
|
|
|
|
| Field | Meaning |
|
|
| --- | --- |
|
|
| `id` | Stable lowercase plugin identifier |
|
|
| `name` | User-facing name in “Open with” |
|
|
| `mode` | hosted `viewer` or hosted `editor` |
|
|
| `media_types` | Exact MIME types or suffix-wildcards such as `image/*` |
|
|
| `extensions` | Lowercase filename extensions |
|
|
| `url_template` | Absolute HTTP(S) URL for the hosted service |
|
|
| `enabled` | Whether it is offered at runtime |
|
|
|
|
Templates can use `{url}`, `{path}`, and `{filename}`. Substitutions are
|
|
URL-encoded. `{url}` is an immutable Revision Raw URL; it never carries a
|
|
third-party token. Only `http:` and `https:` templates are accepted.
|
|
|
|
Native application launch is not implemented. `vim.exe`, `code.exe`,
|
|
`/usr/bin/vim`, custom URL schemes, a local workspace cache, and a Box Edit
|
|
style helper remain a future TODO. Legacy `native` records remain visible but
|
|
are disabled and are never returned as runtime actions.
|
|
|
|
Management and runtime discovery use:
|
|
|
|
```text
|
|
GET /_/api/v1/adapter
|
|
POST /_/api/v1/adapter
|
|
POST /_/api/v1/adapter/delete
|
|
GET /_/api/v1/adapter?path=/semantic/path
|
|
GET /v/open-with?adapter=<id>&path=/semantic/path
|
|
```
|
|
|
|
The server revalidates MIME/extension compatibility when `/v/open-with` is
|
|
called. A context-menu URL is therefore not an authorization bypass.
|
|
|
|
### Hosted Deployment
|
|
|
|
A hosted adapter can be deployed beside Pages Wiki without being part of the
|
|
Pages Wiki process:
|
|
|
|
```text
|
|
pages.example.test -> Pages Wiki
|
|
code.pages.example.test -> code-server + Pages Wiki adapter gateway
|
|
vim.pages.example.test -> vim.wasm adapter
|
|
```
|
|
|
|
The current integrated deployment uses one Pages Wiki origin and an internal
|
|
Gateway service instead of publishing the Gateway port:
|
|
|
|
```text
|
|
pages.anya.jp/_/adaptor/code/* -> adaptor-gateway:3000 -> code-server:8080
|
|
pages.anya.jp/_/adaptor/vim/* -> adaptor-gateway:3000 -> vim.wasm assets/API
|
|
```
|
|
|
|
`adaptor-gateway:3000` is available only on the Compose network. Pages Wiki
|
|
owns the public `/_/adaptor/` route, forwards the prefix to the Gateway, and
|
|
retains the prefix in redirects and cookie paths.
|
|
|
|
The first implementation opens the configured service in a top-level browser
|
|
navigation. Cross-origin iframe embedding is not assumed; a service may opt in
|
|
later with an explicit CSP `frame-ancestors` policy.
|
|
|
|
`vim.wasm` or a purpose-built Web editor can consume the immutable `{url}`
|
|
directly for read-only use. A general code-server container cannot edit an HTTP
|
|
Raw URL as a local workspace by itself. It requires an adapter gateway that:
|
|
|
|
1. exchanges a short-lived, one-use launch ticket;
|
|
2. materializes the selected Revision inside a per-session workspace;
|
|
3. opens that workspace in code-server;
|
|
4. submits changed bytes as a new Revision with optimistic conflict checking;
|
|
5. destroys or expires the workspace.
|
|
|
|
This launch and write-back contract is implemented by the reference
|
|
`adaptor-gateway` container. An editor URL template must contain `{ticket}`;
|
|
the ticket expires after 60 seconds and can be exchanged only once. The
|
|
gateway stores the resulting eight-hour session bearer server-side and sends
|
|
it in an Authorization header. Saving creates a new Revision. If the base
|
|
Revision is no longer current, Pages Wiki returns HTTP 409 and preserves both
|
|
versions without silently overwriting either one.
|
|
|
|
The reference gateway materializes the selected file and its descendant
|
|
documents in an isolated session directory
|
|
on a dedicated workspace volume and opens that directory through an internal
|
|
code-server instance. code-server is not published on a host port and runs
|
|
with authentication disabled only inside the Compose network; the Gateway
|
|
requires its HttpOnly adapter session before proxying `/code/`, including
|
|
WebSocket traffic. A watcher submits changed target bytes as a new Revision.
|
|
Relative hierarchy and a separate optimistic base Revision are retained for
|
|
every file. Hosted plugin settings can request a new browser tab and an
|
|
HTML/Markdown round trip: HTML becomes Markdown in the workspace and is
|
|
rendered back to canonical HTML when saved.
|
|
vim.wasm integrates behind this same Gateway contract rather than receiving
|
|
Pages Wiki database or Document Store access. The Gateway loads the session
|
|
workspace into vim.wasm MEMFS. A `BufWritePost` hook exports the written file,
|
|
and the browser sends those bytes through the scoped session API to create a
|
|
new Revision.
|
|
|
|
vim.wasm, its Worker, Wasm binary, and data archive are served with
|
|
`Cross-Origin-Opener-Policy: same-origin`,
|
|
`Cross-Origin-Embedder-Policy: require-corp`, and
|
|
`Cross-Origin-Resource-Policy: same-origin`. The deployment must also use a
|
|
secure context. A trusted HTTPS adapter URL is preferred; loopback development
|
|
may use `http://localhost`, which browsers treat as potentially trustworthy.
|
|
|
|
```text
|
|
POST /_/api/v1/adapter/session/exchange
|
|
GET /_/api/v1/adapter/session/workspace
|
|
GET /_/api/v1/adapter/session/content
|
|
PUT /_/api/v1/adapter/session/content
|
|
```
|
|
|
|
Recommended container boundaries are:
|
|
|
|
- a dedicated origin and cookie namespace;
|
|
- an allowlist of adapter origins in Pages Wiki;
|
|
- no direct mount of the Pages Wiki Document Store or SQLite database;
|
|
- CPU, memory, process, and workspace quotas per adapter container;
|
|
- immutable Revision download through Pages Wiki APIs only;
|
|
- write-back through the narrowly scoped adapter session capability only.
|
|
- code-server terminal and extension privileges confined to its container;
|
|
- only the adapter workspace volume shared with the Gateway.
|
|
|
|
## File Context Menu
|
|
|
|
Directory, search, and file-manager projections expose the same contextual
|
|
operations: open, rename, copy, paste, move, recursive delete, and “Open
|
|
with”. Copy/paste uses a browser-local clipboard containing Semantic Paths;
|
|
the server performs the actual Domain-scoped copy through
|
|
`POST /_/api/v1/path/copy`.
|
|
|
|
A copied Page receives new Object and Revision UUIDs. Its immutable Document
|
|
hashes are reused, and its Revision history and metadata are preserved. Copy
|
|
therefore means an independent Page, not a new Path link to the source.
|
|
“Open with” is shown only for adapters matching the resolved latest Revision.
|
|
|
|
Fallback rendering is intentional. Recognizing a format and providing an
|
|
interactive renderer are separate capabilities.
|
|
|
|
PDF page counts and non-ZIP archive listings now use the persistent, bounded
|
|
[`projection.metadata.generate` cache](projection-metadata-cache.md). Viewer
|
|
requests enqueue missing metadata and render explicit pending/retry states;
|
|
decoder subprocesses execute only in the scheduler. Cached entry names remain
|
|
plain data and receive authorized, immutable-Revision links at response time.
|
|
|
|
## Metadata UI
|
|
|
|
Viewer must not reserve a full horizontal row for MIME information.
|
|
Document metadata is exposed from the information control at the right of the
|
|
breadcrumb bar. It contains:
|
|
|
|
- filename
|
|
- effective MIME type
|
|
- selected adapter
|
|
- payload size
|
|
|
|
The control must work with both pointer hover and keyboard focus.
|
|
|
|
## Web Upload
|
|
|
|
`/v/upload` accepts a destination Semantic Path and one file. A destination
|
|
ending in `/` appends the uploaded filename. An exact existing Path creates a
|
|
new Revision of the existing Page; it never destroys history. Upload stores the
|
|
original filename and detected MIME type as Revision metadata.
|
|
|
|
The request is bounded to 256 MiB. Specialized resumable upload is a separate
|
|
future adapter and must not silently raise this synchronous request limit.
|
|
|
|
## Media Interaction
|
|
|
|
Image projection opens a modal full-viewport lightbox from the inline preview.
|
|
Audio and video projections retain native controls and add a common toolbar for
|
|
elapsed time and playback rate. Video additionally exposes Picture-in-Picture
|
|
when supported and full-screen projection. Raw media remains Range-capable.
|
|
|
|
ZIP entries are exposed through a scoped archive-entry projection. It resolves
|
|
the archive by Domain and semantic Path and never accepts a host filesystem
|
|
path. Comic mode includes recognized image entries and lazy-loads them in
|
|
archive order.
|
|
|
|
## Folder Search
|
|
|
|
The topbar search form sends the current semantic Path as its scope. SQLite
|
|
searches effective Paths and latest Revision filenames inside that segment
|
|
boundary and returns at most 100 results. The scope parameter is retained so a
|
|
later Domain-wide FTS index can add “all” search without changing the topbar
|
|
contract.
|
|
|
|
## Children Macro Views
|
|
|
|
Macro processing has explicit modes:
|
|
|
|
| Mode | `children` behavior |
|
|
| --- | --- |
|
|
| Authoring HTML → Markdown | Preserve the macro source verbatim |
|
|
| Markdown editor preview | Render Markdown to HTML, then expand the macro |
|
|
| Saved Viewer projection | Expand from canonical stored HTML |
|
|
| Markdown source projection | Preserve authoring source |
|
|
|
|
Expansion must never run before an authoring round trip: generated async
|
|
projection markup is runtime output and must not replace the macro in the
|
|
editor or a subsequent Revision.
|
|
|
|
The macro only selects the child Path. Presentation is viewer state rather
|
|
than document content:
|
|
|
|
```text
|
|
{{children}}
|
|
{{children:/gallery}}
|
|
{{children:/downloads exclude=/downloads/private}}
|
|
```
|
|
|
|
The breadcrumb action area exposes thumbnail, detail, and normal-list views.
|
|
The selected view is stored locally and applies to every `children` projection
|
|
on the page. Legacy `view=`, `mode=`, and `compact` arguments are ignored.
|
|
`exclude=` remains part of the macro because it changes the projected set.
|
|
|
|
Ctrl/Command-click toggles selection without following the link. Right-clicking
|
|
a selected item opens bulk move and recursive-delete actions. Moving a virtual
|
|
folder renames its Path prefix, preserving Object and Revision identity.
|
|
|
|
The macro initially emits a lightweight placeholder. Viewer fetches the
|
|
Domain-scoped `/v/children` projection asynchronously, so Page HTML and other
|
|
macros are not blocked by metadata or thumbnail discovery.
|
|
|
|
Children are delivered in bounded pages. There is no explicit “More” action:
|
|
an IntersectionObserver requests the next page when its sentinel approaches
|
|
the viewport, including touch/drag scrolling. The item grid and labels are
|
|
painted before any preview payload is requested.
|
|
|
|
Icon projection uses generated, immutable thumbnails for images, first-page
|
|
PDF previews, video frames, and Excalidraw scenes. Folders, ZIP archives,
|
|
Debian packages, audio, plain text, and source code use format-specific SVG
|
|
icons. Cache checks are viewport-prioritized. Mutable `path:` raw projections
|
|
use a short private cache; immutable `rev:` projections use a one-year
|
|
immutable cache and ETag.
|
|
|
|
Preview I/O is a background workload. The browser:
|
|
|
|
- paints item frames and labels before scheduling preview work;
|
|
- starts work during an idle period;
|
|
- allows at most two concurrent previews, or one on data-saver/2G links;
|
|
- observes only items within 250 px of the viewport;
|
|
- cancels queued fetches and active Excalidraw requests on page departure.
|
|
|
|
Preview Raw URLs carry `preview=1`. The server isolates those requests behind a
|
|
single global preview slot. Normal Viewer navigation, APIs, and non-preview Raw
|
|
delivery never wait for this slot.
|
|
|
|
File-manager thumbnails no longer use Raw PDF frames or browser video
|
|
decoders. They consume only immutable files produced by the persistent
|
|
[event scheduler](event-scheduler.md). Until generation succeeds, the icon
|
|
projection remains a lightweight format icon.
|
|
|
|
## Raw and Derived Projection URLs
|
|
|
|
Adapters never construct an alternative identity URL. Raw bytes continue to
|
|
use an explicit internal address:
|
|
|
|
```text
|
|
/v/raw/path:/reports/annual.pdf
|
|
/v/raw/uuid:<page-uuid>
|
|
/v/raw/rev:<revision-uuid>
|
|
```
|
|
|
|
Structured viewers use derived endpoints below `/v/<adapter>/` and carry the
|
|
same address forms. Proposed examples:
|
|
|
|
```text
|
|
/v/text/path:/logs/server.log?offset=0&limit=65536
|
|
/v/csv/rev:<revision-uuid>?row=0&limit=200
|
|
/v/archive/rev:<revision-uuid>?path=/usr/bin
|
|
/v/deb/rev:<revision-uuid>
|
|
/v/excalidraw/rev:<revision-uuid>
|
|
```
|
|
|
|
Printable projections use the address-model route
|
|
`/_/view/print/{path:|uuid:|rev:}` and pin every Raw dependency to the selected
|
|
immutable Revision. Text and CSV print output is bounded to 2 MiB; media is
|
|
printed from metadata plus its immutable Raw URL without hydrating its body.
|
|
Structured formats above the synchronous hydration ceiling degrade to a
|
|
download link instead of blocking foreground navigation.
|
|
|
|
A mutable `path:` request may resolve to a revision and return its UUID in the
|
|
response. Paging requests after the first response must use `rev:` so a
|
|
changing current document cannot mix two revisions in one Viewer session.
|
|
|
|
## Large Text and JSON
|
|
|
|
The current inline text threshold is 2 MiB. Crossing the threshold is recorded
|
|
as a requirement for windowed delivery. The complete scalable implementation
|
|
must not hydrate or embed the full document merely to render its first screen.
|
|
|
|
The text adapter will use:
|
|
|
|
1. metadata-only revision lookup;
|
|
2. a seekable Document Store reader;
|
|
3. byte-window retrieval with a fixed server maximum;
|
|
4. UTF boundary repair and explicit byte offsets;
|
|
5. line-index checkpoints built asynchronously or lazily;
|
|
6. client virtualization for visible lines and minimap summaries.
|
|
|
|
JSON uses bounded structural streaming at `/v/json/rev:<revision-uuid>`.
|
|
Expansion fetches a capped direct-child range selected by JSON Pointer. The
|
|
request handler streams tokens and never unmarshals the complete payload;
|
|
nested containers are revisited lazily from the immutable Document. A future
|
|
persistent checkpoint index may accelerate deep pointers without changing the
|
|
API or Revision identity.
|
|
|
|
Responses should include:
|
|
|
|
```text
|
|
revision_uuid
|
|
document_hash
|
|
content_type
|
|
total_bytes
|
|
offset
|
|
next_offset
|
|
line_start
|
|
line_end
|
|
truncated
|
|
```
|
|
|
|
Range limits, parse depth, row count, archive entries, and decompressed bytes
|
|
must be capped server-side.
|
|
|
|
## CSV Viewer
|
|
|
|
CSV detection uses MIME type or `.csv`. The derived adapter must:
|
|
|
|
- sniff delimiter and quoting from a bounded prefix;
|
|
- expose encoding and dialect in metadata;
|
|
- paginate rows;
|
|
- virtualize columns and rows in the browser;
|
|
- preserve the immutable Revision during paging;
|
|
- fall back to the text adapter on malformed input.
|
|
|
|
## nwdiag network diagrams
|
|
|
|
Status: design/TODO. No nwdiag parser or built-in adapter is currently registered.
|
|
|
|
`nwdiag` is a source/document format, not an instruction to execute arbitrary
|
|
code. The adapter parses a bounded, version-pinned grammar and produces a
|
|
static SVG (or PNG) projection. It must reject oversized graphs, recursive
|
|
includes, external image/font imports, unsafe SVG attributes, and unknown
|
|
directives rather than forwarding them to a renderer. The source Revision is
|
|
immutable and remains available through the plain/source view.
|
|
|
|
An optional hosted editor is a separate `viewer`/`editor` adapter. It receives
|
|
only an immutable Revision URL and a one-use save ticket, never the browser
|
|
session token. Save uses optimistic Revision checks and creates a new source
|
|
Revision; rendering is queued and cached by `(document hash, parser version,
|
|
theme, renderer version)`. If the renderer or editor is unavailable, the Viewer
|
|
shows escaped source and the last verified static projection. A future Kroki
|
|
provider may render nwdiag, but network access, origin allowlists, timeout,
|
|
resource quotas, and provider version must be explicit.
|
|
|
|
## ZIP Viewer
|
|
|
|
ZIP projection lists entries without extracting them into the data directory.
|
|
Entry paths must be normalized and zip-slip paths rejected. Limits apply to
|
|
entry count, nested archives, compression ratio, total expanded size, and
|
|
individual previews. Entry download is a derived projection of the containing
|
|
Revision and entry path.
|
|
|
|
## Debian Package Viewer
|
|
|
|
The DEB adapter reads the outer `ar` members and bounded control archive only.
|
|
It exposes package name, version, architecture, dependencies, description,
|
|
installed size, checksums, and a bounded file list. Maintainer scripts render
|
|
as escaped text. No package script is executed.
|
|
|
|
## Excalidraw
|
|
|
|
`.excalidraw` files are recognized before generic JSON. The renderer validates
|
|
a bounded scene graph and produces an isolated canvas projection.
|
|
|
|
The preferred projection is the bundled `org.pageswiki.excalidraw-viewer`
|
|
plugin. It runs `@excalidraw/excalidraw` in read-only view mode in a separate
|
|
container and is routed through `/_/adaptor/excalidraw/`. Pages Wiki passes
|
|
only an immutable same-origin `rev:` Raw URL. The plugin rejects mutable,
|
|
cross-origin, and over-2-MiB inputs, is framed with a restrictive CSP, and
|
|
reports readiness through `postMessage`. A timeout, load failure, or invalid
|
|
scene activates the dependency-free built-in Canvas renderer.
|
|
|
|
Configuration may provide an editor URL template. The action receives a
|
|
read-capable document URL and, only when explicitly authorized, a save callback
|
|
or capability. Tokens must not be placed in a third-party URL query string.
|
|
Without a configured editor, Viewer offers the internal canvas and source
|
|
fallback.
|
|
|
|
## Security and Failure Rules
|
|
|
|
- Normal HTML displays use the common `controlled-v2` allowlist and application CSP;
|
|
saved source remains unchanged. Raw HTML attachment, raw/diagram SVG sandbox,
|
|
and registered application frames retain their separate response contracts.
|
|
There is no executable authored-HTML mode. See
|
|
[HTML projection trust and compatibility](html-projection-trust.md); public
|
|
shares and anonymous viewing still require their own authorization gates.
|
|
- Archive and package adapters never execute contained files.
|
|
- Derived adapters operate on immutable revisions after initial resolution.
|
|
- MIME detection is advisory; parser validation is authoritative.
|
|
- An unavailable or failed specialized adapter falls back to safe text or
|
|
download behavior.
|
|
- Adapter errors do not change document identity or stored content.
|