Files
pages/docs/design/html-projection-trust.md
Anya Tokugawa c1175d389f Implement Knowledge City platform, organizer and DAV integration
Capture accumulated application, UI, CLI, regression tests and design documentation changes, including task project mapping and default CalDAV list naming.

Keep deployment data, backups, local credentials and build artifacts out of Git. Move the Compose enrollment token to an ignored local environment file.
2026-09-27 14:44:43 +09:00

318 lines
20 KiB
Markdown

# HTML projection trust boundary
Audited and remediated: 2026-09-14; controlled external-resource extension:
2026-09-23. Status: **normal HTML projections use `controlled-v2`**. Authored
active HTML remains denied by default. A Domain administrator may separately
approve an exact external resource URL; approved JavaScript is intentionally
executable and is therefore a high-trust Domain configuration action.
## Implemented contract — controlled-v2
`internal/server/html_projection.go` applies a parser-based allowlist using
[bluemonday](https://github.com/microcosm-cc/bluemonday), pinned to v1.0.27. The
sequence is bounded token preflight → authored allowlist → tree normalization
and authorized macro expansion → diagram references → final allowlist. This
changes display projections only, not saved Documents, hashes or Revisions.
- All known authored-HTML surfaces share this boundary: semantic Viewer,
configured home, profile Wiki, object UUID, historical Revision, Preview,
print, and HTML/Markdown editor preview. Object viewing checks adapter MIME
kind; non-HTML is escaped text or a download link, never blind HTML.
- Formatting, tables, code, internal/external links and local passive images
are supported. Author scripts, event attributes, forms, iframes, SVG/MathML,
styles, arbitrary classes and `data-*` hooks are removed. This is intentional
compatibility tightening, not a source migration or an executable trust flag.
- By default images must resolve to local `/v/raw/`, `/_/obj/`, `/static/`, or
`/o/path:/system/static/` resources after path normalization. External images,
relative semantic URLs, data images and arbitrary authenticated GET targets
are not loaded by authored markup. Use immutable asset/raw references.
- The Domain setting “Approved external resources” is the only exception.
External images require an exact normalized HTTPS URL match. External CSS
and JavaScript require both an exact HTTPS URL and pinned SHA-256 SRI. The
server discards authored integrity/crossorigin attributes and emits canonical
`integrity` and `crossorigin=anonymous` attributes from configuration.
Approval entries are converted to unpredictable text placeholders before the
authored and generated allowlists, then restored as server-authored tags;
macro output therefore cannot forge an approval. CSP is derived from the same
list: image/style origins and JavaScript content hashes are added while
`connect-src` remains same-origin. Revocation takes effect on the next render.
Wildcards, HTTP, credentials, fragments, inline script/style and event
attributes remain unsupported. Because approved JavaScript can invoke
same-origin APIs as the viewer, only immutable audited code should be added.
- Authored IDs and fragment targets are SHA-256-namespaced to avoid application
DOM collisions. Only code-language classes survive the authored pass.
- Only text-node macros expand, not macros in attributes, links or code.
Application-owned children hooks cannot be forged by authored `data-*`.
Each macro is dispatched once; lookup failures remain escaped authored text,
including entity-decoded tags and nested-macro-looking strings.
Existing read filtering is retained; City reads inherit request cancellation
and use bounded Store projections. Macro output is not cached across readers.
- Input is at most 1 MiB; token/tree budgets are 16,384 with depth 128; output
is at most 4 MiB before final sanitization; at most 32 macro invocations.
Preflight is deliberately conservative for deeply/unclosed malformed HTML.
Cooperative request/DB work has a two-second deadline. These are bounds, not
hard CPU isolation of the in-process parser. Failure shows a localized warning
and at most 64 KiB of escaped source, never original active HTML.
- `/_/api/v1/render/markdown` accepts optional `html` only with `projection:true`.
Display results are private/no-store. Canonical Markdown-to-HTML conversion
remains separate and preserves authoring semantics; it must not be displayed
without projection. Editor preview requests debounce/abort obsolete work and
fail to escaped text. Saving does not save the sanitized preview.
- Normal display/editor templates use per-response CSP nonces for application
scripts and prohibit authored inline event handlers. Nonces are inserted into
trusted template source **before** request data, never rendered body strings.
Same-origin application frames/resources remain available; document styles
are removed while application/theme styles remain enabled. Config and other
non-document screens are not claimed to have this CSP yet.
- Diagram SVG responses, including cache hits, have sandbox CSP and nosniff.
GET cache use requires current Domain/Path read authority and a live bounded
source reference (128 entries, 32 KiB/source, 30-minute lifetime). These
references are keyed by policy version, Domain, Path and source; responses
are private/no-store so browser caches do not bypass fresh checks. Reopening
the Viewer registers references after restart/eviction. Renderer requests
inherit cancellation; source/cache/output reads are size-bounded. Durable
diagram scheduling, disk quotas, atomic artifact publication and hard decoder
isolation remain follow-up work; foreground diagram GETs are not eliminated.
- There is no grandfathering of old active HTML and no document-level trusted
executable mode. The explicit Domain approval list above is the only active
external-resource exception.
Registered application/plugin frames are separate trust contracts. Anonymous
locality ACLs, ownership appointments, alias/move fences and public cache
revocation must land before enabling new public access.
Verification: `html_projection_test.go`, `html_projection_browser_test.go`, and
`diagram_projection_security_test.go` cover inert attack corpora, resource limits,
cancelled requests, unchanged stored content, cache authority/revocation and CSP.
Real Chromium loads nine display/editor surfaces as an administrator after a
path-scoped writer saves a hostile test document through the real save API.
Authored execution and privileged sentinel requests must stay absent; parser-
inserted non-nonced scripts/events are also blocked by CSP. Existing PDF/archive/
comic, metadata, bookmark and editor browser regressions remain required.
Read-only compatibility sampling of the main DB's latest 500 HTML **revisions**
(not a complete current-Path inventory) found no script/event/iframe/SVG markers
in their first 128 KiB, one style-bearing revision, and no body over 1 MiB.
This limited marker count is not a sanitizer/security proof or a guarantee that
no existing document needs layout changes. No production document was rewritten.
The following sections preserve the original diagnostic and design rationale;
their unsafe behavior describes the **pre-remediation** implementation.
## Finding and release gate
Before remediation, saved HTML executed JavaScript in the Pages Wiki application origin
when a reader opens its semantic Viewer. Document write permission therefore
implicitly grants the ability to execute code as a later reader, including a
reader with stronger permissions. Authentication, path ACLs, and CSRF checks do
not make authored HTML safe after it is inserted into the authenticated page.
HttpOnly cookies cannot be read directly, but same-origin authored code can
issue authenticated requests and read accessible responses or the page DOM.
**Before enabling or broadening shared links, comments, anonymous/public
viewing, or untrusted-author publication, this boundary must be closed and its
browser regression suite must pass.** This is also an existing risk in private
Domains with authors and readers of different privilege levels; it is not
limited to future public features. Do not describe the pre-remediation Viewer as an
isolated HTML viewer, or treat an HTML MIME classification as a trust decision.
## Observed entry paths (before remediation)
The central semantic path is:
`Revision.Body → expandMacrosForRequest → renderDiagramsInHTML → template.HTML → parent page DOM`
The macro and diagram helpers replace recognized constructs; they are not HTML
sanitizers and preserve unrelated scripts and event attributes. Go's
`template.HTML` deliberately bypasses normal template escaping.
| Entry / projection | Current implementation and insertion |
| --- | --- |
| `/{semantic-path}` | `internal/server/semantic_handlers.go`, `handleSemanticRoute`: HTML-kind `Revision.Body` is expanded and cast to `template.HTML`; `templates/pages/page.gohtml` inserts `.RenderedHTML` directly into `.page-html`. |
| `/` with a configured HTML home Path | `internal/server/handlers.go`, `handleHome`: the saved home body is macro-expanded and inserted through `page.gohtml`. |
| `/o/uuid:{object-uuid}` | `internal/server/object_uri_handlers.go`, `handleObjectView`: casts `latest.Body` directly to `template.HTML`, without even the MIME/adapter-kind guard; `object_view.gohtml` inserts it into the parent document. |
| `/r/view/{revision-uuid}` | `internal/server/revision_routes.go`, `handleRevisionView`: HTML-kind historical body is cast directly and inserted by `revision_view.gohtml`. |
| `/v/preview/{path}` | `internal/server/preview_handler.go`, `handlePreview` / `renderPreviewDocument`: HTML-kind body is expanded and inserted by `preview.gohtml`. |
| `/_/view/print/{path:…\|uuid:…\|rev:…}` | `internal/server/print_handler.go`, `handlePrint`: HTML body is expanded and inserted by `print.gohtml`. |
| `/u/{username}` | `internal/server/profile_handler.go`, `handleUserProfile`: the associated user Wiki HTML is expanded and inserted by `profile.gohtml`; child Wiki pages use the semantic handler. |
| `/e/{path}` live Preview | `static/js/editor.js`: HTML source and Markdown projection responses are assigned to `preview.innerHTML`. This is another active-markup sink to cover; script-tag behavior under `innerHTML` differs from initial document parsing, but event attributes and other active content must not be considered safe. |
The first row was verified in a real browser. The remaining rows were traced in
source and require endpoint-specific browser coverage during remediation. This
is an inventory of known sinks, not proof that no additional projection exists.
### Protections that do not cover these paths
Raw HTML download/attachment handling in `internal/server/raw_view.go`, the CSP
sandbox on raw SVG, and the sandboxed EML HTML frame are separate response
paths. They do not constrain saved HTML inserted into the normal Viewer.
The audited semantic response had no `Content-Security-Policy` header and its
authored content was not in a sandboxed iframe. `nosniff` or a correct
`text/html` Content-Type would not remove executable content.
## Local verification
A scratch Go test, injected with `go test -overlay` rather than added to the
repository, created a temporary migrated DB and temporary Document root. It
saved a test-only HTML Revision, bound `/audit/sentinel`, and served the real
semantic handler and templates through a localhost `httptest.Server`. A test
owner session supplied request authorization; no external instance was used.
Headless Chromium loaded that local page.
The payload contained only a paragraph, an inline script setting
`document.documentElement.dataset.htmlScriptAudit`, and an invalid `data:` image
whose `onerror` set `dataset.htmlEventAudit`. Both sentinel values became
`executed` in the parent DOM. The HTTP response still contained the original
inline script and its CSP header was empty. There was no data extraction,
credential access, or application mutation from the payload.
This diagnostic confirmed the current unsafe behavior; it is not a permanent
test that expects the vulnerability to remain. Remediation must invert those
expectations into denial tests.
## Mismatch with existing design
- [Architecture](arch.md) distinguishes immutable Document content from Revision
identity. Canonical HTML support does not itself authorize executable
JavaScript, and the reviewed architecture does not define all authors as
trusted application-script authors.
- [Projection adapters](projection-adapters.md), “Security and Failure Rules,”
says stored HTML and SVG retain their existing isolation rules. In practice
the raw/SVG isolation is not shared by semantic HTML projections. That wording
was insufficient to establish the missing boundary. That document now
records the implemented common projection boundary and its limits.
- [Platform extension architecture](platform-extension-architecture.md)
explicitly says document content may request a widget provider/resource but
may not supply executable JavaScript; providers use a sandboxed frame or a
sanitized server projection. Direct authored HTML execution bypasses that
intended separation.
- The Preview adapter-kind check prevents treating every binary/text file as
HTML, but it does not make a legitimately classified HTML body trusted.
## Policy rationale: immutable source, controlled projection
### Normal HTML is inert
Keep canonical Document bytes, hashes, Revision IDs, history, and authoring
round trips unchanged. Enforce safety when producing a display projection,
using a parser-based allowlist rather than regular-expression replacements.
The normal mode must not allow authored scripts, inline event handlers,
executable URLs, active embedded documents, or attributes that create a second
execution context. Define URL schemes, relative URL resolution, allowed markup,
CSS/layout limits, and external resource/privacy behavior explicitly. SVG,
MathML, `srcdoc`, forms, CSS URLs, and DOM-clobbering attributes need explicit
decisions, not implicit passthrough.
Introduce one projection boundary shared by Viewer, home, profile, historical
Revision, preview, print, and client-side editor preview. Object-UUID viewing
must also use the MIME/adapter registry rather than blindly casting a body.
If projection parsing fails or exceeds resource bounds, fail safely to escaped
source/download; never fall back to unsanitized HTML.
Distinguish authored nodes from server-generated macro/widget output. Parse
recognized macro requests into structured inputs, authorize their data access,
and produce escaped, typed output through trusted renderer contracts. A string
that resembles generated markup is not proof of trust. Avoid sanitizing only
before an expansion that can reintroduce authored unsafe attributes, or only
after expansion without accounting for required application-owned controls.
No general template call site should be able to turn arbitrary `Revision.Body`
into trusted markup by convention alone.
Editor authoring mode continues to preserve source, while its visual preview
uses the same safe projection policy. Do not conflate a safe display result
with a replacement for the canonical body when saving.
### Explicit trusted HTML is isolated
If executable HTML compatibility is genuinely required, make it a separate,
administrator-granted capability and an explicit Domain policy, not an implied
consequence of page write permission. Bind approval to the actual approved
content/revision or to a clearly defined trusted publishing workflow: granting
trust to a mutable Path must not let any later lower-privilege editor inherit
script execution. Define revocation, audit records, and visible mode indicators.
Executable content must not run with application-origin authority. Consider a
dedicated uncredentialed origin or an opaque-origin sandbox, with the minimal
permissions needed for that use case. Never combine same-origin document
content with permissive `allow-scripts allow-same-origin` and call it isolation.
Do not forward application cookies, bearer tokens, unrestricted save callbacks,
or privileged data into that context. Any necessary `postMessage` integration
must be narrowly typed, sender-checked, capability-scoped, and bounded; an
opaque-origin sender requires validating the known frame/window and a dedicated
channel rather than trusting the string `null` as an origin.
Existing application/plugin scripts remain a separate administrator-controlled
surface. Registered widget/plugin frames must not become an escape hatch for
arbitrary page-authored code or inherited reader capabilities. Shared/public
projections and comments must always use the safe policy unless a separately
reviewed, isolated publication contract explicitly permits more.
### Defense in depth and scaling
Adopt an application CSP compatible with approved application scripts, using
nonces/hashes or external script assets rather than unrestricted inline script
execution. CSP is defense in depth, not a substitute for safe markup and
isolation. Frame responses need their own policy; tightening only the parent
page does not fully define each child document's privileges.
Bound input size, parse depth/nodes, output size, CPU time, and any external
resource work. Cache content-only safe projections by immutable Document hash,
renderer version, and safety-policy version. Keep ACL-dependent macro output
separate or key/revalidate it with Domain, principal/grants, and request context;
never reuse a privileged projection as an anonymous/shared response. Cache and
preview artifacts generated under an older unsafe policy need invalidation.
## Compatibility impact and staged implementation
Existing pages may rely on inline scripts, handlers, style blocks, custom
attributes, embedded frames, forms, or SVG. Blanket filtering can break those
pages, diagrams, child-list controls, and editor previews. A restrictive CSP
also affects inline application scripts. The implementation above deliberately
separates these template-owned scripts from authored HTML; original source stays
available for editing/download if custom layout needs adjustment.
1. Approve the default allowlist, external-resource policy, and whether a
trusted executable mode is needed at all; record explicit ownership.
2. Inventory active constructs with a read-only diagnostic and report affected
Paths only to authorized administrators. Do not rewrite stored content.
3. Build the common projection boundary and server/client regression fixtures;
preserve supported macro, diagram, print, and editor behavior deliberately.
4. Integrate every known route, safe error fallbacks, policy-versioned caches,
CSP, and any separately reviewed isolated mode. Do not grandfather all old
HTML as trusted or infer trust from MIME, filename, creation time, or author
display name.
5. Verify upgrades and rollback without changing canonical bytes. Only then
remove the P0 gate for public sharing, comments, and anonymous expansion.
## Required tests and acceptance criteria
- Browser denial fixtures for parser-inserted scripts, event attributes,
executable URL variants, SVG/MathML active content, iframe/srcdoc, malformed
markup, DOM clobbering, and CSS/resource-policy violations. Use harmless DOM
sentinels and local test endpoints, not credential-exfiltration payloads.
- Apply the same corpus to semantic Path, configured home, object UUID,
historical Revision, user Wiki, preview, print, and both HTML/Markdown editor
previews. Test direct navigation, client-side replacement, and restored
revisions, not just the first render of new content.
- Cross-account fixtures: a lower-privilege author cannot execute in a reader's
page or issue reader-authorized mutations. Anonymous/shared viewers and
comments cannot activate markup or receive private macro results.
- Canonical Document bytes, SHA-256, Revision IDs, downloads, and authoring
round trips remain unchanged. Non-HTML bodies are never interpreted as HTML
by the object-UUID route. Unknown formats and parser failures stay inert.
- Supported formatting, links, images, children macros, diagrams, code blocks,
accessibility, themes, and print layouts still work. Template-owned controls
are not confused with forged authored markup.
- If trusted mode is implemented, verify origin isolation, cookie/token
exclusion, navigation/form/download permissions, message validation,
approval-on-edit behavior, revocation, and public-projection restrictions.
- Test CSP headers and violation behavior, resource bounds, cancellation,
oversized/deep input, cache policy-version changes, ACL revocation, and no
privileged-to-public cache reuse. A single endpoint-only sanitizer test is
insufficient to close this gate.
Acceptance requires inactive authored markup in every normal projection, an
explicitly reviewed policy for any executable mode, preserved canonical
content, and passing endpoint/browser integration tests. The normal projection
slice meets this boundary with no executable mode. Additional public/shared
surfaces must inherit it and pass their own ACL/revocation tests before release.