Files
pages/docs/design/implementation-status.md

27 KiB

Implementation status and hardening backlog

2026-11-08 reconciliation: use the current Japanese implementation backlog for actionable IDs, priorities, dependencies and acceptance gates, based on e65ca56. This is a source/document audit, not a production audit or a new deployment.

For the consolidated current reading order, use the Japanese handbook and decision register. Dated entries below preserve history; older statements about missing anonymous navigation, blog publication or RSS are superseded by the newer slices, not additional unfinished implementations.

This document is a working handoff for the current repository state. It is based on the implementation-roadmap, architecture documents, and a static review of the handlers and stores. Checked roadmap items describe a usable first slice; they do not imply production-ready federation, HA, or extension security.

Current state

2026-11-08 digest follow-up: private delivery history and explicit same-day retry are implemented in source. Manual requests are queued for the existing scheduler; attempt IDs fence stale completions, and consent/version/authorization are checked again before SMTP. Detailed terminal attempts expire after 90 days, while daily reservations and request receipts remain. Additive schema v4 and recovery rules preserve duplicate prevention; production rollout remains deferred. See delivery history and migration.

2026-11-08 work history: owner-only bounded history, inline correction/cancellation, immutable audit snapshots and project-level reports are implemented in source. Cancelled spans leave the report but not the audit. Task/dashboard navigation and Japanese/English desktop/mobile UI are connected. Schema migration v3 requires an aligned DB backup before production rollout; do not hot-swap older binaries onto the upgraded DB. See contracts and remaining scope.

The dated task/inbox paragraphs below describe earlier slices. VTODO, time blocks, Pomodoro, active notifications, daily task email, private CardDAV and the organizer dashboard have since been implemented. Their remaining work is shared access, external connectors, conflict resolution and retention, not reimplementation of the existing private interfaces. The Japanese roadmap tracks the current order.

2026-09-25 task reminders: explicit per-task due/planned-date switches and a local notification time now schedule owner-only Inbox reminders. Reservations are updated in the source transaction; the bounded scheduler rechecks current state/authority, skips previous-day catch-up, and commits Inbox insertion plus acknowledgment together. Titles are projected under current authorization rather than copied into notices. Notification badges refresh on visible pages without reload. Calendar reminders, quiet hours, digests and external delivery remain pending. See contract.

2026-09-24 outbox follow-up: City now subscribes only to relevant page/revision/path/object events, with independent consumer checkpoints rather than shared processed flags. Per-type high-water marks avoid repeated full-history discovery. Task/project mutations append metadata-only events in their source transaction; a bounded scheduled bridge replays the earlier task journal idempotently. Notification delivery and retention are not included. See delivery contract.

2026-09-24 task first slice: private organizer.tasks items/projects, Today/Inbox/ Backlog/Review/Completed/Archive UI, conditional updates, idempotent create, immutable CAS revision payloads and UUID-based Wiki references are implemented. Ownership is distinct from Wiki ACLs; even Domain administrators cannot read another principal's tasks through these endpoints. The local atomic change journal is now connected to the common outbox, but notification delivery, time-block/calendar integration, Pomodoro, sharing and VTODO synchronization remain follow-ups. SQLite metadata and the Document Store must be backed up together. See contract and remaining work.

2026-09-19 outbox foundation: activity_outbox now has a persistent, bounded envelope with Domain-local monotonic sequence numbers, actor/target references, idempotency keys, retry state and pending-page retrieval. The append API is transaction-bound and deliberately does not authorize or project payloads. Page creation and Revision writes now append through this envelope in their source transactions. Path rename/delete and Object purge are also recorded; Domain-local consumer checkpoints are durable and monotonic. A debounced City consumer now reconciles pending URL events through the scheduler and advances its checkpoint only after a successful plan. Recipient-scoped notification Inbox storage and an authenticated list/read API are now implemented with idempotent deduplication and keyset pagination; approval request producers, reviewer UI and delivery workers remain pending, so this is not yet a completed activity pipeline.

Street/Spot automatic publication is opt-in through the governance dialog. Current and future eligible heads follow the pinned Path boundary with live audience and enabling-actor authority checks. Domain/Quartier/Street/Spot manual-only floors narrow it; anonymous navigation/blog projection/RSS/share readiness follow the same policy. Existing scopes stay unchanged. Reviewer/Inbox and policy federation remain pending; see automatic-publication.md.

Local diary/blog projection first slice: Campus admins select source Domain and prefix inside the entry dialog. Same-Campus/local placement only, no copied content; both Domains' read policies filter private metadata; editing stays on the source Domain. Target article writes are rejected. Anonymous projection indexes/calendars now derive only from approved source Paths/Revisions, never private registration metadata. Dedicated-domain links now open /YYYY/MM/DD[/keyword] on that host with sanitized published bodies, calendar/month/adjacent-article navigation, target Domain branding and /rss.xml (20 items). Registered titles now explicitly inherit Page publication visibility; category/owner/registration date stay private. No cross-instance transport, source configuration federation or automatic cross-Domain session sharing yet.

RSS now includes sanitized, absolute-URL HTML in content:encoded with bounded generation and exact publication rechecks. Authenticated /v/{blog|diary}/-/new pages reuse atomic composition. Post completion offers X sharing with configured projection-domain URLs, target selection and unpublished-link warnings; no automatic social posting, publication grants or credentials are added.

Domain entry provisioning is implemented in source: Campus-admin dialog per Domain, explicit allowlisted settings, signed manifest v2 when settings are present, and transactional apply together with Domain snapshot/checkpoint. Legacy manifests without entry settings remain v1. Destinations must be upgraded before activating v2 settings; signed enrollment capability proof and Campus-wide preflight are implemented. Create-form integration, apply acknowledgments/freshness and control-release/GC remain pending. Managed nodes cannot override entry settings locally. No new public grants.

Atomic diary/blog composition is now implemented in source: Markdown composer, create/write checks inside the writer transaction, Page/Path/Revision/article/journal/ receipt atomicity and idempotent retries. Shared Revision insertion and asset freezing are reused. Anonymous article-index approval, City/notification outbox dispatch, receipt retention and application acknowledgments remain pending. Existing two-step editor/registration is optional, not the only new-post workflow.

2026-09-17 URL/entry slice (source implementation): per-Domain diary/blog prefixes, date/keyword URL planner, same-day multiple diary entries, and home_layout diary/blog rendering at /. Placement label shortened; placement identity remains separate. The planner is read-only and does not reserve Paths. Editor save and registration remain separate in the legacy flow; public calendar approval is still TODO. Existing article Paths are not migrated.

Diary/blog initial slice: content.diary and content.blog are separate trusted built-ins with Domain enable/disable settings, Page UUID-bound metadata, calendar/recent entries/categories, registration/update/unregister dialogs and fresh read filters. Disabled by default; indices require authentication and registration does not publish a Page. Public index approval/feeds, comments and tDiary import remain explicitly deferred; see detailed design.

2026-09-16 anonymous navigation hardening: public root and virtual parents now offer safe directory landing pages, with read-only /v/browser and separate Page/children links. Trees support all/per-folder expand/collapse, name/Path filtering and keyboard navigation. Responsive public headers, direct City exits, bounded/abortable asynchronous listings, retry and no-JavaScript pagination are implemented. No additional publication or approval policy is enabled.

2026-09-16 navigation follow-up: approved public Pages now expose filtered children/tree/City navigation and breadcrumbs, without a download button. Database generations invalidate bounded metadata caches; authenticated filters remain request-scoped, including group revocation on unchanged cached data. Private parent bodies/titles are not exposed through structural folders. See navigation. Hierarchical approval requirements and the reviewer workflow remain designed and TODO, while the reusable recipient Inbox API is available but not populated by approval requests. Approval request subjects, decisions, generation fencing and bounded expiry sweeps are persisted, but reviewer eligibility and decision UI are not exposed; the current explicit approval workflow remains in force.

2026-09-14 user/group and anonymous follow-up: named local user/group read or read/write settings now intersect ancestor caps, resolve stable subject IDs and observe live membership. Explicit bounded current-revision approvals enable a separate inert anonymous semantic Viewer and raw-media path with revocation and OGP. Existing visibility is unchanged until configured. Important first-slice limits: authenticated ACLs freeze protected namespace/City structural changes; anonymous edits require reapproval, and discovery/dependencies remain private. See implementation contract and remaining tasks.

2026-09-14 locality governance: scoped Quartier/Spot co-owner appointments, versioned publication-cap drafts, transactional authority rechecks, local audit records and themed Japanese/English settings dialogs are implemented. Domain ceiling is in the City Map utility menu. The subsequent bounded anonymous slice and coarse namespace fences are described above; complete impact-aware mutation/dependency handling remains a next gate. See exact first-slice scope.

2026-09-14 P0 HTML remediation, extended 2026-09-23: normal Viewer/home/profile/object/history/print and both Editor previews use the controlled-v2 allowlist with application CSP. External resources remain default-deny; Domain administrators may approve exact HTTPS image URLs or SHA-256-pinned CSS/JavaScript URLs through the shared Projection/CSP policy. Canonical bytes are preserved; authored scripts/styles/frames are not activated. Diagram cache reads recheck Domain/Path authority and sandbox SVG. See contract, compatibility and tests. Ownership/public activation remains separate work, not implicitly enabled by this security fix.

2026-09-14 publication governance design: explicit Quartier/Spot owners may administer publication only within superior ceilings; appointment authority, cap-vs-publication semantics, mutation/alias fencing and the release sequence are documented in locality ownership. The owner API/UI and draft policies have subsequently been implemented as above; production visibility has not been changed.

2026-09-14 social-card follow-up: already-public semantic Pages, home layouts and City views now emit OGP/X metadata with bounded canonical-text summaries, safe absolute URLs and a common PNG cover (original images for image documents). Enforcing domains retain authentication and suppress content-card metadata; no visibility policy was changed. See contract and limits.

2026-09-14 City reference follow-up: parent-qualified readable Street URLs now replace ambiguous name-only links in map/search/macros. Details require ancestor read permission and fetch only the requested node; map focus and locality search work for Quartier, Street and Spot. Literal Page link encoding is shared by the map and its inspector. See read integrity.

2026-09-14 City/search options follow-up: metadata-only Quartier/Street/Spot matches appear ahead of Pages, with visible ancestor context and safe links. Search now supports file categories, one extension, and Domain-local calendar date ranges over creation/update times. Filters run before candidate limits; City and Page reads share the existing deadline. See search options and their exact semantics.

2026-09-14 search timeout follow-up: corrected a production Path-first repeated FTS scan and costly snippet generation. Input/time-bounded optional excerpts now preserve search results; query-plan and realistic-volume regressions plus read-only live-data timing cover the original blind spot. See search correction.

2026-09-14 hardening follow-up: authentication and inherited authorization, recursive mutations, federation snapshots and garbage collection now have bounded, denial/concurrency-focused regression coverage. City generation and PDF/archive metadata work are off the request/rendering path; search and history have bounded, clearer projections. See the execution ledger for validation/deployment and explicit deferrals. The subsequently corrected normal HTML boundary is recorded above; broader publication gates remain open.

2026-09-14 City UI follow-up: Spot clicks now explore their courtyards, with surrounding Page links, bounded nonoverlapping close-ups and a modal utility menu. Manual title/selection/merge/move controls remain available. See City projection and its remaining limits.

2026-09-13 follow-up: execution and resource panels are now one monitor with only running jobs. Completed jobs and lazy, paginated logs share a history panel. Active queue counts avoid scanning completed history; job queries use indexed status filters and keyset pagination. Legacy panel=resources resolves to the monitor. The first resumed hardening task protects group owners from ordinary update downgrades and checks last-owner removal within the DELETE itself (including concurrent removals and manager authority against the target's current role).

Current follow-up adds viewport-responsive bookmark columns, persistent background resource monitoring, and a separate trusted system-extension catalog. Other system services are designed and sequenced in System extensions; third-party plugin execution is not yet enabled. vim.wasm runtime work is paused at the user's request.

Latest UI follow-up: application asset authorization and /e/ target decoding are corrected; home bookmarks support admin management, section cards and bounded asynchronous child links. Hosted Vim now explicitly loads its vimrc, local font and pure-Vimscript navigation plugin. See implementation and remaining limits.

Viewer shortcut follow-up: e/f/v/c/u/s/p and the four Ctrl-arrow navigation bindings are implemented in static/js/viewer-shortcuts.js. They suppress themselves while typing or inside dialogs; sharing copies the current canonical URL and opens a user-confirmed X Intent. The X action is cataloged as the trusted builtin sharing.x extension; it does not grant publication or create a share credential. Permission shortcut behavior depends on the City governance context.

The repository currently has working first slices for:

  • Campus and Domain scoped semantic paths, local/remote placement, controller enrollment, signed manifest reconciliation, and the initial Knowledge City topology/map UI.
  • UUID/Object, immutable Revision, SHA-256 Document storage, language fallback, asset HEAD/history, bounded text and JSON projections, archive/CSV/EML/PDF adapters, and hosted code-server/vim.wasm/Excalidraw adapters.
  • Password/TOTP sessions, OIDC, LDAP/LDAPS, and SMTP magic-link flows, Domain capabilities, built-in campus/domain administrator groups, group permission profiles, and a centralized endpoint policy classifier.
  • Persistent event jobs with priority, retry, cancellation, concurrency settings, preview generation, import jobs, and manual scheduler controls.

The current deployment is a single writable SQLite application node plus a managed external-domain node. This is a supported development topology, not HA: canonical metadata, session revocation, scheduler claims, and adapter state are still process/instance local.

Confirmed corrections (2026-09-14)

The execution ledger records verification and deployment separately. Completed, regression-tested slices include:

  • OIDC issuer JWKS signatures, algorithm/claim checks, browser-bound state, bounded key/discovery fetches and account-binding collision rejection; verification contract.
  • SMTP connection/protocol deadlines, bounded replies, STARTTLS, mailbox/header validation, one-use challenges and live membership checks; delivery limits.
  • Campus-admin provenance and transactional derived-membership revocation while retaining independent local grants; inheritance contract.
  • Same-origin browser mutation protection, POST logout, typed browser/API credentials, atomic session-state checks and request-scoped batch Path reads; request boundaries.
  • Domain scope for Object reads/saves/binds, exact exceptional Object grants and atomic API creation provenance; Object scope.
  • Transactional complete-binding snapshots for recursive Path changes and all-alias Domain-local Object purges; mutation integrity.
  • Atomic controller manifest sequence allocation and managed snapshot, inherited-permission and checkpoint application; authority collision and unsigned-downgrade rejection; manifest integrity.
  • Private, per-result City map/details/macros/badges, recomputed visible portals/counts/relations, local generation Worker, cancellation/cache bounds, directional keyboard navigation and immersive courtyards; navigation contract.
  • Persisted scheduler projections for PDF page counts and non-ZIP archive metadata, localized pending/retry UI and immutable Revision entry links; projection cache.
  • Bounded compact search with full Paths, matching-source badges and safe excerpts/highlighting; search contract.
  • Bounded metadata-only Object/Revision APIs and paginated history, retaining array response shapes and explicit continuation; Object pagination, Revision pagination.
  • Literal URL encoding in all three children views, including filenames with question marks, fragments, percent signs and non-ASCII characters.
  • One-connection-safe City read transactions with deadlines/ceilings and literal boundary matching; read integrity.
  • Atomic bounded zombie selection/purge, current-Domain provenance for HTTP administration, retained global maintenance grace and orphan-writer guards; GC integrity.

These are not a claim that every endpoint, derived projection or write path has finished authorization and concurrency review.

2026-09-23: user-space audience controls

  • /u/{username}/_/edit now separates profile metadata, avatar, Wiki-root and descendant audiences. Descendant exact/subtree overrides can only narrow the default, are CAS-versioned/audited, and follow atomic Path rename.
  • A dedicated anonymous live-head reader supplies user profile, Viewer/raw, children, scoped Tree and Browser projections without adding /u/ to global anonymous navigation. It rechecks the Domain public ceiling per request and rejects aliases and UUID/Revision bypasses.
  • Authenticated listings and direct Object/Revision authorization also enforce private user-space audiences. Owner and Domain-admin recovery are explicit. Focused revocation, alias, navigation and recursive-rename tests pass. See user-space publication.

Remaining gates and risks

P0: authority and content trust

  1. Finish authorization-policy/version fencing inside every write transaction. Recursive binding snapshots protect namespace races, not a concurrent revocation of the grants themselves. Typed decoded mutation targets, handler-method closure and idempotency remain wider cross-handler work.
  2. Complete publication-impact review and mutation fencing on the implemented owner/draft-policy and inert HTML boundary. Keep public enabling gated on structural mutation/alias fences and common anonymous-read policy/revocation tests. See ownership design. Extend denial tests to each new projection; CSP is not an authorization grant.
  3. Local-email ownership, explicit external-account linking, provider subject migrations and administrator last-owner suspension/transfer need dedicated lifecycle workflows. Rejecting ambiguous email matches is not email proof.
  4. Domain-to-Domain PSK identity/group/grant inheritance needs its own origin, replay, conflict and revocation contract. The completed Campus-admin local projection is not that cross-instance synchronization feature.
  5. Audit remaining list/projection, asset and adapter paths with denial-focused tests before claiming complete Domain isolation. Legacy unknown-origin unbound Objects require explicit grants or trusted recovery. HTTP zombie GC is now provenance-scoped; trusted global maintenance remains a distinct instance authority rather than a Domain administrator operation.

P1: City correctness and responsiveness

See City review and reconciliation. CITY-A through CITY-D remain partially open. Remaining work includes shallow Path/portal inference, deterministic generated Spot name collisions, idempotent no-op application, explicit generated/manual ownership, exclusion tombstones, event watermarks and a transactional mutation outbox.

Worker generation and bounded courtyard DOM do not solve connected/fair road allocation or all main-thread topology/layout work. Lazy locality/Page projections and a directory index remain important for much larger Wikis. The Viewer sidebar / paginated navigation API is also separate from the already-implemented home layouts and scoped Tree View.

P1: projection and storage scaling

  1. Non-ZIP entry extraction still runs a bounded foreground decoder. Durable entry artifacts need quotas, leases, Range semantics and archive-bomb policy. ZIP central-directory parsing and nested/compressed TAR expansion need their own bounded worker/index contract.
  2. Native decoders have input/output/thread/time bounds, not hard memory/cgroup limits. A weighted scheduler resource budget and decoder isolation remain.
  3. Search FTS still contains historical revisions; current-only indexing, Japanese infix search and sanitized plain-text indexing are separate tasks. SQL deadlines/candidate bounds limit work, not unlimited recall.
  4. Browser/children/Tree and some link projections still aggregate large Path sets. Request-scoped ACL removes repeated grant queries but does not replace a directory index, bounded SQL candidates or fully opaque continuations.
  5. Large-binary restore and several trusted full-list helpers still hydrate or enumerate complete data. HTTP Object/Revision pagination does not silently change explicit CLI full exports.
  6. Remote Document Store backends, read-repair, retention proofs, cross-instance job leases, dead-letter operations and writable metadata fencing remain.

Operations and intentionally deferred high-risk work

  • Existing deployment is two application instances with separate SQLite state, not a shared-writer HA service. DR/LB require replication, fencing, key custody and recovery drills; no live SQLite-over-network-filesystem shortcut.
  • Enrollment tokens currently present in compose must be moved to a selected secret mechanism and rotated deliberately. No credentials were automatically replaced, printed or copied as part of these repairs.
  • Untrusted plugin execution/native helpers remain disabled/deferred until the runtime capability/signature/isolation model exists. vim.wasm runtime work remains on the user's explicit hold.
  • Live S3/SSHFS/WebDAV/MWDB/Karton/SNMP/LDAP integration requires selected endpoints, credentials and test services. No external service was provisioned implicitly.

Ordered next work

  1. Add complete publication-impact previews and structural mutation/alias fences to locality governance; keep anonymous access disabled until all publication gates and fresh activation confirmations are satisfied.
  2. Complete remaining mutation/policy transaction fences and account ownership lifecycle; preserve denial tests and audited provenance.
  3. Finish bounded/lazy City and directory projections; implement idempotent URL inference with generated/manual ownership before automatic event-driven application.
  4. Add the transactional Domain activity outbox, verified actor attribution, durable consumer watermarks and recent-edit projections.
  5. Then add delegated shared links, notifications/inbox, moderated comments, rankings/quality, diary calendar/recent entries/comments and resumable tDiary import. Their detailed contracts remain in System extensions.
  6. External connectors, isolated third-party plugins and DR/LB/HA follow only after their authority, retention and operations prerequisites are met.

Verification expectations

Every hardening slice must run go test ./..., include a regression test for the denied case, and update the relevant design document and roadmap checkbox. Deployment validation must confirm both app nodes are healthy and that a foreground request cannot expose another Domain's Object, Revision, Path, or adapter workspace.