72 Commits
Author SHA1 Message Date
Anya Tokugawa 4049acf76e fix(tasks): recover deleted captures without reviving tombstones
Build and publish container images / images (push) Successful in 12m0s
Release container images / release (push) Successful in 19s
2026-11-08 19:26:50 +09:00
Anya Tokugawa 43a2d8c5a1 docs: record v0.14.0 production deployment [skip ci] 2026-11-08 18:21:06 +09:00
Anya Tokugawa 38a4fcf530 feat(tasks): add explicit deletion with DAV tombstones
Build and publish container images / images (push) Successful in 11m42s
Release container images / release (push) Successful in 27s
2026-11-08 18:04:54 +09:00
Anya Tokugawa dc978231e2 docs: record v0.13.0 production deployment [skip ci] 2026-11-08 16:32:53 +09:00
Anya Tokugawa 5321d613b4 feat(feeds): add opt-in Gecko relay and folder shortcuts
Build and publish container images / images (push) Successful in 11m45s
Release container images / release (push) Successful in 21s
Document engine redistribution requirements; exclude upstream engine assets and keep the viewer disabled by default.
2026-11-08 16:14:34 +09:00
Anya Tokugawa b23e63d4c6 docs: record v0.12.0 production release [skip ci] 2026-11-08 14:57:15 +09:00
Anya Tokugawa f4869c7c8b feat(feeds): add optional sandboxed in-tab article preview
Build and publish container images / images (push) Successful in 10m41s
Release container images / release (push) Successful in 24s
2026-11-08 13:55:22 +09:00
Anya Tokugawa ea1ca432c5 feat(feeds): share original articles through X extension
Build and publish container images / images (push) Successful in 11m18s
2026-11-08 13:50:24 +09:00
Anya Tokugawa c2bc637704 fix(feeds): use opaque themed capture dialog surface
Build and publish container images / images (push) Successful in 11m27s
2026-11-08 13:46:46 +09:00
Anya Tokugawa a5f90f933d feat(feeds): add templated capture paths and inline modal
Build and publish container images / images (push) Successful in 11m2s
2026-11-08 13:44:20 +09:00
Anya Tokugawa 78da33d549 feat(feeds): prefill Wiki capture path from personal defaults
Build and publish container images / images (push) Successful in 11m41s
2026-11-08 13:39:54 +09:00
Anya Tokugawa 50a90bcc14 fix(feeds): dismiss shortcut help without leaving article
Build and publish container images / images (push) Successful in 11m20s
2026-11-08 13:37:56 +09:00
Anya Tokugawa bf9a9c2847 feat(feeds): inherit FreshRSS folders in cached subscription tree
Build and publish container images / images (push) Successful in 10m33s
2026-11-08 13:34:22 +09:00
Anya Tokugawa 1057be6d2b feat(feeds): expose retained operation history and read-only rechecks
Build and publish container images / images (push) Successful in 11m14s
2026-11-08 06:35:53 +09:00
Anya Tokugawa e4c184d71c docs: record approved backup and image retention cleanup [skip ci] 2026-11-08 06:35:53 +09:00
Anya Tokugawa 4711a0efbf docs: record RSS release and verified production rollout [skip ci] 2026-11-08 06:23:36 +09:00
Anya Tokugawa 61ecd1ffbd ci: isolate temporary test databases from runner overlay I/O
Build and publish container images / images (push) Successful in 10m42s
2026-11-08 06:08:09 +09:00
Anya Tokugawa 131a7ca01f feat(feeds): queue explicit FreshRSS state changes with uncertain-outcome recovery
Build and publish container images / images (push) Failing after 12m40s
2026-11-08 05:51:47 +09:00
Anya Tokugawa 16e572397b ci: serialize test packages to reduce SQLite initialization contention
Build and publish container images / images (push) Successful in 13m10s
Release container images / release (push) Successful in 25s
2026-11-08 05:42:30 +09:00
Anya Tokugawa bfed3d20f5 feat(feeds): cache FreshRSS subscriptions and category navigation
Build and publish container images / images (push) Failing after 6m7s
2026-11-08 04:48:55 +09:00
Anya Tokugawa 7fe64113c7 feat: add previewed atomic OPML subscription import and export
Build and publish container images / images (push) Failing after 7m31s
2026-11-08 04:39:54 +09:00
Anya Tokugawa 38daea737d feat: render RSS HTML bodies and simplify reader navigation
Build and publish container images / images (push) Successful in 12m32s
2026-11-08 04:00:27 +09:00
Anya Tokugawa 0d8997972c feat: add private RSS reader and read-only FreshRSS integration
Build and publish container images / images (push) Failing after 6m19s
2026-11-08 03:35:02 +09:00
Anya Tokugawa 0c2cf263b6 docs: define embedded and FreshRSS source authority [skip ci] 2026-11-08 02:48:14 +09:00
Anya Tokugawa 269f835c3a docs: prioritize feature subprojects starting with RSS [skip ci] 2026-11-08 02:45:25 +09:00
Anya Tokugawa b9e775de13 docs: reconcile implementation backlog and priorities [skip ci] 2026-11-08 02:40:55 +09:00
Anya Tokugawa e65ca56f92 feat: add audited work records and controlled digest retries
Build and publish container images / images (push) Successful in 12m27s
2026-11-08 02:35:29 +09:00
Anya Tokugawa 6e0aecdfb3 docs: record wallpaper palette v0.10.0 deployment [skip ci] 2026-11-07 23:14:51 +09:00
Anya Tokugawa 0964fdc5b7 feat: adapt home appearance to wallpaper colors
Build and publish container images / images (push) Successful in 10m56s
Release container images / release (push) Successful in 18s
2026-11-07 22:51:56 +09:00
Anya Tokugawa c7aa4fd7dd docs: record monthly mail layout release and deployment [skip ci] 2026-11-06 10:07:41 +09:00
Anya Tokugawa 1a451023f0 feat: support mailing-list and monthly mail capture paths
Build and publish container images / images (push) Successful in 11m59s
Release container images / release (push) Successful in 22s
2026-11-06 09:51:45 +09:00
Anya Tokugawa 543814cbd6 docs: record Japanese mail subject fix and v0.8.1 deployment [skip ci] 2026-11-06 09:18:02 +09:00
Anya Tokugawa 3262f88005 fix: use subject-independent identities for captured mail Wiki paths
Build and publish container images / images (push) Successful in 11m59s
Release container images / release (push) Successful in 25s
2026-11-06 09:02:34 +09:00
Anya Tokugawa 0b6343aff0 docs: record mail Wiki release and production deployment [skip ci] 2026-11-06 08:49:38 +09:00
Anya Tokugawa 80d297dab4 feat: atomically capture task mail into Wiki and persist destination defaults
Build and publish container images / images (push) Successful in 11m26s
Release container images / release (push) Successful in 20s
2026-11-06 08:34:29 +09:00
Anya Tokugawa 52b501d1ce docs: record pages-cli v0.2.0 TLS option release [skip ci] 2026-11-06 07:48:29 +09:00
Anya Tokugawa 7187af42ed docs: record CLI runner release and move NeoMutt guidance [skip ci] 2026-11-06 07:21:58 +09:00
Anya Tokugawa 8733645747 docs: record public CLI repository and runner prerequisite [skip ci] 2026-11-06 07:09:53 +09:00
Anya Tokugawa 9861b19f7b docs: record v0.7.1 deployment and CLI distribution preparation [skip ci] 2026-11-06 02:25:02 +09:00
Anya Tokugawa fdb411a896 Document v0.7.1 release artifacts and compatibility [skip ci] 2026-11-04 10:25:49 +09:00
Anya Tokugawa a6bf3dfe41 Refine task editor layout and personal reminder defaults
Build and publish container images / images (push) Successful in 11m12s
Release container images / release (push) Successful in 17s
2026-11-04 10:12:25 +09:00
Anya Tokugawa 8fd0b6ec8b Record v0.7.0 organizer production deployment [skip ci] 2026-11-04 00:24:46 +09:00
Anya Tokugawa 6ba30bc9f4 Unify organizer navigation and add personal widget dashboard
Build and publish container images / images (push) Successful in 12m38s
Release container images / release (push) Successful in 23s
2026-11-04 00:05:16 +09:00
Anya Tokugawa a9249009f0 docs: record v0.6.0 CardDAV rollout and backup retention [skip ci] 2026-11-02 09:06:59 +09:00
Anya Tokugawa 19e12f96a8 Add private CardDAV synchronization and contact revision recovery
Build and publish container images / images (push) Successful in 13m39s
Release container images / release (push) Successful in 18s
2026-11-02 08:41:15 +09:00
Anya Tokugawa 433169c7e7 docs: record v0.5.0 productivity and contacts rollout [skip ci] 2026-11-01 23:18:14 +09:00
Anya Tokugawa 8988e5a2f8 Apply task defaults to inline subtask capture
Build and publish container images / images (push) Successful in 16m25s
Release container images / release (push) Successful in 24s
2026-11-01 22:32:55 +09:00
Anya Tokugawa b4792bb175 Add task capture, work reports, daily digests and private contacts
Build and publish container images / images (push) Successful in 16m37s
2026-11-01 22:28:16 +09:00
Anya Tokugawa 50b8b0323f docs: record v0.4.0 CLI editing rollout [skip ci] 2026-11-01 21:50:31 +09:00
Anya Tokugawa 93f4ecb8f4 Enable CLI language-aware editing and reuse OIDC authentication
Build and publish container images / images (push) Successful in 14m31s
Release container images / release (push) Successful in 27s
2026-11-01 21:31:04 +09:00
Anya Tokugawa 5821254535 docs: record v0.3.0 CLI OIDC rollout and verification [skip ci] 2026-11-01 20:55:25 +09:00
Anya Tokugawa 60f5fb1881 Add OIDC browser approval and PKCE session handoff for pages CLI
Build and publish container images / images (push) Successful in 14m39s
Release container images / release (push) Successful in 25s
2026-11-01 20:26:45 +09:00
Anya Tokugawa 2af87b4f58 docs: record v0.2.1 logging hotfix rollout [skip ci] 2026-11-01 20:03:38 +09:00
Anya Tokugawa a36934e72c Fix event log writer contention and report persistence failures safely
Build and publish container images / images (push) Successful in 16m17s
Release container images / release (push) Successful in 39s
Take the SQLite writer lock before reading job/policy facts, bound acquisition and cancellation, preserve atomic raw logs and counters, and avoid replaying successful business operations on logging failure. Tested the isolated nine-file release slice with the full Go suite, race tests and go vet.
2026-11-01 19:40:26 +09:00
Anya Tokugawa 0f686aa2fb docs: record v0.2.0 production rollout and logging follow-up [skip ci] 2026-11-01 15:42:12 +09:00
Anya Tokugawa d5d21b7841 Refactor architecture into eight components and consolidate reliability work
Build and publish container images / images (push) Successful in 9m34s
Release container images / release (push) Successful in 36s
Extract Grammaratur, Dregarnuhr, Geduldh, Mestionora, Ordoschnellic, Schutzaria, Kunstzeal and Seheweit behind compatible application contracts.

Include accumulated editor resilience, navigation and theme UX, remote CLI, storage/schema and DR groundwork, tests and design/operations documentation.

Verified: uncached full Go suite; go vet ./...; race tests for all eight components; 11 targeted Chromium tests; Node suites; 19 Python CLI tests; reproducible editor build. No push, release tag or deployment in this checkpoint.
2026-11-01 15:23:31 +09:00
Anya Tokugawa 9dd425f765 Fix UUID generation in browsers without crypto.randomUUID
Build and publish container images / images (push) Successful in 7m55s
Release container images / release (push) Successful in 18s
2026-09-29 12:39:58 +09:00
Anya Tokugawa b312a50d37 Adopt SemVer 2.0.0 and promote release images by digest
Build and publish container images / images (push) Successful in 10m38s
Release container images / release (push) Successful in 24s
2026-09-27 19:52:58 +09:00
Anya Tokugawa 2893685aef Simplify image CI and pin the job runtime
Build and publish container images / images (push) Successful in 11m4s
2026-09-27 18:11:44 +09:00
Anya Tokugawa d8bfae99a7 Remove curl dependency from CI connectivity checks
Test and publish container images / images (push) Failing after 1s
Use verified Node HTTPS with bounded timeouts and separate tool, DNS, TLS and HTTP diagnostics. Cover the preflight with regression tests and document explicit Docker job labels for Runner 3.
2026-09-27 17:50:15 +09:00
Anya Tokugawa c3bea8a3b7 Document runner host mapping and check Gitea connectivity
Test and publish container images / images (push) Failing after 0s
2026-09-27 17:34:35 +09:00
Anya Tokugawa 02883e404b Add Gitea Actions image build and registry publication
Test and publish container images / images (push) Failing after 0s
Test Go packages in Docker, build and inspect three application images, then publish commit-specific tags. Document dedicated runner and package credentials, and provide an opt-in registry Compose overlay without automatic deployment.
2026-09-27 15:00:42 +09:00
anya 68f7d7a571 Merge pull request 'merge mk_district' (#2) from mk_district into main
Reviewed-on: anya/wikipages#2
2026-09-27 14:49:50 +09:00
Anya Tokugawa c1175d389f Implement Knowledge City platform, organizer and DAV integration
Capture accumulated application, UI, CLI, regression tests and design documentation changes, including task project mapping and default CalDAV list naming.

Keep deployment data, backups, local credentials and build artifacts out of Git. Move the Compose enrollment token to an ignored local environment file.
2026-09-27 14:44:43 +09:00
Anya Tokugawa 288e6e94ac Localize config page labels 2026-07-05 00:06:33 +09:00
Anya Tokugawa b37427535c Add district settings management 2026-07-05 00:01:24 +09:00
Anya Tokugawa 8a2ac56844 rename space to district 2026-07-04 23:19:18 +09:00
Anya Tokugawa 94d56f9a8f impl. Kroki, compile path, and code-editor feature 2026-06-18 00:06:16 +09:00
Anya Tokugawa c1bc8f0c03 add theme: Terminal Green, Amber CRT, Lain dark, Nanoha dark 2026-06-01 00:36:15 +09:00
Anya Tokugawa 97970cf105 fix tree view css 2026-06-01 00:27:32 +09:00
Anya Tokugawa 5a596985e0 Add Timezone, Zombie GC. Improved and select theme. 2026-06-01 00:13:30 +09:00
Anya Tokugawa ea3eba180b Improved breadcrumbs list 2026-05-31 23:39:12 +09:00
1032 changed files with 188815 additions and 3150 deletions

No files matched your search

+18
View File
@@ -0,0 +1,18 @@
.git
.agents
.codex
.deployment-backups
data
data2
assets/editor/node_modules
assets/city-generator/node_modules
assets/excalidraw-viewer/node_modules
assets/excalidraw-viewer/dist
assets/vimwasm/node_modules
pageswiki
*.db
*.token
.env
.env.*
user.txt
adapter-gateway
+3
View File
@@ -0,0 +1,3 @@
# Copy to .env and set the controller-issued enrollment token for app2.
# .env is local-only and must not be committed.
PAGESWIKI_EXTERNAL_CAMPUS_TOKEN=
+113
View File
@@ -0,0 +1,113 @@
name: Build and publish container images
on:
push:
branches: [main, mk_district]
workflow_dispatch:
jobs:
images:
runs-on: ${{ vars.IMAGE_RUNNER_LABEL || 'ubuntu-latest' }}
# Verified linux/amd64 job tools. The runner label only selects a runner.
container:
image: docker.gitea.com/runner-images:ubuntu-latest@sha256:fd911d7417bfbf0f454530e447da95b58001e1df41bbc5e1a8dd35d432575aae
env:
REGISTRY: ${{ vars.IMAGE_REGISTRY || 'git.anya.jp' }}
IMAGE_OWNER: ${{ vars.IMAGE_OWNER || 'anya' }}
IMAGE_TAG: sha-${{ github.sha }}
# The pinned image stores Node in its tool cache. Make it available to
# both shell steps and JavaScript Actions without login-shell setup.
PATH: /opt/acttoolcache/node/24.19.0/x64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
DOCKER_BUILD_SUMMARY: 'false'
DOCKER_BUILD_RECORD_UPLOAD: 'false'
steps:
- name: Checkout
uses: https://github.com/actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ github.sha }}
persist-credentials: false
- name: Set up Buildx
uses: https://github.com/docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
with:
version: v0.34.0
buildkitd-config-inline: |
[worker.oci]
max-parallelism = 2
- name: Test release tooling and wallpaper palette
run: node --test tools/ci/release-images.test.mjs static/js/home-palette-core.test.cjs
- name: Test
uses: https://github.com/docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
file: Dockerfile
target: test
platforms: linux/amd64
push: false
- name: Log in to registry
uses: https://github.com/docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1 # v3.5.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Build PagesWiki
uses: https://github.com/docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
file: Dockerfile
target: runtime
platforms: linux/amd64
load: true
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_OWNER }}/pageswiki-app:${{ env.IMAGE_TAG }}
labels: |
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
- name: Build adaptor gateway
uses: https://github.com/docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
file: Dockerfile.adapter-gateway
platforms: linux/amd64
load: true
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_OWNER }}/pageswiki-adaptor-gateway:${{ env.IMAGE_TAG }}
labels: |
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
- name: Build Excalidraw viewer
uses: https://github.com/docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
file: Dockerfile.excalidraw-viewer
platforms: linux/amd64
load: true
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_OWNER }}/pageswiki-excalidraw-viewer:${{ env.IMAGE_TAG }}
labels: |
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
# Publish only after all builds succeed. Use the host daemon for registry
# access; BuildKit does not inherit the runner's /etc/hosts entries.
- name: Push images
shell: bash
run: |
set -euo pipefail
for image in pageswiki-app pageswiki-adaptor-gateway pageswiki-excalidraw-viewer; do
docker push "$REGISTRY/$IMAGE_OWNER/$image:$IMAGE_TAG"
done
- name: Remove local CI image tags
if: always()
shell: bash
run: |
for image in pageswiki-app pageswiki-adaptor-gateway pageswiki-excalidraw-viewer; do
ref="$REGISTRY/$IMAGE_OWNER/$image:$IMAGE_TAG"
if docker image inspect "$ref" >/dev/null 2>&1; then
docker image rm "$ref" || echo "Could not remove local CI tag: $ref" >&2
fi
done
+46
View File
@@ -0,0 +1,46 @@
name: Release container images
on:
push:
tags: ['v*']
# Retry against the same tag if a registry operation was interrupted.
workflow_dispatch:
jobs:
release:
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ${{ vars.IMAGE_RUNNER_LABEL || 'ubuntu-latest' }}
container:
image: docker.gitea.com/runner-images:ubuntu-latest@sha256:fd911d7417bfbf0f454530e447da95b58001e1df41bbc5e1a8dd35d432575aae
env:
REGISTRY: ${{ vars.IMAGE_REGISTRY || 'git.anya.jp' }}
IMAGE_OWNER: ${{ vars.IMAGE_OWNER || 'anya' }}
RELEASE_REF: ${{ github.ref }}
PATH: /opt/acttoolcache/node/24.19.0/x64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
DOCKER_BUILD_SUMMARY: 'false'
DOCKER_BUILD_RECORD_UPLOAD: 'false'
steps:
- name: Checkout release tag
uses: https://github.com/actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ github.ref }}
persist-credentials: false
- name: Validate release version
run: node tools/ci/release-images.mjs --validate
- name: Set up Buildx tools
uses: https://github.com/docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
with:
version: v0.34.0
driver: docker
- name: Log in to registry
uses: https://github.com/docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1 # v3.5.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Promote existing images without rebuilding
run: node tools/ci/release-images.mjs
+23 -1
View File
@@ -1,3 +1,25 @@
data/
data/*
!data/pageswiki-cli
!data/test_cli_assets.py
!data/pageswiki_cli_test.py
!data/test_cli_dr.py
!data/pageswiki-remote
/data2/
/.codex/
/.agents/
/.env
/.env.*
!/.env.example
/user.txt
/pageswiki
/adapter-gateway
**/node_modules/
**/__pycache__/
*.pyc
.tmp/
.deployment-backups/
*.sh
assets/editor/node_modules/
assets/city-generator/node_modules/
assets/excalidraw-viewer/node_modules/
assets/excalidraw-viewer/dist/
+13 -2
View File
@@ -1,14 +1,25 @@
FROM golang:1.22-alpine
FROM golang:1.25-alpine AS source
WORKDIR /src
RUN apk add git
RUN apk add --no-cache ca-certificates git ffmpeg poppler-utils 7zip xz
COPY go.mod go.sum ./
RUN go mod download
COPY . .
FROM source AS test
# Test packages create independent SQLite stores. Serialize packages to avoid
# concurrent server/store initialization exhausting the shared runner's I/O.
# Keep production database deadlines and the complete test suite unchanged.
# Temporary test databases use bounded RAM-backed storage rather than the
# runner's overlay filesystem. Compiler temporary files remain on disk.
RUN --mount=type=tmpfs,target=/test-tmp,size=2147483648 \
TMPDIR=/test-tmp GOTMPDIR=/tmp go test -p 1 ./...
FROM source AS runtime
RUN go build -o /pageswiki ./cmd/pageswiki
EXPOSE 8081
+20
View File
@@ -0,0 +1,20 @@
FROM golang:1.25-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go build -o /adapter-gateway ./cmd/adapter-gateway
FROM node:22-alpine AS vimwasm
WORKDIR /build
COPY assets/vimwasm/package.json assets/vimwasm/package-lock.json ./
RUN npm ci --omit=dev
COPY assets/vimwasm/app ./app
RUN mkdir -p /out/vendor && cp app/* /out/ && cp node_modules/vim-wasm/vimwasm.js node_modules/vim-wasm/vim.js node_modules/vim-wasm/vim.wasm node_modules/vim-wasm/vim.data /out/vendor/
FROM alpine:3.22
COPY --from=build /adapter-gateway /adapter-gateway
COPY --from=vimwasm /out /vimwasm
COPY static/fonts/hack/HackNerdFont-Regular.ttf static/fonts/hack/LICENSE.md /vimwasm/
EXPOSE 3000
CMD ["/adapter-gateway"]
+11
View File
@@ -0,0 +1,11 @@
FROM node:22-alpine AS build
WORKDIR /src
COPY assets/excalidraw-viewer/package.json ./
COPY assets/excalidraw-viewer/package-lock.json ./
RUN npm ci
COPY assets/excalidraw-viewer/ ./
RUN npm run build
FROM nginx:1.29-alpine
COPY assets/excalidraw-viewer/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=build /src/dist /usr/share/nginx/html
+18
View File
@@ -0,0 +1,18 @@
# The relay image contains no browser engine. Mount the independently verified
# fixed Gecko distribution read-only at /gecko (see docs/operations/gecko-relay.md).
FROM golang:1.25-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY internal/geckorelay ./internal/geckorelay
COPY cmd/gecko-relay ./cmd/gecko-relay
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /gecko-relay ./cmd/gecko-relay
FROM scratch
COPY --from=build /gecko-relay /usr/local/bin/gecko-relay
COPY --from=build /usr/local/go/LICENSE /licenses/Go.txt
COPY --from=build /go/pkg/mod/golang.org/x/net@v0.35.0/LICENSE /licenses/golang-x-net.txt
COPY internal/geckorelay/NOTICE.txt /licenses/NOTICE.txt
USER 10001:10001
EXPOSE 3012
ENTRYPOINT ["/usr/local/bin/gecko-relay"]
+85
View File
@@ -1,3 +1,88 @@
# pageswiki
Simplest and powerful wiki architecture for personal
## Container CI
Gitea Actions tests and builds the application, adaptor gateway, and Excalidraw
viewer, then publishes them to the Gitea container registry with commit-SHA tags.
See [Gitea image builds and registry setup](docs/operations/gitea-images.md) for
runner requirements, registry secrets, and the optional registry Compose overlay.
The workflow does not deploy to production.
PagesWiki follows [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html).
Release tags (starting with `v0.1.0`) promote existing tested images without
rebuilding. See the [versioning and release policy](docs/operations/versioning.md).
## Knowledge City Model
Pages Wiki uses urban language for its content hierarchy:
```text
Campus → Domain → Zone (Quartier / Street) → Spot → Page
```
- **Campus** manages a group of Domains.
- **Domain** is a site space selected by a host or subdomain.
- **Zone** is the managed tree containing Quartiers and Streets.
- **Quartier** is a planned aggregation of Streets.
- **Street** groups Spots as the smallest city block.
- **Spot** is the smallest knowledge group.
- **Page** is the smallest knowledge unit and is identified by UUID, not path.
A Page may be linked from multiple Spots. Semantic paths are projections and
must not be treated as Page identity.
## Data CLI
The CLI in `data/pageswiki-cli` provides rsync-like transfers and an
interactive Semantic Path shell over SQLite and `data/documents/`:
```bash
data/pageswiki-cli put article.html path:/quartier/street/spot/page
data/pageswiki-cli get path:/quartier/street/spot/page article.html
data/pageswiki-cli shell
PAGESWIKI_DOMAIN=engineering data/pageswiki-cli ls /
```
The shell supports `ls`, `cd`, `cat`, `view`, `edit`, transfer, Revision,
diff/restore, and link commands, with tab completion and persistent history.
`view` uses `PAGER`; `edit` uses `EDITOR`.
Set `PAGESWIKI_DATA_DIR` when inspecting a different data directory.
`PAGESWIKI_DOMAIN` selects the active Domain by name, UUID, or host. In the
interactive shell, `domain [NAME|UUID|HOST]` displays or changes it.
An instance can enroll into a Controller Campus during container startup:
The included Docker Compose configuration enrolls `app2` into a Controller
Campus. Copy `.env.example` to `.env` and set
`PAGESWIKI_EXTERNAL_CAMPUS_TOKEN` to the controller-issued enrollment token
before running Compose. The token is intentionally not committed; `.env` is
excluded from Git and Docker build contexts. Keep it private (for example,
`chmod 600 .env`).
```bash
PAGESWIKI_INSTANCE_FQDN=node.example.test \
pageswiki --add-external-campus campus.example.test "$CAMPUS_TOKEN"
```
For an explicitly trusted development network without TLS:
```bash
PAGESWIKI_INSTANCE_FQDN=node.internal.test \
pageswiki --add-external-campus-http campus.internal.test "$CAMPUS_TOKEN"
```
For a node dedicated to externally managed Domains, add
`--no-default-campus`. Its bootstrap Campus and Domain remain while approval
is pending, then are removed after the first assigned Domain is pulled:
```bash
PAGESWIKI_INSTANCE_FQDN=node.internal.test \
pageswiki --no-default-campus \
--add-external-campus-http campus.internal.test "$CAMPUS_TOKEN"
```
The controller records the instance public-key fingerprint. Domain placement
only offers instances whose fingerprint has been approved.
+506
View File
@@ -0,0 +1,506 @@
{
"name": "pageswiki-city-generator",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "pageswiki-city-generator",
"dependencies": {
"gen-city": "1.4.6"
},
"devDependencies": {
"esbuild": "0.25.8"
}
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.25.8.tgz",
"integrity": "sha512-urAvrUedIqEiFR3FYSLTWQgLu5tb+m0qZw0NBEasUeo6wuqatkMDaRT+1uABiGXEu5vqgPd7FGE1BhsAIy9QVA==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.25.8.tgz",
"integrity": "sha512-RONsAvGCz5oWyePVnLdZY/HHwA++nxYWIX1atInlaW6SEkwq6XkP3+cb825EUcRs5Vss/lGh/2YxAb5xqc07Uw==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.25.8.tgz",
"integrity": "sha512-OD3p7LYzWpLhZEyATcTSJ67qB5D+20vbtr6vHlHWSQYhKtzUYrETuWThmzFpZtFsBIxRvhO07+UgVA9m0i/O1w==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.25.8.tgz",
"integrity": "sha512-yJAVPklM5+4+9dTeKwHOaA+LQkmrKFX96BM0A/2zQrbS6ENCmxc4OVoBs5dPkCCak2roAD+jKCdnmOqKszPkjA==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.25.8.tgz",
"integrity": "sha512-Jw0mxgIaYX6R8ODrdkLLPwBqHTtYHJSmzzd+QeytSugzQ0Vg4c5rDky5VgkoowbZQahCbsv1rT1KW72MPIkevw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.25.8.tgz",
"integrity": "sha512-Vh2gLxxHnuoQ+GjPNvDSDRpoBCUzY4Pu0kBqMBDlK4fuWbKgGtmDIeEC081xi26PPjn+1tct+Bh8FjyLlw1Zlg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.25.8.tgz",
"integrity": "sha512-YPJ7hDQ9DnNe5vxOm6jaie9QsTwcKedPvizTVlqWG9GBSq+BuyWEDazlGaDTC5NGU4QJd666V0yqCBL2oWKPfA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.25.8.tgz",
"integrity": "sha512-MmaEXxQRdXNFsRN/KcIimLnSJrk2r5H8v+WVafRWz5xdSVmWLoITZQXcgehI2ZE6gioE6HirAEToM/RvFBeuhw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.25.8.tgz",
"integrity": "sha512-FuzEP9BixzZohl1kLf76KEVOsxtIBFwCaLupVuk4eFVnOZfU+Wsn+x5Ryam7nILV2pkq2TqQM9EZPsOBuMC+kg==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.25.8.tgz",
"integrity": "sha512-WIgg00ARWv/uYLU7lsuDK00d/hHSfES5BzdWAdAig1ioV5kaFNrtK8EqGcUBJhYqotlUByUKz5Qo6u8tt7iD/w==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.25.8.tgz",
"integrity": "sha512-A1D9YzRX1i+1AJZuFFUMP1E9fMaYY+GnSQil9Tlw05utlE86EKTUA7RjwHDkEitmLYiFsRd9HwKBPEftNdBfjg==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.25.8.tgz",
"integrity": "sha512-O7k1J/dwHkY1RMVvglFHl1HzutGEFFZ3kNiDMSOyUrB7WcoHGf96Sh+64nTRT26l3GMbCW01Ekh/ThKM5iI7hQ==",
"cpu": [
"loong64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.25.8.tgz",
"integrity": "sha512-uv+dqfRazte3BzfMp8PAQXmdGHQt2oC/y2ovwpTteqrMx2lwaksiFZ/bdkXJC19ttTvNXBuWH53zy/aTj1FgGw==",
"cpu": [
"mips64el"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.25.8.tgz",
"integrity": "sha512-GyG0KcMi1GBavP5JgAkkstMGyMholMDybAf8wF5A70CALlDM2p/f7YFE7H92eDeH/VBtFJA5MT4nRPDGg4JuzQ==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.25.8.tgz",
"integrity": "sha512-rAqDYFv3yzMrq7GIcen3XP7TUEG/4LK86LUPMIz6RT8A6pRIDn0sDcvjudVZBiiTcZCY9y2SgYX2lgK3AF+1eg==",
"cpu": [
"riscv64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.25.8.tgz",
"integrity": "sha512-Xutvh6VjlbcHpsIIbwY8GVRbwoviWT19tFhgdA7DlenLGC/mbc3lBoVb7jxj9Z+eyGqvcnSyIltYUrkKzWqSvg==",
"cpu": [
"s390x"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.25.8.tgz",
"integrity": "sha512-ASFQhgY4ElXh3nDcOMTkQero4b1lgubskNlhIfJrsH5OKZXDpUAKBlNS0Kx81jwOBp+HCeZqmoJuihTv57/jvQ==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.25.8.tgz",
"integrity": "sha512-d1KfruIeohqAi6SA+gENMuObDbEjn22olAR7egqnkCD9DGBG0wsEARotkLgXDu6c4ncgWTZJtN5vcgxzWRMzcw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.25.8.tgz",
"integrity": "sha512-nVDCkrvx2ua+XQNyfrujIG38+YGyuy2Ru9kKVNyh5jAys6n+l44tTtToqHjino2My8VAY6Lw9H7RI73XFi66Cg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.25.8.tgz",
"integrity": "sha512-j8HgrDuSJFAujkivSMSfPQSAa5Fxbvk4rgNAS5i3K+r8s1X0p1uOO2Hl2xNsGFppOeHOLAVgYwDVlmxhq5h+SQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.25.8.tgz",
"integrity": "sha512-1h8MUAwa0VhNCDp6Af0HToI2TJFAn1uqT9Al6DJVzdIBAd21m/G0Yfc77KDM3uF3T/YaOgQq3qTJHPbTOInaIQ==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openharmony-arm64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.25.8.tgz",
"integrity": "sha512-r2nVa5SIK9tSWd0kJd9HCffnDHKchTGikb//9c7HX+r+wHYCpQrSgxhlY6KWV1nFo1l4KFbsMlHk+L6fekLsUg==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openharmony"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.25.8.tgz",
"integrity": "sha512-zUlaP2S12YhQ2UzUfcCuMDHQFJyKABkAjvO5YSndMiIkMimPmxA+BYSBikWgsRpvyxuRnow4nS5NPnf9fpv41w==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.25.8.tgz",
"integrity": "sha512-YEGFFWESlPva8hGL+zvj2z/SaK+pH0SwOM0Nc/d+rVnW7GSTFlLBGzZkuSU9kFIGIo8q9X3ucpZhu8PDN5A2sQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.25.8.tgz",
"integrity": "sha512-hiGgGC6KZ5LZz58OL/+qVVoZiuZlUYlYHNAmczOm7bs2oE1XriPFi5ZHHrS8ACpV5EjySrnoCKmcbQMN+ojnHg==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.25.8.tgz",
"integrity": "sha512-cn3Yr7+OaaZq1c+2pe+8yxC8E144SReCQjN6/2ynubzYjvyqZjTXfQJpAcQpsdJq3My7XADANiYGHoFC69pLQw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/esbuild": {
"version": "0.25.8",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.8.tgz",
"integrity": "sha512-vVC0USHGtMi8+R4Kz8rt6JhEWLxsv9Rnu/lGYbPR8u47B+DCBksq9JarW0zOO7bs37hyOK1l2/oqtbciutL5+Q==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"bin": {
"esbuild": "bin/esbuild"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.25.8",
"@esbuild/android-arm": "0.25.8",
"@esbuild/android-arm64": "0.25.8",
"@esbuild/android-x64": "0.25.8",
"@esbuild/darwin-arm64": "0.25.8",
"@esbuild/darwin-x64": "0.25.8",
"@esbuild/freebsd-arm64": "0.25.8",
"@esbuild/freebsd-x64": "0.25.8",
"@esbuild/linux-arm": "0.25.8",
"@esbuild/linux-arm64": "0.25.8",
"@esbuild/linux-ia32": "0.25.8",
"@esbuild/linux-loong64": "0.25.8",
"@esbuild/linux-mips64el": "0.25.8",
"@esbuild/linux-ppc64": "0.25.8",
"@esbuild/linux-riscv64": "0.25.8",
"@esbuild/linux-s390x": "0.25.8",
"@esbuild/linux-x64": "0.25.8",
"@esbuild/netbsd-arm64": "0.25.8",
"@esbuild/netbsd-x64": "0.25.8",
"@esbuild/openbsd-arm64": "0.25.8",
"@esbuild/openbsd-x64": "0.25.8",
"@esbuild/openharmony-arm64": "0.25.8",
"@esbuild/sunos-x64": "0.25.8",
"@esbuild/win32-arm64": "0.25.8",
"@esbuild/win32-ia32": "0.25.8",
"@esbuild/win32-x64": "0.25.8"
}
},
"node_modules/gen-city": {
"version": "1.4.6",
"resolved": "https://registry.npmjs.org/gen-city/-/gen-city-1.4.6.tgz",
"integrity": "sha512-7Vw/HZvVbnDIkLkQbYffPtXp42tFsf2Szf8ptbimyTNg+Iv8ELj6hzgrlznGlRDe8MJg9gu3zISb4AEcvoFsWg==",
"license": "MIT"
}
}
}
+7
View File
@@ -0,0 +1,7 @@
{
"name": "pageswiki-city-generator",
"private": true,
"scripts": { "build": "esbuild src/adapter.js --bundle --format=iife --global-name=PagesWikiGenCity --outfile=../../static/vendor/city-generator/gen-city.bundle.js" },
"dependencies": { "gen-city": "1.4.6" },
"devDependencies": { "esbuild": "0.25.8" }
}
+22
View File
@@ -0,0 +1,22 @@
import { City, CityGenerationMode } from 'gen-city';
const seed = value => {
let state = 2166136261;
for (const char of String(value)) { state ^= char.charCodeAt(0); state = Math.imul(state, 16777619); }
return Array.from({length:256}, () => { state = Math.imul(state ^ state >>> 15, 2246822519); return Math.abs(state % 1001); });
};
export async function generateFabric(key, width = 180, height = 100) {
if (typeof key !== 'string' || !key || key.length > 256 || !Number.isInteger(width) || !Number.isInteger(height) || width < 64 || width > 320 || height < 64 || height > 240) {
throw new RangeError('City generation bounds exceeded');
}
const city = new City({width, height});
await city.generate({mode:CityGenerationMode.SEED,seed:seed(key),streetMinLength:9,probabilityIntersection:.12,probabilityTurn:.09,probabilityStreetEnd:.012,buildingMinSize:3,buildingMaxSize:7,buildingMinSpace:1,buildingMaxSpace:2,buildingOffset:1});
const fabric = {
generator:'gen-city', version:'1.4.6', width, height,
roads:city.getAllPaths().map(path => { const p=path.getPositions(); return [p.beg.x,p.beg.y,p.end.x,p.end.y]; }),
buildings:city.getAllBuildings().map(building => building.vertices.map(point => [point.x,point.y]))
};
if (fabric.roads.length > 4096 || fabric.buildings.length > 4096) throw new RangeError('City geometry budget exceeded');
return fabric;
}
+9
View File
@@ -0,0 +1,9 @@
import { build } from "esbuild";
await build({
entryPoints: ["src/editor.js"],
bundle: true,
minify: false,
sourcemap: true,
outfile: "../../static/vendor/codemirror/editor.bundle.js",
});
+19
View File
@@ -0,0 +1,19 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import '../../static/js/editor-core.js';
const {merge,validDraft}=globalThis.PagesEditorCore;
test('three-way merge keeps disjoint changes',()=>{
assert.equal(merge('one two three','ONE two three','one two THREE'),'ONE two THREE');
assert.equal(merge('abc','abc','Abc'),'Abc');
assert.equal(merge('abc','ABC','ABC'),'ABC');
assert.equal(merge('abc','Abc','aBc'),null); // adjacent ranges
assert.equal(merge('abc','axbc','aybc'),null); // competing insertions
assert.equal(merge('abc','axbc','abc!'),'axbc!');
assert.equal(merge('one two three','one LOCAL three','one REMOTE three'),null);
});
test('draft format, date, expiry and content validated',()=>{
const now=Date.now(),draft={version:1,text:'source',baseBody:'base',baseRevision:'uuid',format:'markdown',savedAt:now};
assert.ok(validDraft(draft,now));
assert.ok(validDraft({...draft,format:'plain',text:'<script>literal</script>'},now));
for(const fields of [{version:2},{text:null},{baseRevision:1},{format:'projection'},{savedAt:now-8*86400000},{savedAt:now+120000}]) assert.ok(!validDraft({...draft,...fields},now));
});
+872
View File
@@ -0,0 +1,872 @@
{
"name": "editor",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "editor",
"version": "1.0.0",
"license": "ISC",
"dependencies": {
"@codemirror/autocomplete": "^6.20.3",
"@codemirror/commands": "^6.10.3",
"@codemirror/lang-cpp": "^6.0.3",
"@codemirror/lang-css": "^6.3.1",
"@codemirror/lang-go": "^6.0.1",
"@codemirror/lang-html": "^6.4.11",
"@codemirror/lang-java": "^6.0.2",
"@codemirror/lang-javascript": "^6.2.5",
"@codemirror/lang-json": "^6.0.2",
"@codemirror/lang-markdown": "^6.5.0",
"@codemirror/lang-python": "^6.2.1",
"@codemirror/lang-rust": "^6.0.2",
"@codemirror/lang-sql": "^6.10.0",
"@codemirror/lang-yaml": "^6.1.3",
"@codemirror/language": "^6.12.3",
"@codemirror/legacy-modes": "^6.5.3",
"@codemirror/state": "^6.6.0",
"@codemirror/view": "^6.43.0",
"@lezer/highlight": "^1.2.3",
"@replit/codemirror-vim": "^6.3.0",
"esbuild": "^0.28.0"
}
},
"node_modules/@codemirror/autocomplete": {
"version": "6.20.3",
"resolved": "https://registry.npmjs.org/@codemirror/autocomplete/-/autocomplete-6.20.3.tgz",
"integrity": "sha512-tlosUqb+3BbxCxZdu4tKeRghPFC+QM7q4X5YhKV2eCmPG+1r2F3f4AaSz5sCrFqUtX4Jh20VFTKecl16MgiV9g==",
"dependencies": {
"@codemirror/language": "^6.0.0",
"@codemirror/state": "^6.0.0",
"@codemirror/view": "^6.17.0",
"@lezer/common": "^1.0.0"
}
},
"node_modules/@codemirror/commands": {
"version": "6.10.3",
"resolved": "https://registry.npmjs.org/@codemirror/commands/-/commands-6.10.3.tgz",
"integrity": "sha512-JFRiqhKu+bvSkDLI+rUhJwSxQxYb759W5GBezE8Uc8mHLqC9aV/9aTC7yJSqCtB3F00pylrLCwnyS91Ap5ej4Q==",
"dependencies": {
"@codemirror/language": "^6.0.0",
"@codemirror/state": "^6.6.0",
"@codemirror/view": "^6.27.0",
"@lezer/common": "^1.1.0"
}
},
"node_modules/@codemirror/lang-cpp": {
"version": "6.0.3",
"resolved": "https://registry.npmjs.org/@codemirror/lang-cpp/-/lang-cpp-6.0.3.tgz",
"integrity": "sha512-URM26M3vunFFn9/sm6rzqrBzDgfWuDixp85uTY49wKudToc2jTHUrKIGGKs+QWND+YLofNNZpxcNGRynFJfvgA==",
"license": "MIT",
"dependencies": {
"@codemirror/language": "^6.0.0",
"@lezer/cpp": "^1.0.0"
}
},
"node_modules/@codemirror/lang-css": {
"version": "6.3.1",
"resolved": "https://registry.npmjs.org/@codemirror/lang-css/-/lang-css-6.3.1.tgz",
"integrity": "sha512-kr5fwBGiGtmz6l0LSJIbno9QrifNMUusivHbnA1H6Dmqy4HZFte3UAICix1VuKo0lMPKQr2rqB+0BkKi/S3Ejg==",
"license": "MIT",
"dependencies": {
"@codemirror/autocomplete": "^6.0.0",
"@codemirror/language": "^6.0.0",
"@codemirror/state": "^6.0.0",
"@lezer/common": "^1.0.2",
"@lezer/css": "^1.1.7"
}
},
"node_modules/@codemirror/lang-go": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/@codemirror/lang-go/-/lang-go-6.0.1.tgz",
"integrity": "sha512-7fNvbyNylvqCphW9HD6WFnRpcDjr+KXX/FgqXy5H5ZS0eC5edDljukm/yNgYkwTsgp2busdod50AOTIy6Jikfg==",
"license": "MIT",
"dependencies": {
"@codemirror/autocomplete": "^6.0.0",
"@codemirror/language": "^6.6.0",
"@codemirror/state": "^6.0.0",
"@lezer/common": "^1.0.0",
"@lezer/go": "^1.0.0"
}
},
"node_modules/@codemirror/lang-html": {
"version": "6.4.11",
"resolved": "https://registry.npmjs.org/@codemirror/lang-html/-/lang-html-6.4.11.tgz",
"integrity": "sha512-9NsXp7Nwp891pQchI7gPdTwBuSuT3K65NGTHWHNJ55HjYcHLllr0rbIZNdOzas9ztc1EUVBlHou85FFZS4BNnw==",
"dependencies": {
"@codemirror/autocomplete": "^6.0.0",
"@codemirror/lang-css": "^6.0.0",
"@codemirror/lang-javascript": "^6.0.0",
"@codemirror/language": "^6.4.0",
"@codemirror/state": "^6.0.0",
"@codemirror/view": "^6.17.0",
"@lezer/common": "^1.0.0",
"@lezer/css": "^1.1.0",
"@lezer/html": "^1.3.12"
}
},
"node_modules/@codemirror/lang-java": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/@codemirror/lang-java/-/lang-java-6.0.2.tgz",
"integrity": "sha512-m5Nt1mQ/cznJY7tMfQTJchmrjdjQ71IDs+55d1GAa8DGaB8JXWsVCkVT284C3RTASaY43YknrK2X3hPO/J3MOQ==",
"license": "MIT",
"dependencies": {
"@codemirror/language": "^6.0.0",
"@lezer/java": "^1.0.0"
}
},
"node_modules/@codemirror/lang-javascript": {
"version": "6.2.5",
"resolved": "https://registry.npmjs.org/@codemirror/lang-javascript/-/lang-javascript-6.2.5.tgz",
"integrity": "sha512-zD4e5mS+50htS7F+TYjBPsiIFGanfVqg4HyUz6WNFikgOPf2BgKlx+TQedI1w6n/IqRBVBbBWmGFdLB/7uxO4A==",
"license": "MIT",
"dependencies": {
"@codemirror/autocomplete": "^6.0.0",
"@codemirror/language": "^6.6.0",
"@codemirror/lint": "^6.0.0",
"@codemirror/state": "^6.0.0",
"@codemirror/view": "^6.17.0",
"@lezer/common": "^1.0.0",
"@lezer/javascript": "^1.0.0"
}
},
"node_modules/@codemirror/lang-json": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/@codemirror/lang-json/-/lang-json-6.0.2.tgz",
"integrity": "sha512-x2OtO+AvwEHrEwR0FyyPtfDUiloG3rnVTSZV1W8UteaLL8/MajQd8DpvUb2YVzC+/T18aSDv0H9mu+xw0EStoQ==",
"license": "MIT",
"dependencies": {
"@codemirror/language": "^6.0.0",
"@lezer/json": "^1.0.0"
}
},
"node_modules/@codemirror/lang-markdown": {
"version": "6.5.0",
"resolved": "https://registry.npmjs.org/@codemirror/lang-markdown/-/lang-markdown-6.5.0.tgz",
"integrity": "sha512-0K40bZ35jpHya6FriukbgaleaqzBLZfOh7HuzqbMxBXkbYMJDxfF39c23xOgxFezR+3G+tR2/Mup+Xk865OMvw==",
"dependencies": {
"@codemirror/autocomplete": "^6.7.1",
"@codemirror/lang-html": "^6.0.0",
"@codemirror/language": "^6.3.0",
"@codemirror/state": "^6.0.0",
"@codemirror/view": "^6.0.0",
"@lezer/common": "^1.2.1",
"@lezer/markdown": "^1.0.0"
}
},
"node_modules/@codemirror/lang-python": {
"version": "6.2.1",
"resolved": "https://registry.npmjs.org/@codemirror/lang-python/-/lang-python-6.2.1.tgz",
"integrity": "sha512-IRjC8RUBhn9mGR9ywecNhB51yePWCGgvHfY1lWN/Mrp3cKuHr0isDKia+9HnvhiWNnMpbGhWrkhuWOc09exRyw==",
"license": "MIT",
"dependencies": {
"@codemirror/autocomplete": "^6.3.2",
"@codemirror/language": "^6.8.0",
"@codemirror/state": "^6.0.0",
"@lezer/common": "^1.2.1",
"@lezer/python": "^1.1.4"
}
},
"node_modules/@codemirror/lang-rust": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/@codemirror/lang-rust/-/lang-rust-6.0.2.tgz",
"integrity": "sha512-EZaGjCUegtiU7kSMvOfEZpaCReowEf3yNidYu7+vfuGTm9ow4mthAparY5hisJqOHmJowVH3Upu+eJlUji6qqA==",
"license": "MIT",
"dependencies": {
"@codemirror/language": "^6.0.0",
"@lezer/rust": "^1.0.0"
}
},
"node_modules/@codemirror/lang-sql": {
"version": "6.10.0",
"resolved": "https://registry.npmjs.org/@codemirror/lang-sql/-/lang-sql-6.10.0.tgz",
"integrity": "sha512-6ayPkEd/yRw0XKBx5uAiToSgGECo/GY2NoJIHXIIQh1EVwLuKoU8BP/qK0qH5NLXAbtJRLuT73hx7P9X34iO4w==",
"license": "MIT",
"dependencies": {
"@codemirror/autocomplete": "^6.0.0",
"@codemirror/language": "^6.0.0",
"@codemirror/state": "^6.0.0",
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.0.0",
"@lezer/lr": "^1.0.0"
}
},
"node_modules/@codemirror/lang-yaml": {
"version": "6.1.3",
"resolved": "https://registry.npmjs.org/@codemirror/lang-yaml/-/lang-yaml-6.1.3.tgz",
"integrity": "sha512-AZ8DJBuXGVHybpBQhmZtgew5//4hv3tdkXnr3vDmOUMJRuB6vn/uuwtmTOTlqEaQFg3hQSVeA90NmvIQyUV6FQ==",
"license": "MIT",
"dependencies": {
"@codemirror/autocomplete": "^6.0.0",
"@codemirror/language": "^6.0.0",
"@codemirror/state": "^6.0.0",
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.2.0",
"@lezer/lr": "^1.0.0",
"@lezer/yaml": "^1.0.0"
}
},
"node_modules/@codemirror/language": {
"version": "6.12.3",
"resolved": "https://registry.npmjs.org/@codemirror/language/-/language-6.12.3.tgz",
"integrity": "sha512-QwCZW6Tt1siP37Jet9Tb02Zs81TQt6qQrZR2H+eGMcFsL1zMrk2/b9CLC7/9ieP1fjIUMgviLWMmgiHoJrj+ZA==",
"dependencies": {
"@codemirror/state": "^6.0.0",
"@codemirror/view": "^6.23.0",
"@lezer/common": "^1.5.0",
"@lezer/highlight": "^1.0.0",
"@lezer/lr": "^1.0.0",
"style-mod": "^4.0.0"
}
},
"node_modules/@codemirror/legacy-modes": {
"version": "6.5.3",
"resolved": "https://registry.npmjs.org/@codemirror/legacy-modes/-/legacy-modes-6.5.3.tgz",
"integrity": "sha512-xCsmIzH78MyWkib9jlPaaun57XNkfbMIhagfaZVd0iLTqlpw3jXaIcbZm72MTmmn64eTZpBVNjbyYh+QXnxRsg==",
"license": "MIT",
"dependencies": {
"@codemirror/language": "^6.0.0"
}
},
"node_modules/@codemirror/lint": {
"version": "6.9.6",
"resolved": "https://registry.npmjs.org/@codemirror/lint/-/lint-6.9.6.tgz",
"integrity": "sha512-6Kp7r6XfCi/D/5sdXieMfg9pJU1bUEx96WITuLU6ESaKizCz0QHFMjY/TaFSbigDdEAIgi93itLBIUETP4oK+A==",
"dependencies": {
"@codemirror/state": "^6.0.0",
"@codemirror/view": "^6.42.0",
"crelt": "^1.0.5"
}
},
"node_modules/@codemirror/search": {
"version": "6.7.0",
"resolved": "https://registry.npmjs.org/@codemirror/search/-/search-6.7.0.tgz",
"integrity": "sha512-ZvGm99wc/s2cITtMT15LFdn8aH/aS+V+DqyGq/N5ZlV5vWtH+nILvC2nw0zX7ByNoHHDZ2IxxdW38O0tc5nVHg==",
"peer": true,
"dependencies": {
"@codemirror/state": "^6.0.0",
"@codemirror/view": "^6.37.0",
"crelt": "^1.0.5"
}
},
"node_modules/@codemirror/state": {
"version": "6.6.0",
"resolved": "https://registry.npmjs.org/@codemirror/state/-/state-6.6.0.tgz",
"integrity": "sha512-4nbvra5R5EtiCzr9BTHiTLc+MLXK2QGiAVYMyi8PkQd3SR+6ixar/Q/01Fa21TBIDOZXgeWV4WppsQolSreAPQ==",
"dependencies": {
"@marijn/find-cluster-break": "^1.0.0"
}
},
"node_modules/@codemirror/view": {
"version": "6.43.0",
"resolved": "https://registry.npmjs.org/@codemirror/view/-/view-6.43.0.tgz",
"integrity": "sha512-V7ZCLQO3Jus9hzh2jVCCPW3mO4IBMr43O37PqSUYautJSnnJF41YlgLw21x0fLJTYvJ+Vkm6Gp+qKGH9pltgXA==",
"dependencies": {
"@codemirror/state": "^6.6.0",
"crelt": "^1.0.6",
"style-mod": "^4.1.0",
"w3c-keyname": "^2.2.4"
}
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.0.tgz",
"integrity": "sha512-lhRUCeuOyJQURhTxl4WkpFTjIsbDayJHih5kZC1giwE+MhIzAb7mEsQMqMf18rHLsrb5qI1tafG20mLxEWcWlA==",
"cpu": [
"ppc64"
],
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.0.tgz",
"integrity": "sha512-wqh0ByljabXLKHeWXYLqoJ5jKC4XBaw6Hk08OfMrCRd2nP2ZQ5eleDZC41XHyCNgktBGYMbqnrJKq/K/lzPMSQ==",
"cpu": [
"arm"
],
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.0.tgz",
"integrity": "sha512-+WzIXQOSaGs33tLEgYPYe/yQHf0WTU0X42Jca3y8NWMbUVhp7rUnw+vAsRC/QiDrdD31IszMrZy+qwPOPjd+rw==",
"cpu": [
"arm64"
],
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.0.tgz",
"integrity": "sha512-+VJggoaKhk2VNNqVL7f6S189UzShHC/mR9EE8rDdSkdpN0KflSwWY/gWjDrNxxisg8Fp1ZCD9jLMo4m0OUfeUA==",
"cpu": [
"x64"
],
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.0.tgz",
"integrity": "sha512-0T+A9WZm+bZ84nZBtk1ckYsOvyA3x7e2Acj1KdVfV4/2tdG4fzUp91YHx+GArWLtwqp77pBXVCPn2We7Letr0Q==",
"cpu": [
"arm64"
],
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.0.tgz",
"integrity": "sha512-fyzLm/DLDl/84OCfp2f/XQ4flmORsjU7VKt8HLjvIXChJoFFOIL6pLJPH4Yhd1n1gGFF9mPwtlN5Wf82DZs+LQ==",
"cpu": [
"x64"
],
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.0.tgz",
"integrity": "sha512-l9GeW5UZBT9k9brBYI+0WDffcRxgHQD8ShN2Ur4xWq/NFzUKm3k5lsH4PdaRgb2w7mI9u61nr2gI2mLI27Nh3Q==",
"cpu": [
"arm64"
],
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.0.tgz",
"integrity": "sha512-BXoQai/A0wPO6Es3yFJ7APCiKGc1tdAEOgeTNy3SsB491S3aHn4S4r3e976eUnPdU+NbdtmBuLncYir2tMU9Nw==",
"cpu": [
"x64"
],
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.0.tgz",
"integrity": "sha512-CjaaREJagqJp7iTaNQjjidaNbCKYcd4IDkzbwwxtSvjI7NZm79qiHc8HqciMddQ6CKvJT6aBd8lO9kN/ZudLlw==",
"cpu": [
"arm"
],
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.0.tgz",
"integrity": "sha512-RVyzfb3FWsGA55n6WY0MEIEPURL1FcbhFE6BffZEMEekfCzCIMtB5yyDcFnVbTnwk+CLAgTujmV/Lgvih56W+A==",
"cpu": [
"arm64"
],
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.0.tgz",
"integrity": "sha512-KBnSTt1kxl9x70q+ydterVdl+Cn0H18ngRMRCEQfrbqdUuntQQ0LoMZv47uB97NljZFzY6HcfqEZ2SAyIUTQBQ==",
"cpu": [
"ia32"
],
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.0.tgz",
"integrity": "sha512-zpSlUce1mnxzgBADvxKXX5sl8aYQHo2ezvMNI8I0lbblJtp8V4odlm3Yzlj7gPyt3T8ReksE6bK+pT3WD+aJRg==",
"cpu": [
"loong64"
],
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.0.tgz",
"integrity": "sha512-2jIfP6mmjkdmeTlsX/9vmdmhBmKADrWqN7zcdtHIeNSCH1SqIoNI63cYsjQR8J+wGa4Y5izRcSHSm8K3QWmk3w==",
"cpu": [
"mips64el"
],
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.0.tgz",
"integrity": "sha512-bc0FE9wWeC0WBm49IQMPSPILRocGTQt3j5KPCA8os6VprfuJ7KD+5PzESSrJ6GmPIPJK965ZJHTUlSA6GNYEhg==",
"cpu": [
"ppc64"
],
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.0.tgz",
"integrity": "sha512-SQPZOwoTTT/HXFXQJG/vBX8sOFagGqvZyXcgLA3NhIqcBv1BJU1d46c0rGcrij2B56Z2rNiSLaZOYW5cUk7yLQ==",
"cpu": [
"riscv64"
],
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.0.tgz",
"integrity": "sha512-SCfR0HN8CEEjnYnySJTd2cw0k9OHB/YFzt5zgJEwa+wL/T/raGWYMBqwDNAC6dqFKmJYZoQBRfHjgwLHGSrn3Q==",
"cpu": [
"s390x"
],
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.0.tgz",
"integrity": "sha512-us0dSb9iFxIi8srnpl931Nvs65it/Jd2a2K3qs7fz2WfGPHqzfzZTfec7oxZJRNPXPnNYZtanmRc4AL/JwVzHQ==",
"cpu": [
"x64"
],
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.0.tgz",
"integrity": "sha512-CR/RYotgtCKwtftMwJlUU7xCVNg3lMYZ0RzTmAHSfLCXw3NtZtNpswLEj/Kkf6kEL3Gw+BpOekRX0BYCtklhUw==",
"cpu": [
"arm64"
],
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.0.tgz",
"integrity": "sha512-nU1yhmYutL+fQ71Kxnhg8uEOdC0pwEW9entHykTgEbna2pw2dkbFSMeqjjyHZoCmt8SBkOSvV+yNmm94aUrrqw==",
"cpu": [
"x64"
],
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.0.tgz",
"integrity": "sha512-cXb5vApOsRsxsEl4mcZ1XY3D4DzcoMxR/nnc4IyqYs0rTI8ZKmW6kyyg+11Z8yvgMfAEldKzP7AdP64HnSC/6g==",
"cpu": [
"arm64"
],
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.0.tgz",
"integrity": "sha512-8wZM2qqtv9UP3mzy7HiGYNH/zjTA355mpeuA+859TyR+e+Tc08IHYpLJuMsfpDJwoLo1ikIJI8jC3GFjnRClzA==",
"cpu": [
"x64"
],
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openharmony-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.0.tgz",
"integrity": "sha512-FLGfyizszcef5C3YtoyQDACyg95+dndv79i2EekILBofh5wpCa1KuBqOWKrEHZg3zrL3t5ouE5jgr94vA+Wb2w==",
"cpu": [
"arm64"
],
"optional": true,
"os": [
"openharmony"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.0.tgz",
"integrity": "sha512-1ZgjUoEdHZZl/YlV76TSCz9Hqj9h9YmMGAgAPYd+q4SicWNX3G5GCyx9uhQWSLcbvPW8Ni7lj4gDa1T40akdlw==",
"cpu": [
"x64"
],
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.0.tgz",
"integrity": "sha512-Q9StnDmQ/enxnpxCCLSg0oo4+34B9TdXpuyPeTedN/6+iXBJ4J+zwfQI28u/Jl40nOYAxGoNi7mFP40RUtkmUA==",
"cpu": [
"arm64"
],
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.0.tgz",
"integrity": "sha512-zF3ag/gfiCe6U2iczcRzSYJKH1DCI+ByzSENHlM2FcDbEeo5Zd2C86Aq0tKUYAJJ1obRP84ymxIAksZUcdztHA==",
"cpu": [
"ia32"
],
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.0.tgz",
"integrity": "sha512-pEl1bO9mfAmIC+tW5btTmrKaujg3zGtUmWNdCw/xs70FBjwAL3o9OEKNHvNmnyylD6ubxUERiEhdsL0xBQ9efw==",
"cpu": [
"x64"
],
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@lezer/common": {
"version": "1.5.2",
"resolved": "https://registry.npmjs.org/@lezer/common/-/common-1.5.2.tgz",
"integrity": "sha512-sxQE460fPZyU3sdc8lafxiPwJHBzZRy/udNFynGQky1SePYBdhkBl1kOagA9uT3pxR8K09bOrmTUqA9wb/PjSQ=="
},
"node_modules/@lezer/cpp": {
"version": "1.1.6",
"resolved": "https://registry.npmjs.org/@lezer/cpp/-/cpp-1.1.6.tgz",
"integrity": "sha512-vh9gWWJOXFVY8HBHK3Twzq8MgwG2iN4GSyzBP9sCGTe37P15x2R14VaBQk0VA0ezTRN1KHYBBsHhvpGZ2Xy/pA==",
"license": "MIT",
"dependencies": {
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.0.0",
"@lezer/lr": "^1.0.0"
}
},
"node_modules/@lezer/css": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/@lezer/css/-/css-1.3.3.tgz",
"integrity": "sha512-RzBo8r+/6QJeow7aPHIpGVIH59xTcJXp399820gZoMo9noQDRVpJLheIBUicYwKcsbOYoBRoLZlf2720dG/4Tg==",
"dependencies": {
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.0.0",
"@lezer/lr": "^1.3.0"
}
},
"node_modules/@lezer/go": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@lezer/go/-/go-1.0.1.tgz",
"integrity": "sha512-xToRsYxwsgJNHTgNdStpcvmbVuKxTapV0dM0wey1geMMRc9aggoVyKgzYp41D2/vVOx+Ii4hmE206kvxIXBVXQ==",
"license": "MIT",
"dependencies": {
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.0.0",
"@lezer/lr": "^1.3.0"
}
},
"node_modules/@lezer/highlight": {
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/@lezer/highlight/-/highlight-1.2.3.tgz",
"integrity": "sha512-qXdH7UqTvGfdVBINrgKhDsVTJTxactNNxLk7+UMwZhU13lMHaOBlJe9Vqp907ya56Y3+ed2tlqzys7jDkTmW0g==",
"dependencies": {
"@lezer/common": "^1.3.0"
}
},
"node_modules/@lezer/html": {
"version": "1.3.13",
"resolved": "https://registry.npmjs.org/@lezer/html/-/html-1.3.13.tgz",
"integrity": "sha512-oI7n6NJml729m7pjm9lvLvmXbdoMoi2f+1pwSDJkl9d68zGr7a9Btz8NdHTGQZtW2DA25ybeuv/SyDb9D5tseg==",
"dependencies": {
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.0.0",
"@lezer/lr": "^1.0.0"
}
},
"node_modules/@lezer/java": {
"version": "1.1.3",
"resolved": "https://registry.npmjs.org/@lezer/java/-/java-1.1.3.tgz",
"integrity": "sha512-yHquUfujwg6Yu4Fd1GNHCvidIvJwi/1Xu2DaKl/pfWIA2c1oXkVvawH3NyXhCaFx4OdlYBVX5wvz2f7Aoa/4Xw==",
"license": "MIT",
"dependencies": {
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.0.0",
"@lezer/lr": "^1.0.0"
}
},
"node_modules/@lezer/javascript": {
"version": "1.5.4",
"resolved": "https://registry.npmjs.org/@lezer/javascript/-/javascript-1.5.4.tgz",
"integrity": "sha512-vvYx3MhWqeZtGPwDStM2dwgljd5smolYD2lR2UyFcHfxbBQebqx8yjmFmxtJ/E6nN6u1D9srOiVWm3Rb4tmcUA==",
"dependencies": {
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.1.3",
"@lezer/lr": "^1.3.0"
}
},
"node_modules/@lezer/json": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/@lezer/json/-/json-1.0.3.tgz",
"integrity": "sha512-BP9KzdF9Y35PDpv04r0VeSTKDeox5vVr3efE7eBbx3r4s3oNLfunchejZhjArmeieBH+nVOpgIiBJpEAv8ilqQ==",
"license": "MIT",
"dependencies": {
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.0.0",
"@lezer/lr": "^1.0.0"
}
},
"node_modules/@lezer/lr": {
"version": "1.4.10",
"resolved": "https://registry.npmjs.org/@lezer/lr/-/lr-1.4.10.tgz",
"integrity": "sha512-rnCpTIBafOx4mRp43xOxDJbFipJm/c0cia/V5TiGlhmMa+wsSdoGmUN3w5Bqrks/09Q/D4tNAmWaT8p6NRi77A==",
"dependencies": {
"@lezer/common": "^1.0.0"
}
},
"node_modules/@lezer/markdown": {
"version": "1.6.4",
"resolved": "https://registry.npmjs.org/@lezer/markdown/-/markdown-1.6.4.tgz",
"integrity": "sha512-N0SxazMj4k65DBfaf1azqtMZd6u7MqluP84/NZnB/io8Td9aleFmAhz9hcbvSfsxT5tdYlJ5qgv5aMJGY4zEtA==",
"dependencies": {
"@lezer/common": "^1.5.0",
"@lezer/highlight": "^1.0.0"
}
},
"node_modules/@lezer/python": {
"version": "1.1.19",
"resolved": "https://registry.npmjs.org/@lezer/python/-/python-1.1.19.tgz",
"integrity": "sha512-MhQIURHRytsNzP/YXnqpYKW6la6voAH3kyplTOOiCdjyFY6cWWGFVmYVdHIPrElqSDf4iCDktQCockB9FxuhzQ==",
"license": "MIT",
"dependencies": {
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.0.0",
"@lezer/lr": "^1.0.0"
}
},
"node_modules/@lezer/rust": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/@lezer/rust/-/rust-1.0.2.tgz",
"integrity": "sha512-Lz5sIPBdF2FUXcWeCu1//ojFAZqzTQNRga0aYv6dYXqJqPfMdCAI0NzajWUd4Xijj1IKJLtjoXRPMvTKWBcqKg==",
"license": "MIT",
"dependencies": {
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.0.0",
"@lezer/lr": "^1.0.0"
}
},
"node_modules/@lezer/yaml": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/@lezer/yaml/-/yaml-1.0.4.tgz",
"integrity": "sha512-2lrrHqxalACEbxIbsjhqGpSW8kWpUKuY6RHgnSAFZa6qK62wvnPxA8hGOwOoDbwHcOFs5M4o27mjGu+P7TvBmw==",
"license": "MIT",
"dependencies": {
"@lezer/common": "^1.2.0",
"@lezer/highlight": "^1.0.0",
"@lezer/lr": "^1.4.0"
}
},
"node_modules/@marijn/find-cluster-break": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/@marijn/find-cluster-break/-/find-cluster-break-1.0.2.tgz",
"integrity": "sha512-l0h88YhZFyKdXIFNfSWpyjStDjGHwZ/U7iobcK1cQQD8sejsONdQtTVU+1wVN1PBw40PiiHB1vA5S7VTfQiP9g=="
},
"node_modules/@replit/codemirror-vim": {
"version": "6.3.0",
"resolved": "https://registry.npmjs.org/@replit/codemirror-vim/-/codemirror-vim-6.3.0.tgz",
"integrity": "sha512-aTx931ULAMuJx6xLf7KQDOL7CxD+Sa05FktTDrtLaSy53uj01ll3Zf17JdKsriER248oS55GBzg0CfCTjEneAQ==",
"peerDependencies": {
"@codemirror/commands": "6.x.x",
"@codemirror/language": "6.x.x",
"@codemirror/search": "6.x.x",
"@codemirror/state": "6.x.x",
"@codemirror/view": "6.x.x"
}
},
"node_modules/crelt": {
"version": "1.0.6",
"resolved": "https://registry.npmjs.org/crelt/-/crelt-1.0.6.tgz",
"integrity": "sha512-VQ2MBenTq1fWZUH9DJNGti7kKv6EeAuYr3cLwxUWhIu1baTaXh4Ib5W2CqHVqib4/MqbYGJqiL3Zb8GJZr3l4g=="
},
"node_modules/esbuild": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.0.tgz",
"integrity": "sha512-sNR9MHpXSUV/XB4zmsFKN+QgVG82Cc7+/aaxJ8Adi8hyOac+EXptIp45QBPaVyX3N70664wRbTcLTOemCAnyqw==",
"hasInstallScript": true,
"bin": {
"esbuild": "bin/esbuild"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.28.0",
"@esbuild/android-arm": "0.28.0",
"@esbuild/android-arm64": "0.28.0",
"@esbuild/android-x64": "0.28.0",
"@esbuild/darwin-arm64": "0.28.0",
"@esbuild/darwin-x64": "0.28.0",
"@esbuild/freebsd-arm64": "0.28.0",
"@esbuild/freebsd-x64": "0.28.0",
"@esbuild/linux-arm": "0.28.0",
"@esbuild/linux-arm64": "0.28.0",
"@esbuild/linux-ia32": "0.28.0",
"@esbuild/linux-loong64": "0.28.0",
"@esbuild/linux-mips64el": "0.28.0",
"@esbuild/linux-ppc64": "0.28.0",
"@esbuild/linux-riscv64": "0.28.0",
"@esbuild/linux-s390x": "0.28.0",
"@esbuild/linux-x64": "0.28.0",
"@esbuild/netbsd-arm64": "0.28.0",
"@esbuild/netbsd-x64": "0.28.0",
"@esbuild/openbsd-arm64": "0.28.0",
"@esbuild/openbsd-x64": "0.28.0",
"@esbuild/openharmony-arm64": "0.28.0",
"@esbuild/sunos-x64": "0.28.0",
"@esbuild/win32-arm64": "0.28.0",
"@esbuild/win32-ia32": "0.28.0",
"@esbuild/win32-x64": "0.28.0"
}
},
"node_modules/style-mod": {
"version": "4.1.3",
"resolved": "https://registry.npmjs.org/style-mod/-/style-mod-4.1.3.tgz",
"integrity": "sha512-i/n8VsZydrugj3Iuzll8+x/00GH2vnYsk1eomD8QiRrSAeW6ItbCQDtfXCeJHd0iwiNagqjQkvpvREEPtW3IoQ=="
},
"node_modules/w3c-keyname": {
"version": "2.2.8",
"resolved": "https://registry.npmjs.org/w3c-keyname/-/w3c-keyname-2.2.8.tgz",
"integrity": "sha512-dpojBhNsCNN7T82Tm7k26A6G9ML3NkhDsnw9n/eoxSRlVBB4CEtIQ/KTCLI2Fwf3ataSXRhYFkQi3SlnFwPvPQ=="
}
}
}
+36
View File
@@ -0,0 +1,36 @@
{
"name": "editor",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "node --test editor-core.test.mjs",
"build": "node build.mjs"
},
"keywords": [],
"author": "",
"license": "ISC",
"dependencies": {
"@lezer/highlight": "^1.2.3",
"@codemirror/autocomplete": "^6.20.3",
"@codemirror/commands": "^6.10.3",
"@codemirror/lang-cpp": "^6.0.3",
"@codemirror/lang-css": "^6.3.1",
"@codemirror/lang-go": "^6.0.1",
"@codemirror/lang-html": "^6.4.11",
"@codemirror/lang-java": "^6.0.2",
"@codemirror/lang-javascript": "^6.2.5",
"@codemirror/lang-json": "^6.0.2",
"@codemirror/lang-markdown": "^6.5.0",
"@codemirror/lang-python": "^6.2.1",
"@codemirror/lang-rust": "^6.0.2",
"@codemirror/lang-sql": "^6.10.0",
"@codemirror/lang-yaml": "^6.1.3",
"@codemirror/language": "^6.12.3",
"@codemirror/legacy-modes": "^6.5.3",
"@codemirror/state": "^6.6.0",
"@codemirror/view": "^6.43.0",
"@replit/codemirror-vim": "^6.3.0",
"esbuild": "^0.28.0"
}
}
+411
View File
@@ -0,0 +1,411 @@
import { EditorView, keymap, lineNumbers, highlightActiveLine, highlightActiveLineGutter } from "@codemirror/view";
import { EditorState } from "@codemirror/state";
import { defaultKeymap, history, historyKeymap } from "@codemirror/commands";
import {
bracketMatching,
HighlightStyle,
StreamLanguage,
syntaxHighlighting,
} from "@codemirror/language";
import {
autocompletion,
completionKeymap,
closeBrackets,
closeBracketsKeymap,
} from "@codemirror/autocomplete";
import { html } from "@codemirror/lang-html";
import { markdown } from "@codemirror/lang-markdown";
import { javascript } from "@codemirror/lang-javascript";
import { python } from "@codemirror/lang-python";
import { json } from "@codemirror/lang-json";
import { css } from "@codemirror/lang-css";
import { sql } from "@codemirror/lang-sql";
import { yaml } from "@codemirror/lang-yaml";
import { go } from "@codemirror/lang-go";
import { rust } from "@codemirror/lang-rust";
import { java } from "@codemirror/lang-java";
import { cpp } from "@codemirror/lang-cpp";
import { shell } from "@codemirror/legacy-modes/mode/shell";
import { toml } from "@codemirror/legacy-modes/mode/toml";
import { vim, Vim, getCM } from "@replit/codemirror-vim";
import { tags, highlightTree } from "@lezer/highlight";
// Stable semantic styles instead of selectors for generated Lezer class names.
// CSS variables follow the selected Wiki theme without rebuilding the document.
const pagesWikiHighlightStyle = HighlightStyle.define([
{tag: tags.heading, class: "pw-syntax-heading"},
{tag: [tags.link, tags.url], class: "pw-syntax-link"},
{tag: tags.emphasis, class: "pw-syntax-emphasis"},
{tag: tags.strong, class: "pw-syntax-strong"},
{tag: [tags.keyword, tags.operatorKeyword], class: "pw-syntax-keyword"},
{tag: [tags.tagName, tags.typeName, tags.monospace], class: "pw-syntax-type"},
{tag: [tags.string, tags.special(tags.string)], class: "pw-syntax-string"},
{tag: [tags.number, tags.bool, tags.null, tags.attributeName], class: "pw-syntax-number"},
{tag: tags.comment, class: "pw-syntax-comment"},
{tag: [tags.variableName, tags.propertyName], class: "pw-syntax-variable"},
{tag: tags.function(tags.variableName), class: "pw-syntax-function"},
{tag: tags.operator, class: "pw-syntax-operator"},
{tag: tags.invalid, class: "pw-syntax-invalid"},
]);
const pagesWikiTheme = EditorView.theme({
".cm-selectionBackground": {
backgroundColor: "var(--editor-selection) !important",
},
"&.cm-focused .cm-selectionBackground": {
backgroundColor: "var(--editor-selection) !important",
},
".cm-selectionLayer .cm-selectionBackground": {
backgroundColor: "var(--editor-selection) !important",
},
".cm-content ::selection": {
backgroundColor: "var(--editor-selection) !important",
color: "var(--fg) !important",
},
});
let vimMode = "INSERT";
const languageAliases = {js:'javascript', ts:'typescript', py:'python', sh:'shell', bash:'shell', zsh:'shell', yml:'yaml', md:'markdown', c:'cpp', 'c++':'cpp', xml:'html', htm:'html'};
function languageExtension(mode) {
mode = String(mode || "").toLowerCase();
switch (languageAliases[mode] || mode) {
case "html":
return html();
case "markdown":
return markdown({codeLanguages: info => {
const mode = info.trim().split(/\s+/)[0].toLowerCase();
if (['md','markdown'].includes(mode)) return null;
const support = languageExtension(mode);
return support?.language || support;
}});
case "javascript":
return javascript({ jsx: true });
case "typescript":
return javascript({ jsx: true, typescript: true });
case "python":
return python();
case "json":
return json();
case "css":
return css();
case "sql":
return sql();
case "yaml":
return yaml();
case "go":
return go();
case "rust":
return rust();
case "java":
return java();
case "cpp":
return cpp();
case "shell":
return StreamLanguage.define(shell);
case "toml":
return StreamLanguage.define(toml);
default:
return null;
}
}
function attachMinimap(view, minimap, text) {
if (!minimap) {
return;
}
const lines = text.split("\n");
const count = Math.max(lines.length, 1);
const step = Math.max(1, Math.ceil(count / 2000));
for (let index = 0; index < count; index += step) {
const line = document.createElement("span");
line.className = "content-code-minimap-line";
line.style.top = (index / count * 100) + "%";
const length = (lines[index] || "").trimEnd().length;
line.style.width = Math.min(96, Math.max(3, length * 1.4)) + "%";
minimap.appendChild(line);
}
const viewport = document.createElement("span");
viewport.className = "content-code-minimap-viewport";
minimap.appendChild(viewport);
const scroller = view.scrollDOM;
const update = () => {
const scrollHeight = Math.max(scroller.scrollHeight, 1);
viewport.style.top =
(scroller.scrollTop / scrollHeight * 100) + "%";
viewport.style.height =
Math.min(100, scroller.clientHeight / scrollHeight * 100) + "%";
};
scroller.addEventListener("scroll", update, { passive: true });
new ResizeObserver(update).observe(scroller);
minimap.addEventListener("pointerdown", (event) => {
const rect = minimap.getBoundingClientRect();
const ratio = Math.max(
0,
Math.min(1, (event.clientY - rect.top) / rect.height),
);
scroller.scrollTop =
ratio * scroller.scrollHeight - scroller.clientHeight / 2;
});
update();
}
function setEditorMode(mode) {
vimMode = mode;
document.body.dataset.editorMode = mode.toLowerCase();
const el =
document.getElementById("editor-vim-mode");
if (!el) {
return;
}
const cls =
"mode-" + mode.toLowerCase();
el.innerHTML =
"<strong class=\"" + cls + "\">" + mode + "</strong>";
}
function markdownCompletions(context) {
const word =
context.matchBefore(/[A-Za-z#`\-\[\]\{\}:\/]*/);
if (!word || (word.from === word.to && !context.explicit)) {
return null;
}
if (!context.explicit && !word.text.startsWith("{{")) return null;
return {
from: word.from,
options: [
{ label: "# ", type: "keyword", apply: "# " },
{ label: "## ", type: "keyword", apply: "## " },
{ label: "### ", type: "keyword", apply: "### " },
{ label: "- ", type: "keyword", apply: "- " },
{ label: "```text", type: "snippet", apply: "```text\n\n```" },
{ label: "```go", type: "snippet", apply: "```go\n\n```" },
{ label: "```bash", type: "snippet", apply: "```bash\n\n```" },
{ label: "[link](url)", type: "snippet", apply: "[text](url)" },
{ label: "{{children}}", type: "function", apply: "{{children}}", boost: 5 },
{ label: "{{children:/}}", type: "function", apply: "{{children:/}}" },
{ label: "{{children:/ exclude=/draft}}", type: "function", apply: "{{children:/ exclude=/draft}}" },
].map(option => option.label.startsWith("{{") ? {...option, apply(view, completion, from, to) {
// Existing close-bracket text is part of this macro replacement.
const tail = view.state.sliceDoc(to, to + 2).match(/^\}{1,2}/)?.[0] || "";
view.dispatch({changes:{from,to:to+tail.length,insert:option.label},selection:{anchor:from+option.label.length}});
}} : option),
};
}
function installVimCommands() {
if (!Vim || !Vim.defineEx || window.PagesWikiVimCommandsInstalled) {
return;
}
window.PagesWikiVimCommandsInstalled = true;
const save =
() => {
if (window.PagesEditor && window.PagesEditor.saveNow) {
window.PagesEditor.saveNow();
}
};
const quit =
() => {
location.href =
window.PagesWikiCurrentPath || "/";
};
const saveQuit = async () => {
if (await window.PagesEditor?.saveNow()) quit();
};
// Register both canonical and short names.
Vim.defineEx("write", "w", save);
Vim.defineEx("w", "w", save);
Vim.defineEx("quit", "q", quit);
Vim.defineEx("q", "q", quit);
Vim.defineEx("wq", "wq", saveQuit);
Vim.defineEx("x", "x", saveQuit);
}
function attachVimModeTracking(view) {
// Observe actual Vim transitions, including A/I/c/operator motions and
// command-line searches, rather than guessing the mode from individual keys.
getCM(view)?.on("vim-mode-change", event => {
const mode = String(event.mode || "normal").toUpperCase();
setEditorMode(mode);
view.dom.classList.toggle("pageswiki-vim-visual", mode === "VISUAL");
});
}
function attachStatusBar(view) {
const cursorEl =
document.getElementById("editor-cursor");
function updateCursor() {
if (!cursorEl) {
return;
}
const head =
view.state.selection.main.head;
const line =
view.state.doc.lineAt(head);
cursorEl.textContent =
"Ln " +
line.number +
", Col " +
String(head - line.from + 1);
}
updateCursor();
setEditorMode(vimMode || "INSERT");
view.dom.addEventListener("keyup", updateCursor);
view.dom.addEventListener("mouseup", updateCursor);
view.dom.addEventListener("focusin", updateCursor);
}
window.PagesWikiEditor = {
highlightCodeBlock(code, mode) {
mode = String(mode || '').toLowerCase();
const language = languageExtension(mode);
const text = code.textContent;
// Projection only: never change canonical HTML, unlabelled/plain text,
// or large samples. Large code files retain the windowed viewer.
if (!language || text.length > 32768) return false;
const parser = (language.language || language).parser;
const fragment = document.createDocumentFragment();
let offset = 0;
highlightTree(parser.parse(text), pagesWikiHighlightStyle, (from, to, classes) => {
if (from > offset) fragment.append(document.createTextNode(text.slice(offset, from)));
const span = document.createElement('span'); span.className = classes;
span.textContent = text.slice(from, to); fragment.append(span); offset = to;
});
fragment.append(document.createTextNode(text.slice(offset)));
code.replaceChildren(fragment);
return true;
},
createViewer(target, text, mode, minimap) {
const extensions = [
lineNumbers(),
syntaxHighlighting(pagesWikiHighlightStyle),
pagesWikiTheme,
EditorState.readOnly.of(true),
EditorView.editable.of(false),
];
const language = languageExtension(mode);
if (language) {
extensions.push(language);
}
const view = new EditorView({
state: EditorState.create({
doc: text,
extensions,
}),
parent: target,
});
attachMinimap(view, minimap, text);
return view;
},
create(target, initialText, mode, options = {}) {
const mobile = window.matchMedia("(pointer: coarse) and (hover: none)").matches;
const keymapName = mobile ? "default" : String(options.keymap || "default").toLowerCase();
const extensions = [
lineNumbers(),
highlightActiveLine(),
highlightActiveLineGutter(),
history(),
bracketMatching(),
closeBrackets(),
pagesWikiTheme,
EditorView.lineWrapping,
EditorView.updateListener.of((update) => {
if (update.docChanged) {
schedulePagesWikiPreview(update.view);
}
}),
];
if (keymapName === "vim") {
installVimCommands();
extensions.push(vim());
setEditorMode("NORMAL");
} else {
setEditorMode("INSERT");
}
extensions.push(
keymap.of([
...defaultKeymap,
...historyKeymap,
...completionKeymap,
...closeBracketsKeymap,
])
);
if (options.syntax !== false) {
extensions.push(syntaxHighlighting(pagesWikiHighlightStyle));
}
if (options.autocomplete !== false) {
extensions.push(
autocompletion({
activateOnTyping: true,
override: mode === "markdown"
? [markdownCompletions]
: undefined,
})
);
}
const language = languageExtension(mode);
if (language) {
extensions.push(language);
}
if (mode === "html" || mode === "markdown") {
extensions.push(EditorState.languageData.of(() => [{closeBrackets:{brackets:["(", "[", "'", '"', "`"]}}]));
}
const view =
new EditorView({
state: EditorState.create({
doc: initialText,
extensions,
}),
parent: target,
});
attachStatusBar(view);
if (keymapName === "vim") {
attachVimModeTracking(view);
}
return view;
},
};
function schedulePagesWikiPreview(view) {
const textarea = document.getElementById("editor");
if (textarea) {
textarea.value =
view.state.doc.toString();
textarea.dispatchEvent(new Event("input", {bubbles:true}));
}
}
+17
View File
@@ -0,0 +1,17 @@
{
"id": "org.pageswiki.excalidraw-viewer",
"type": "projection",
"version": "0.1.0",
"pageswiki": ">=0.1.0",
"entry_point": "/_/adaptor/excalidraw/",
"media_types": ["application/json"],
"extensions": [".excalidraw"],
"capabilities": ["revision.read"],
"outbound_hosts": [],
"limits": {
"input_bytes": 2097152,
"elements": 5000,
"points": 100000
},
"health_probe": "/_/adaptor/excalidraw/health"
}
+12
View File
@@ -0,0 +1,12 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Excalidraw Viewer</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.jsx"></script>
</body>
</html>
+19
View File
@@ -0,0 +1,19 @@
server {
listen 8080;
server_name _;
root /usr/share/nginx/html;
location = /health {
default_type text/plain;
return 200 "ok\n";
}
location / {
try_files $uri $uri/ /index.html;
add_header Cache-Control "public, max-age=31536000, immutable" always;
}
location = /index.html {
add_header Cache-Control "no-cache" always;
}
}
File diff suppressed because it is too large. Load diff
+27
View File
@@ -0,0 +1,27 @@
{
"name": "pageswiki-excalidraw-viewer",
"private": true,
"version": "0.1.0",
"type": "module",
"scripts": {
"build": "vite build"
},
"dependencies": {
"@excalidraw/excalidraw": "0.18.1",
"react": "18.3.1",
"react-dom": "18.3.1"
},
"devDependencies": {
"@vitejs/plugin-react": "5.1.4",
"vite": "7.3.6"
},
"overrides": {
"@excalidraw/excalidraw": {
"nanoid": "3.3.18"
},
"@excalidraw/mermaid-to-excalidraw": {
"nanoid": "5.1.16"
},
"lodash-es": "4.18.1"
}
}
+112
View File
@@ -0,0 +1,112 @@
import React, { useEffect, useState } from "react"
import { createRoot } from "react-dom/client"
import { Excalidraw } from "@excalidraw/excalidraw"
import "@excalidraw/excalidraw/index.css"
import "./viewer.css"
const MAX_SCENE_BYTES = 2 * 1024 * 1024
const notifyParent = (type, detail = "") => {
window.parent.postMessage({ source: "pageswiki-excalidraw", type, detail }, window.location.origin)
}
const sceneURL = () => {
const value = new URLSearchParams(window.location.search).get("url") || ""
const parsed = new URL(value, window.location.origin)
if (parsed.origin !== window.location.origin || !parsed.pathname.startsWith("/v/raw/rev:")) {
throw new Error("An immutable same-origin Revision URL is required")
}
return parsed.href
}
const readBoundedText = async response => {
const declared = Number(response.headers.get("content-length") || 0)
if (declared > MAX_SCENE_BYTES) throw new Error("Scene exceeds the 2 MiB viewer limit")
if (!response.body) return response.text()
const reader = response.body.getReader()
const chunks = []
let size = 0
while (true) {
const { done, value } = await reader.read()
if (done) break
size += value.byteLength
if (size > MAX_SCENE_BYTES) {
await reader.cancel()
throw new Error("Scene exceeds the 2 MiB viewer limit")
}
chunks.push(value)
}
const joined = new Uint8Array(size)
let offset = 0
chunks.forEach(chunk => {
joined.set(chunk, offset)
offset += chunk.byteLength
})
return new TextDecoder().decode(joined)
}
function Viewer() {
const [scene, setScene] = useState(null)
const [error, setError] = useState("")
useEffect(() => {
const controller = new AbortController()
;(async () => {
try {
const response = await fetch(sceneURL(), {
credentials: "same-origin",
cache: "force-cache",
signal: controller.signal,
headers: { Accept: "application/json" }
})
if (!response.ok) throw new Error(`Revision fetch failed: HTTP ${response.status}`)
const loaded = JSON.parse(await readBoundedText(response))
if (loaded?.type !== "excalidraw" || !Array.isArray(loaded.elements)) {
throw new Error("Invalid Excalidraw scene")
}
setScene(loaded)
} catch (reason) {
if (reason?.name === "AbortError") return
const message = reason instanceof Error ? reason.message : String(reason)
setError(message)
notifyParent("error", message)
}
})()
return () => controller.abort()
}, [])
if (error) return <main className="viewer-status viewer-error">{error}</main>
if (!scene) return <main className="viewer-status">Loading Excalidraw…</main>
return (
<main className="viewer-shell">
<Excalidraw
initialData={{
elements: scene.elements,
appState: scene.appState,
files: scene.files
}}
viewModeEnabled
zenModeEnabled
langCode={(navigator.language || "en").toLowerCase().startsWith("ja") ? "ja-JP" : "en"}
UIOptions={{
canvasActions: {
changeViewBackgroundColor: false,
clearCanvas: false,
export: false,
loadScene: false,
saveAsImage: false,
saveToActiveFile: false,
toggleTheme: true
}
}}
excalidrawAPI={api => {
requestAnimationFrame(() => api.scrollToContent(undefined, { fitToContent: true }))
notifyParent("ready")
}}
/>
</main>
)
}
createRoot(document.getElementById("root")).render(<Viewer />)
+24
View File
@@ -0,0 +1,24 @@
html,
body,
#root,
.viewer-shell {
width: 100%;
height: 100%;
margin: 0;
overflow: hidden;
}
.viewer-status {
display: grid;
place-items: center;
width: 100%;
height: 100%;
margin: 0;
font: 14px/1.5 system-ui, sans-serif;
color: #555;
background: #fff;
}
.viewer-error {
color: #9d2020;
}
+14
View File
@@ -0,0 +1,14 @@
import { defineConfig } from "vite"
import react from "@vitejs/plugin-react"
export default defineConfig({
base: "./",
plugins: [react()],
define: {
"process.env.IS_PREACT": JSON.stringify("false")
},
build: {
sourcemap: false,
chunkSizeWarningLimit: 2500
}
})
+9
View File
@@ -0,0 +1,9 @@
@font-face { font-family: PagesWikiHack; src: url('./HackNerdFont-Regular.ttf') format('truetype'); font-weight:400; font-style:normal; font-display:block; }
:root { color-scheme: dark; font-family: PagesWikiHack, ui-monospace, monospace; }
* { box-sizing: border-box; }
html, body { width: 100%; height: 100%; margin: 0; overflow: hidden; background: #282c34; }
body { display: block; }
main { position: relative; width: 100%; height: 100%; }
#vim-canvas { display: block; width: 100%; height: 100%; }
#vim-input { position: fixed; width: 1px; height: 1px; opacity: 0; pointer-events: none; }
#error { position: fixed; inset: 1rem 1rem auto; z-index: 2; padding: 1rem; white-space: pre-wrap; color: #ffd7d7; background: #721c24; border-radius: 6px; }
+110
View File
@@ -0,0 +1,110 @@
import {VimWasm, checkBrowserCompatibility} from "./vendor/vimwasm.js"
const errorBox = document.getElementById("error")
const fail = error => {
errorBox.hidden = false
errorBox.textContent = error instanceof Error ? error.message : String(error)
}
const compatibilityError = checkBrowserCompatibility()
if (compatibilityError || !crossOriginIsolated) {
fail(compatibilityError || "Cross-origin isolation is not active. Check COOP/COEP headers.")
} else {
try {
const response = await fetch("./api/session", {cache: "no-store", credentials: "same-origin"})
if (!response.ok) throw new Error(`Workspace request failed: HTTP ${response.status}`)
const workspace = await response.json()
const files = {}
const directories = new Set(["/work"])
for (const file of workspace.files || []) {
const path = `/work/${file.path}`
files[path] = file.content
const parts = path.split("/").slice(1, -1)
let directory = ""
for (const part of parts) {
directory += `/${part}`
directories.add(directory)
}
}
const vimrcResponse = await fetch("./vimrc", {cache: "no-store"})
if (!vimrcResponse.ok) throw new Error(`vimrc request failed: HTTP ${vimrcResponse.status}`)
// The pinned runtime creates files with wx+ and silently skips the
// bundled /.vim/vimrc. Use a new path and an explicit -u argument.
files["/pageswiki.vimrc"] = await vimrcResponse.text()
// Font metrics must be final before Vim sizes its canvas/cell grid.
await document.fonts.load('16px PagesWikiHack')
const pluginResponse = await fetch('./pageswiki.vim', {cache: 'no-store'})
if (!pluginResponse.ok) throw new Error(`Plugin request failed: HTTP ${pluginResponse.status}`)
files['/.vim/plugin/pageswiki.vim'] = await pluginResponse.text()
directories.add('/.vim')
directories.add('/.vim/plugin')
const safeMode = new URLSearchParams(location.search).get('safe') === '1'
const settings = safeMode ? {} : (workspace.vim_settings || {})
const fontSize = Number.isInteger(settings.font_size) && settings.font_size >= 10 && settings.font_size <= 32 ? settings.font_size : 16
let userConfig = `\nset guifont=PagesWikiHack:h${fontSize}\n`
if (settings.enabled) {
if (settings.colorscheme) {
directories.add('/pageswiki-config')
directories.add('/pageswiki-config/colors')
files['/pageswiki-config/colors/pageswiki_custom.vim'] = settings.colorscheme
if (/^[A-Za-z0-9_-]+\.vim$/.test(settings.color_name || '')) {
files[`/pageswiki-config/colors/${settings.color_name}`] = settings.colorscheme
}
userConfig += 'set runtimepath^=/pageswiki-config\n'
}
if (settings.vimrc) {
files['/pageswiki-user.vimrc'] = settings.vimrc
userConfig += 'source /pageswiki-user.vimrc\n'
}
if (settings.colorscheme) userConfig += 'colorscheme pageswiki_custom\n'
}
files['/pageswiki.vimrc'] += userConfig
const vim = new VimWasm({
canvas: document.getElementById("vim-canvas"),
input: document.getElementById("vim-input"),
workerScriptPath: "./vendor/vim.js"
})
vim.onVimInit = () => {
vim.focus()
}
vim.onVimExit = () => {
window.close()
document.body.textContent = "Vim exited. You may close this tab."
}
vim.onTitleUpdate = title => { document.title = `${title} · vim.wasm` }
vim.onError = fail
vim.onFileExport = async (fullpath, contents) => {
try {
const bytes = new Uint8Array(contents)
let binary = ""
for (let offset = 0; offset < bytes.length; offset += 0x8000) {
binary += String.fromCharCode(...bytes.subarray(offset, offset + 0x8000))
}
const save = await fetch("./api/save", {
method: "PUT",
credentials: "same-origin",
headers: {"Content-Type": "application/json"},
body: JSON.stringify({path: fullpath, content_base64: btoa(binary)})
})
if (!save.ok) throw new Error((await save.text()).trim() || `Save failed: HTTP ${save.status}`)
} catch (error) {
fail(error)
await vim.showError(String(error))
}
}
const paths = (workspace.files || []).map(file => `/work/${file.path}`).sort()
const initial = `/work/${workspace.initial}`
const cmdArgs = ['-u', '/pageswiki.vimrc', initial, ...paths.filter(path => path !== initial)]
vim.start({
dirs: Array.from(directories).sort((a, b) => a.split("/").length - b.split("/").length),
files,
clipboard: Boolean(navigator.clipboard),
cmdArgs
})
} catch (error) {
fail(error)
}
}
+17
View File
@@ -0,0 +1,17 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Pages Wiki · vim.wasm</title>
<link rel="stylesheet" href="app.css">
</head>
<body>
<main>
<canvas id="vim-canvas"></canvas>
<input id="vim-input" autocomplete="off" autofocus aria-label="Vim keyboard input">
</main>
<div id="error" role="alert" hidden></div>
<script type="module" src="app.js"></script>
</body>
</html>
+12
View File
@@ -0,0 +1,12 @@
" Small, bundled, pure Vimscript workspace navigation. No shell or network.
if exists('g:loaded_pageswiki') | finish | endif
let g:loaded_pageswiki = 1
set path=/work/**
set wildignore+=*.swp,*.bak
command! Files call feedkeys(':find ', 'n')
nnoremap <silent> <leader>f :Files<CR>
nnoremap <leader>b :ls<CR>:buffer<Space>
nnoremap <silent> ]b :bnext<CR>
nnoremap <silent> [b :bprevious<CR>
nnoremap <silent> <leader>w :write<CR>
nnoremap <silent> <leader>/ :nohlsearch<CR>
+55
View File
@@ -0,0 +1,55 @@
set nocompatible
set runtimepath^=/.vim
set number
set guifont=PagesWikiHack:h16
set background=dark
set cursorline
set laststatus=2
set showcmd
set ruler
set scrolloff=4
set sidescrolloff=4
set noswapfile
set nobackup
set nowritebackup
set display+=lastline
set linebreak
set listchars=tab:»\ ,trail:·,extends:›,precedes:‹
let mapleader = " "
set hidden
set encoding=utf-8
set fileencoding=utf-8
set mouse=a
set backspace=indent,eol,start
set wildmenu
set wildmode=longest:full,full
set incsearch
set hlsearch
set ignorecase
set smartcase
set autoindent
set expandtab
set shiftwidth=4
set softtabstop=4
set tabstop=4
syntax enable
filetype plugin indent on
highlight Normal guifg=#e6edf3 guibg=#1e222a
highlight LineNr guifg=#8994a6 guibg=#1e222a
highlight CursorLine guibg=#2b3240
highlight StatusLine guifg=#e6edf3 guibg=#35415a gui=NONE
" Close the hosted Vim session with :q even when the argument list contains
" more files. Unsaved buffers still require :q! explicitly.
cnoreabbrev <expr> q getcmdtype() ==# ':' && getcmdline() ==# 'q' ? 'qa' : 'q'
cnoreabbrev <expr> q! getcmdtype() ==# ':' && getcmdline() ==# 'q!' ? 'qa!' : 'q!'
" Pages Wiki writes each exported buffer as a new Revision.
augroup pageswiki
autocmd!
autocmd BufWritePost /work/* execute 'export ' . expand('%:p')
augroup END
" Pure Vimscript plugins are preloaded into /.vim/plugin by app.js.
" vim-plug installation and fzf's native executable do not work in MEMFS.
" Vendoring fzf.vim alone does not remove that native dependency.
+21
View File
@@ -0,0 +1,21 @@
{
"name": "pageswiki-vimwasm-adapter",
"version": "0.1.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "pageswiki-vimwasm-adapter",
"version": "0.1.0",
"dependencies": {
"vim-wasm": "0.0.13"
}
},
"node_modules/vim-wasm": {
"version": "0.0.13",
"resolved": "https://registry.npmjs.org/vim-wasm/-/vim-wasm-0.0.13.tgz",
"integrity": "sha512-2EkJ4OfMgvA/bURJeu43wqIQ4tpCJymst++Y4R/AKuQlnrvdI4hTpG2jf0yL5TrLt1ui12leLRghrwPOybyu8w==",
"license": "Vim"
}
}
}
+8
View File
@@ -0,0 +1,8 @@
{
"name": "pageswiki-vimwasm-adapter",
"private": true,
"version": "0.1.0",
"dependencies": {
"vim-wasm": "0.0.13"
}
}
+515
View File
@@ -0,0 +1,515 @@
package main
import (
"bytes"
"crypto/rand"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"html/template"
"io"
"log"
"net/http"
"net/http/httputil"
"net/url"
"os"
"path/filepath"
"strings"
"sync"
"time"
)
type gatewaySession struct {
VimSettings json.RawMessage
Token string
PluginID string
Filename string
ContentType string
ExpiresAtMS int64
Workspace string
Files map[string]gatewayFile
LastError string
}
type gatewayFile struct {
SemanticPath string
RevisionUUID string
LastContent string
}
type gateway struct {
origin string
client *http.Client
mu sync.Mutex
sessions map[string]gatewaySession
codeProxy *httputil.ReverseProxy
excalidrawProxy *httputil.ReverseProxy
}
func main() {
origin := strings.TrimRight(os.Getenv("PAGESWIKI_ORIGIN"), "/")
if origin == "" {
origin = "http://app:3000"
}
codeOrigin := os.Getenv("CODE_SERVER_ORIGIN")
if codeOrigin == "" {
codeOrigin = "http://code-server:8080"
}
codeURL, err := url.Parse(codeOrigin)
if err != nil {
log.Fatal(err)
}
proxy := httputil.NewSingleHostReverseProxy(codeURL)
originalDirector := proxy.Director
proxy.Director = func(req *http.Request) {
req.URL.Path = strings.TrimPrefix(req.URL.Path, "/code")
if req.URL.Path == "" {
req.URL.Path = "/"
}
originalDirector(req)
}
excalidrawOrigin := os.Getenv("EXCALIDRAW_VIEWER_ORIGIN")
if excalidrawOrigin == "" {
excalidrawOrigin = "http://excalidraw-viewer:8080"
}
excalidrawURL, err := url.Parse(excalidrawOrigin)
if err != nil {
log.Fatal(err)
}
excalidrawProxy := httputil.NewSingleHostReverseProxy(excalidrawURL)
excalidrawDirector := excalidrawProxy.Director
excalidrawProxy.Director = func(req *http.Request) {
req.URL.Path = strings.TrimPrefix(req.URL.Path, "/excalidraw")
if req.URL.Path == "" {
req.URL.Path = "/"
}
excalidrawDirector(req)
}
g := &gateway{origin: origin, client: &http.Client{Timeout: 30 * time.Second}, sessions: map[string]gatewaySession{}, codeProxy: proxy, excalidrawProxy: excalidrawProxy}
mux := http.NewServeMux()
mux.HandleFunc("/health", func(w http.ResponseWriter, _ *http.Request) { w.Write([]byte("ok")) })
mux.HandleFunc("/open", g.open)
mux.HandleFunc("/code/open", g.open)
mux.HandleFunc("/vim/open", g.open)
mux.HandleFunc("/edit", g.edit)
mux.HandleFunc("/save", g.save)
mux.HandleFunc("/code/", g.code)
mux.HandleFunc("/vim/api/session", g.vimSession)
mux.HandleFunc("/vim/api/save", g.vimSave)
vimFiles := http.FileServer(http.Dir("/vimwasm"))
vimHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/vim/" {
data, err := os.ReadFile("/vimwasm/index.html")
if err != nil {
http.Error(w, "vim.wasm frontend is unavailable", http.StatusServiceUnavailable)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
_, _ = w.Write(data)
return
}
http.StripPrefix("/vim/", vimFiles).ServeHTTP(w, r)
})
mux.Handle("/vim/", isolationHeaders(vimHandler))
mux.Handle("/excalidraw/", excalidrawHeaders(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
g.excalidrawProxy.ServeHTTP(w, r)
})))
log.Printf("adapter gateway origin=%s listen=:3000", origin)
log.Fatal(http.ListenAndServe(":3000", mux))
}
func excalidrawHeaders(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Security-Policy", "default-src 'self'; script-src 'self' 'unsafe-eval' 'wasm-unsafe-eval'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob:; font-src 'self' data:; worker-src 'self' blob:; connect-src 'self'; frame-ancestors 'self'; object-src 'none'; base-uri 'none'")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.Header().Set("Referrer-Policy", "no-referrer")
next.ServeHTTP(w, r)
})
}
func isolationHeaders(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Cross-Origin-Opener-Policy", "same-origin")
w.Header().Set("Cross-Origin-Embedder-Policy", "require-corp")
w.Header().Set("Cross-Origin-Resource-Policy", "same-origin")
w.Header().Set("Origin-Agent-Cluster", "?1")
next.ServeHTTP(w, r)
})
}
func randomID() string {
b := make([]byte, 24)
_, _ = rand.Read(b)
return hex.EncodeToString(b)
}
func ensureCoderDirectory(root, directory string) error {
relative, err := filepath.Rel(root, directory)
if err != nil || relative == ".." || strings.HasPrefix(relative, ".."+string(filepath.Separator)) {
return fmt.Errorf("workspace directory escapes root")
}
current := root
for _, component := range strings.Split(relative, string(filepath.Separator)) {
if component == "" || component == "." {
continue
}
current = filepath.Join(current, component)
if err := os.Mkdir(current, 0700); err != nil && !os.IsExist(err) {
return err
}
if err := os.Chown(current, 1000, 1000); err != nil {
return err
}
}
return nil
}
func (g *gateway) request(method, path, bearer string, input any, output any) error {
var body io.Reader
if input != nil {
data, _ := json.Marshal(input)
body = bytes.NewReader(data)
}
req, err := http.NewRequest(method, g.origin+path, body)
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
if bearer != "" {
req.Header.Set("Authorization", "Bearer "+bearer)
}
res, err := g.client.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
if res.StatusCode < 200 || res.StatusCode >= 300 {
message, _ := io.ReadAll(io.LimitReader(res.Body, 4096))
return fmt.Errorf("Pages Wiki HTTP %d: %s", res.StatusCode, strings.TrimSpace(string(message)))
}
return json.NewDecoder(res.Body).Decode(output)
}
func (g *gateway) open(w http.ResponseWriter, r *http.Request) {
publicPrefix := strings.TrimRight(strings.TrimSpace(r.Header.Get("X-Forwarded-Prefix")), "/")
ticket := strings.TrimSpace(r.URL.Query().Get("ticket"))
if ticket == "" {
http.Error(w, "ticket is required", 400)
return
}
var exchanged struct {
SessionToken string `json:"session_token"`
PluginID string `json:"plugin_id"`
Filename string `json:"filename"`
ContentType string `json:"content_type"`
ExpiresAtMS int64 `json:"expires_at_ms"`
}
if err := g.request("POST", "/_/api/v1/adapter/session/exchange", "",
map[string]string{"ticket": ticket}, &exchanged); err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
return
}
var workspaceData struct {
VimSettings json.RawMessage `json:"vim_settings"`
Files []struct {
Path string `json:"path"`
SemanticPath string `json:"semantic_path"`
RevisionUUID string `json:"revision_uuid"`
ContentBase64 string `json:"content_base64"`
} `json:"files"`
}
if err := g.request("GET", "/_/api/v1/adapter/session/workspace", exchanged.SessionToken, nil, &workspaceData); err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
return
}
id := randomID()
workspace := filepath.Join("/workspaces", id)
if err := os.MkdirAll(workspace, 0700); err != nil {
http.Error(w, err.Error(), 500)
return
}
_ = os.Chown(workspace, 1000, 1000)
files := make(map[string]gatewayFile, len(workspaceData.Files))
filename := "document.txt"
for index, source := range workspaceData.Files {
clean := filepath.Clean(filepath.FromSlash(source.Path))
if clean == "." || filepath.IsAbs(clean) || strings.HasPrefix(clean, ".."+string(filepath.Separator)) {
continue
}
data, decodeErr := base64.StdEncoding.DecodeString(source.ContentBase64)
if decodeErr != nil {
http.Error(w, "invalid workspace content", http.StatusBadGateway)
return
}
target := filepath.Join(workspace, clean)
if _, duplicate := files[clean]; duplicate {
http.Error(w, "workspace contains duplicate filenames", http.StatusConflict)
return
}
if err := ensureCoderDirectory(workspace, filepath.Dir(target)); err != nil {
http.Error(w, err.Error(), 500)
return
}
if err := os.WriteFile(target, data, 0600); err != nil {
http.Error(w, err.Error(), 500)
return
}
_ = os.Chown(target, 1000, 1000)
files[clean] = gatewayFile{SemanticPath: source.SemanticPath,
RevisionUUID: source.RevisionUUID, LastContent: source.ContentBase64}
if index == 0 {
filename = clean
}
}
if len(files) == 0 {
http.Error(w, "workspace contains no editable files", http.StatusBadGateway)
return
}
g.mu.Lock()
g.sessions[id] = gatewaySession{Token: exchanged.SessionToken, PluginID: exchanged.PluginID, Filename: filename,
ContentType: exchanged.ContentType, ExpiresAtMS: exchanged.ExpiresAtMS,
Workspace: workspace, Files: files, VimSettings: workspaceData.VimSettings}
g.mu.Unlock()
cookiePath := "/"
if publicPrefix != "" {
cookiePath = publicPrefix + "/"
}
http.SetCookie(w, &http.Cookie{Name: "pageswiki_adapter", Value: id, Path: cookiePath, HttpOnly: true, SameSite: http.SameSiteLaxMode, MaxAge: 8 * 3600})
if exchanged.PluginID == "vim.wasm" {
go g.expire(id)
http.Redirect(w, r, publicPrefix+"/vim/", http.StatusFound)
return
}
go g.watch(id)
http.Redirect(w, r, publicPrefix+"/code/?folder="+url.QueryEscape(workspace), http.StatusFound)
}
func (g *gateway) session(r *http.Request) (gatewaySession, bool) {
cookie, err := r.Cookie("pageswiki_adapter")
if err != nil {
return gatewaySession{}, false
}
g.mu.Lock()
defer g.mu.Unlock()
s, ok := g.sessions[cookie.Value]
if !ok || s.ExpiresAtMS < time.Now().UnixMilli() {
delete(g.sessions, cookie.Value)
return gatewaySession{}, false
}
return s, true
}
var editorTemplate = template.Must(template.New("editor").Parse(`<!doctype html>
<html><head><meta charset="utf-8"><title>{{.Filename}}</title>
<style>html,body,form{height:100%;margin:0}form{display:grid;grid-template-rows:auto 1fr}.bar{padding:.5rem;background:#20242b;color:#eee}textarea{resize:none;border:0;padding:1rem;font:14px/1.5 monospace;background:#11151a;color:#e8e8e8;outline:0}</style></head>
<body><form method="post" action="/save"><div class="bar"><strong>{{.Filename}}</strong> <button type="submit">Save as new Revision</button> {{.Status}}</div><textarea name="content" spellcheck="false">{{.Content}}</textarea></form></body></html>`))
func (g *gateway) edit(w http.ResponseWriter, r *http.Request) {
s, ok := g.session(r)
if !ok {
http.Error(w, "session expired", 401)
return
}
var content struct {
ContentBase64 string `json:"content_base64"`
}
if err := g.request("GET", "/_/api/v1/adapter/session/content", s.Token, nil, &content); err != nil {
http.Error(w, err.Error(), 502)
return
}
data, err := base64.StdEncoding.DecodeString(content.ContentBase64)
if err != nil {
http.Error(w, "invalid content", 502)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
_ = editorTemplate.Execute(w, map[string]any{"Filename": s.Filename, "Content": string(data), "Status": r.URL.Query().Get("status")})
}
func (g *gateway) save(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.Error(w, "method not allowed", 405)
return
}
s, ok := g.session(r)
if !ok {
http.Error(w, "session expired", 401)
return
}
r.Body = http.MaxBytesReader(w, r.Body, 65<<20)
if err := r.ParseForm(); err != nil {
http.Error(w, err.Error(), 400)
return
}
var output map[string]any
err := g.request("PUT", "/_/api/v1/adapter/session/content", s.Token, map[string]string{
"content_base64": base64.StdEncoding.EncodeToString([]byte(r.FormValue("content"))),
"message": "hosted editor: " + s.Filename,
}, &output)
if err != nil {
http.Error(w, err.Error(), http.StatusConflict)
return
}
http.Redirect(w, r, "/edit?status=saved", http.StatusFound)
}
func (g *gateway) code(w http.ResponseWriter, r *http.Request) {
if _, ok := g.session(r); !ok {
http.Error(w, "session expired", http.StatusUnauthorized)
return
}
g.codeProxy.ServeHTTP(w, r)
}
func (g *gateway) vimSession(w http.ResponseWriter, r *http.Request) {
session, ok := g.session(r)
if !ok || session.PluginID != "vim.wasm" {
http.Error(w, "session expired", http.StatusUnauthorized)
return
}
type vimFile struct {
Path string `json:"path"`
Content string `json:"content"`
}
files := make([]vimFile, 0, len(session.Files))
for relative, tracked := range session.Files {
data, err := base64.StdEncoding.DecodeString(tracked.LastContent)
if err != nil {
continue
}
files = append(files, vimFile{Path: filepath.ToSlash(relative), Content: string(data)})
}
w.Header().Set("Content-Type", "application/json")
w.Header().Set("Cache-Control", "no-store")
_ = json.NewEncoder(w).Encode(map[string]any{"initial": filepath.ToSlash(session.Filename), "files": files, "vim_settings": session.VimSettings})
}
func (g *gateway) vimSave(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPut {
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
session, ok := g.session(r)
if !ok || session.PluginID != "vim.wasm" {
http.Error(w, "session expired", http.StatusUnauthorized)
return
}
var input struct {
Path string `json:"path"`
ContentBase64 string `json:"content_base64"`
}
r.Body = http.MaxBytesReader(w, r.Body, 65<<20)
if err := json.NewDecoder(r.Body).Decode(&input); err != nil {
http.Error(w, "invalid JSON", http.StatusBadRequest)
return
}
clean := filepath.Clean(filepath.FromSlash(strings.TrimPrefix(input.Path, "/work/")))
tracked, exists := session.Files[clean]
if !exists {
http.Error(w, "file is outside the adapter workspace", http.StatusForbidden)
return
}
var output struct {
RevisionUUID string `json:"revision_uuid"`
}
err := g.request("PUT", "/_/api/v1/adapter/session/content", session.Token,
map[string]string{"content_base64": input.ContentBase64, "path": tracked.SemanticPath,
"revision_uuid": tracked.RevisionUUID, "message": "vim.wasm write: " + clean}, &output)
if err != nil {
http.Error(w, err.Error(), http.StatusConflict)
return
}
cookie, _ := r.Cookie("pageswiki_adapter")
g.mu.Lock()
current, exists := g.sessions[cookie.Value]
if exists {
updated := current.Files[clean]
updated.LastContent = input.ContentBase64
updated.RevisionUUID = output.RevisionUUID
current.Files[clean] = updated
g.sessions[cookie.Value] = current
}
g.mu.Unlock()
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]any{"revision_uuid": output.RevisionUUID})
}
func (g *gateway) expire(id string) {
g.mu.Lock()
session, ok := g.sessions[id]
g.mu.Unlock()
if !ok {
return
}
delay := time.Until(time.UnixMilli(session.ExpiresAtMS))
if delay > 0 {
timer := time.NewTimer(delay)
defer timer.Stop()
<-timer.C
}
g.mu.Lock()
session, ok = g.sessions[id]
delete(g.sessions, id)
g.mu.Unlock()
if ok {
if err := os.RemoveAll(session.Workspace); err != nil {
log.Printf("remove expired workspace %s: %v", session.Workspace, err)
}
}
}
func (g *gateway) watch(id string) {
ticker := time.NewTicker(2 * time.Second)
defer ticker.Stop()
defer func() {
g.mu.Lock()
session, ok := g.sessions[id]
delete(g.sessions, id)
g.mu.Unlock()
if ok {
if err := os.RemoveAll(session.Workspace); err != nil {
log.Printf("remove expired workspace %s: %v", session.Workspace, err)
}
}
}()
for range ticker.C {
g.mu.Lock()
session, ok := g.sessions[id]
g.mu.Unlock()
if !ok || session.ExpiresAtMS < time.Now().UnixMilli() {
return
}
for relative, tracked := range session.Files {
data, err := os.ReadFile(filepath.Join(session.Workspace, relative))
if err != nil {
continue
}
encoded := base64.StdEncoding.EncodeToString(data)
if encoded == tracked.LastContent {
continue
}
var output struct {
RevisionUUID string `json:"revision_uuid"`
}
err = g.request("PUT", "/_/api/v1/adapter/session/content", session.Token,
map[string]string{"content_base64": encoded, "path": tracked.SemanticPath,
"revision_uuid": tracked.RevisionUUID, "message": "code-server autosave: " + relative}, &output)
g.mu.Lock()
current, exists := g.sessions[id]
if exists {
currentFile := current.Files[relative]
if err == nil {
currentFile.LastContent = encoded
currentFile.RevisionUUID = output.RevisionUUID
current.Files[relative] = currentFile
current.LastError = ""
} else {
current.LastError = err.Error()
}
g.sessions[id] = current
}
g.mu.Unlock()
}
}
}
+76
View File
@@ -0,0 +1,76 @@
package main
import (
"crypto/rand"
"encoding/hex"
"flag"
"io"
"log"
"net"
"net/http"
"os"
"strings"
"time"
"pageswiki/internal/geckorelay"
)
func main() {
status := flag.Bool("status", false, "print bounded relay resource counters")
flag.Parse()
// File mount avoids placing the shared secret in image layers or process args.
key, err := os.ReadFile(os.Getenv("PAGESWIKI_GECKO_KEY_FILE"))
if err != nil {
log.Fatal("Gecko relay key file unavailable")
}
secret := strings.TrimSpace(string(key))
addr := os.Getenv("PAGESWIKI_GECKO_LISTEN")
if addr == "" {
addr = ":3012"
}
if *status {
host, port, err := net.SplitHostPort(addr)
if err != nil {
log.Fatal("invalid listen address")
}
if host == "" || host == "0.0.0.0" {
host = "127.0.0.1"
}
if host == "::" {
host = "::1"
}
nonce := make([]byte, 24)
if _, err = rand.Read(nonce); err != nil {
log.Fatal("random source unavailable")
}
req, _ := http.NewRequest("GET", "http://"+net.JoinHostPort(host, port)+"/status", nil)
req.Header.Set(geckorelay.GrantHeader, geckorelay.Sign(secret, geckorelay.Grant{Subject: "operator:status", Nonce: hex.EncodeToString(nonce), Expires: time.Now().Add(30 * time.Second).Unix()}))
client := http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{Proxy: nil}, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
resp, err := client.Do(req)
if err != nil {
log.Fatal("relay status unavailable")
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
log.Fatal("relay status denied")
}
if _, err = io.Copy(os.Stdout, io.LimitReader(resp.Body, 4096)); err != nil {
log.Fatal("relay status read failed")
}
return
}
assets := os.Getenv("PAGESWIKI_GECKO_ASSETS")
if assets == "" {
assets = "/gecko"
}
if err := geckorelay.VerifyAssets(assets); err != nil {
log.Fatal(err)
}
relay, err := geckorelay.New(secret, assets)
if err != nil {
log.Fatal(err)
}
s := &http.Server{Addr: addr, Handler: relay, ReadHeaderTimeout: 5 * time.Second, IdleTimeout: 30 * time.Second, MaxHeaderBytes: 16 << 10}
log.Printf("Gecko relay listening; authenticated TCP 80/443 only")
log.Fatal(s.ListenAndServe())
}
+127
View File
@@ -0,0 +1,127 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"os"
"time"
"pageswiki/internal/dr"
)
// All role changes are operator-only and offline. Never expose this as a
// domain-admin API: a data directory contains multiple campuses and domains.
func runDR(args []string, out io.Writer) error {
const usage = "pageswiki dr {init|status|snapshot|verify|restore|promote|resume-workers} [flags]\nUse COMMAND --help. Snapshot requires all writers stopped; promotion requires external fencing."
if len(args) == 0 || args[0] == "--help" {
_, err := fmt.Fprintln(out, usage)
return err
}
cmd := args[0]
switch cmd {
case "init", "status", "snapshot", "verify", "restore", "promote", "resume-workers":
default:
return errors.New(usage)
}
f := flag.NewFlagSet("dr "+cmd, flag.ContinueOnError)
f.SetOutput(out)
defaultDir := os.Getenv("PAGESWIKI_DATA_DIR")
if defaultDir == "" {
defaultDir = "./data"
}
dir := f.String("data-dir", defaultDir, "instance data directory")
var output, snapshot, hash, target, fenced, reference *string
var stopped, loss, reviewed *bool
switch cmd {
case "resume-workers":
reviewed = f.Bool("reviewed-restored-jobs", false, "attest restored queues and federation state have been reviewed for replay/duplicate effects")
case "snapshot":
output = f.String("output", "", "new snapshot directory outside data directory")
stopped = f.Bool("writers-stopped", false, "attest ALL servers, CLIs and external writers are stopped")
case "verify", "restore":
snapshot = f.String("snapshot", "", "snapshot directory")
hash = f.String("manifest-sha256", "", "manifest SHA-256 received over an authenticated channel")
if cmd == "restore" {
target = f.String("target", "", "new, nonexistent data directory (always standby)")
}
case "promote":
fenced = f.String("fenced-node", "", "exact former primary node UUID")
reference = f.String("fence-reference", "", "operator evidence/ticket for verified power/storage/network fencing")
loss = f.Bool("accept-data-loss", false, "accept loss of writes since the snapshot cutoff")
}
if err := f.Parse(args[1:]); err != nil {
if errors.Is(err, flag.ErrHelp) {
return nil
}
return err
}
if f.NArg() != 0 {
return errors.New("unexpected positional arguments")
}
encode := func(v any) error { e := json.NewEncoder(out); e.SetIndent("", " "); return e.Encode(v) }
ctx := context.Background()
switch cmd {
case "resume-workers":
c, err := dr.ResumeWorkers(*dir, *reviewed)
if err != nil {
return err
}
return encode(c)
case "init":
c, err := dr.Init(*dir)
if err != nil {
return err
}
return encode(c)
case "status":
c, err := dr.Load(*dir)
if err != nil {
return err
}
return encode(struct {
Node *dr.Config `json:"node"`
StandbyReadable bool `json:"standby_readable"`
}{c, c != nil && c.Readable(time.Now())})
case "snapshot":
if *output == "" || !*stopped {
return errors.New("snapshot requires --output and --writers-stopped; stop every writer first")
}
m, sum, err := dr.Snapshot(ctx, *dir, *output)
if err != nil {
return err
}
return encode(struct {
Manifest *dr.Manifest `json:"manifest"`
SHA256 string `json:"manifest_sha256"`
}{m, sum})
case "verify":
if *snapshot == "" {
return errors.New("--snapshot is required")
}
m, err := dr.Verify(ctx, *snapshot, *hash)
if err != nil {
return err
}
return encode(m)
case "restore":
if *snapshot == "" || *target == "" {
return errors.New("--snapshot and --target are required")
}
c, err := dr.Restore(ctx, *snapshot, *hash, *target)
if err != nil {
return err
}
return encode(c)
case "promote":
c, err := dr.Promote(*dir, *fenced, *reference, *loss)
if err != nil {
return err
}
return encode(c)
}
return errors.New("unknown DR command")
}
+21
View File
@@ -0,0 +1,21 @@
package main
import (
"bytes"
"strings"
"testing"
)
func TestDRCLIRequiresExplicitOfflineAttestation(t *testing.T) {
var out bytes.Buffer
err := runDR([]string{"snapshot", "--data-dir", t.TempDir(), "--output", "/not-used"}, &out)
if err == nil || !strings.Contains(err.Error(), "writers-stopped") {
t.Fatalf("unsafe invocation: %v", err)
}
if err = runDR([]string{"restore"}, &out); err == nil {
t.Fatal("missing restore target accepted")
}
if err = runDR([]string{"snapshot", "--help"}, &out); err != nil || !strings.Contains(out.String(), "writers-stopped") {
t.Fatal(err)
}
}
+254
View File
@@ -0,0 +1,254 @@
package main
import (
"bytes"
"crypto/ed25519"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"time"
"pageswiki/internal/store"
)
type externalCampusEnrollment struct {
FQDN string
Token string
HTTP bool
NoDefaultCampus bool
}
type campusEnrollmentSnapshot struct {
Campus store.Campus
Instance store.FederationInstance
Domains []store.Domain
Manifest store.SignedCampusManifest `json:"manifest"`
}
func parseExternalCampusEnrollment(args []string) (*externalCampusEnrollment, error) {
if len(args) == 0 {
return nil, nil
}
noDefaultCampus := false
filtered := make([]string, 0, len(args))
for _, arg := range args {
if arg == "--no-default-campus" {
if noDefaultCampus {
return nil, errors.New("--no-default-campus was specified twice")
}
noDefaultCampus = true
continue
}
filtered = append(filtered, arg)
}
if len(filtered) != 3 ||
(filtered[0] != "--add-external-campus" &&
filtered[0] != "--add-external-campus-http") {
return nil, errors.New(
"usage: pageswiki [--no-default-campus] " +
"[--add-external-campus FQDN TOKEN | " +
"--add-external-campus-http FQDN TOKEN]",
)
}
return &externalCampusEnrollment{
FQDN: strings.TrimSpace(filtered[1]),
Token: strings.TrimSpace(filtered[2]),
HTTP: filtered[0] == "--add-external-campus-http",
NoDefaultCampus: noDefaultCampus,
}, nil
}
func enrollExternalCampus(
dataDir string,
st *store.Store,
enrollment *externalCampusEnrollment,
) error {
if enrollment == nil {
return nil
}
instanceFQDN := strings.TrimSpace(os.Getenv("PAGESWIKI_INSTANCE_FQDN"))
if instanceFQDN == "" {
return errors.New(
"PAGESWIKI_INSTANCE_FQDN is required for external Campus enrollment",
)
}
snapshot, scheme, err := fetchCampusSnapshot(
dataDir,
enrollment,
instanceFQDN,
)
if err != nil {
return err
}
if err := applyEnrollmentSnapshot(st, snapshot, "", instanceFQDN, scheme); err != nil {
return err
}
if err := saveCampusEnrollment(
dataDir,
snapshot.Campus.UUID,
enrollment,
); err != nil {
return err
}
if enrollment.NoDefaultCampus {
_, err = st.RemoveUnusedBootstrapCampus()
}
return err
}
func applyEnrollmentSnapshot(st *store.Store, snapshot *campusEnrollmentSnapshot, expectedCampusUUID, instanceFQDN, scheme string) error {
if snapshot.Manifest.SchemaVersion != 0 {
return st.ApplySignedManagedCampusManifest(snapshot.Manifest, expectedCampusUUID, instanceFQDN, scheme)
}
// Transitional compatibility for pre-manifest controllers. New controllers
// always emit a signed manifest; this branch can be removed after upgrades.
return st.ApplyManagedCampusSnapshot(snapshot.Campus, snapshot.Domains, instanceFQDN, scheme)
}
func fetchCampusSnapshot(
dataDir string,
enrollment *externalCampusEnrollment,
instanceFQDN string,
) (*campusEnrollmentSnapshot, string, error) {
privateKey, fingerprint, err := ensureInstanceIdentity(dataDir)
if err != nil {
return nil, "", err
}
payload, err := json.Marshal(map[string]any{
"manifest_version": 2,
"capability_signature": base64.RawStdEncoding.EncodeToString(ed25519.Sign(privateKey, []byte(store.FederationCapabilityMessage(instanceFQDN, enrollment.Token)))),
"fqdn": instanceFQDN,
"fingerprint": fingerprint,
"public_key": base64.RawStdEncoding.EncodeToString(
privateKey.Public().(ed25519.PublicKey),
),
"signature": base64.RawStdEncoding.EncodeToString(
ed25519.Sign(
privateKey,
[]byte(instanceFQDN+"\n"+enrollment.Token),
),
),
})
if err != nil {
return nil, "", err
}
scheme := "https"
if enrollment.HTTP {
scheme = "http"
}
endpoint := fmt.Sprintf(
"%s://%s/_/api/v1/federation/enroll",
scheme,
enrollment.FQDN,
)
request, err := http.NewRequest(
http.MethodPost,
endpoint,
bytes.NewReader(payload),
)
if err != nil {
return nil, "", err
}
request.Header.Set("Content-Type", "application/json")
request.Header.Set("Authorization", "Bearer "+enrollment.Token)
client := &http.Client{Timeout: 20 * time.Second}
response, err := client.Do(request)
if err != nil {
return nil, "", err
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
message, _ := io.ReadAll(io.LimitReader(response.Body, 4<<10))
detail := strings.TrimSpace(string(message))
if detail != "" {
return nil, "", fmt.Errorf(
"Campus controller enrollment failed: %s: %s",
response.Status,
detail,
)
}
return nil, "", fmt.Errorf(
"Campus controller enrollment failed: %s",
response.Status,
)
}
var snapshot campusEnrollmentSnapshot
if err := json.NewDecoder(response.Body).Decode(&snapshot); err != nil {
return nil, "", err
}
return &snapshot, scheme, nil
}
func saveCampusEnrollment(
dataDir string,
campusUUID string,
enrollment *externalCampusEnrollment,
) error {
tokenDir := filepath.Join(dataDir, "federation")
if err := os.MkdirAll(tokenDir, 0700); err != nil {
return err
}
tokenPath := filepath.Join(tokenDir, campusUUID+".token")
if err := os.WriteFile(
tokenPath,
[]byte(enrollment.Token+"\n"),
0600,
); err != nil {
return err
}
if enrollment.NoDefaultCampus {
markerPath := filepath.Join(
tokenDir,
campusUUID+".no-default-campus",
)
if err := os.WriteFile(markerPath, nil, 0600); err != nil {
return err
}
}
scheme := "https"
if enrollment.HTTP {
scheme = "http"
}
endpointPath := filepath.Join(tokenDir, campusUUID+".endpoint")
return os.WriteFile(
endpointPath,
[]byte(scheme+"://"+enrollment.FQDN+"\n"),
0600,
)
}
func ensureInstanceIdentity(
dataDir string,
) (ed25519.PrivateKey, string, error) {
keyPath := filepath.Join(dataDir, "instance-ed25519.key")
privateKey, err := os.ReadFile(keyPath)
if errors.Is(err, os.ErrNotExist) {
_, generated, generateErr := ed25519.GenerateKey(rand.Reader)
if generateErr != nil {
return nil, "", generateErr
}
if writeErr := os.WriteFile(keyPath, generated, 0600); writeErr != nil {
return nil, "", writeErr
}
privateKey = generated
} else if err != nil {
return nil, "", err
}
if len(privateKey) != ed25519.PrivateKeySize {
return nil, "", errors.New("invalid instance Ed25519 private key")
}
publicKey := ed25519.PrivateKey(privateKey).Public().(ed25519.PublicKey)
sum := sha256.Sum256(publicKey)
return ed25519.PrivateKey(privateKey),
"sha256:" + hex.EncodeToString(sum[:]),
nil
}
+80
View File
@@ -0,0 +1,80 @@
package main
import "testing"
func TestExternalCampusEnrollmentArguments(t *testing.T) {
enrollment, err := parseExternalCampusEnrollment([]string{
"--add-external-campus",
"campus.example.test",
"secret-token",
})
if err != nil {
t.Fatal(err)
}
if enrollment.FQDN != "campus.example.test" ||
enrollment.Token != "secret-token" {
t.Fatalf("enrollment = %#v", enrollment)
}
}
func TestExternalCampusHTTPEnrollmentArguments(t *testing.T) {
enrollment, err := parseExternalCampusEnrollment([]string{
"--add-external-campus-http",
"campus.internal.test",
"secret-token",
})
if err != nil {
t.Fatal(err)
}
if !enrollment.HTTP {
t.Fatal("HTTP enrollment did not select HTTP transport")
}
}
func TestExternalCampusNoDefaultCampusArgument(t *testing.T) {
for _, args := range [][]string{
{
"--no-default-campus",
"--add-external-campus",
"campus.example.test",
"secret-token",
},
{
"--add-external-campus-http",
"campus.internal.test",
"secret-token",
"--no-default-campus",
},
} {
enrollment, err := parseExternalCampusEnrollment(args)
if err != nil {
t.Fatal(err)
}
if !enrollment.NoDefaultCampus {
t.Fatalf("NoDefaultCampus is false for %#v", args)
}
}
}
func TestNoDefaultCampusRequiresExternalCampus(t *testing.T) {
if _, err := parseExternalCampusEnrollment([]string{
"--no-default-campus",
}); err == nil {
t.Fatal("--no-default-campus without enrollment was accepted")
}
}
func TestInstanceFingerprintIsPersistent(t *testing.T) {
dataDir := t.TempDir()
_, first, err := ensureInstanceIdentity(dataDir)
if err != nil {
t.Fatal(err)
}
_, second, err := ensureInstanceIdentity(dataDir)
if err != nil {
t.Fatal(err)
}
if first != second {
t.Fatalf("fingerprints differ: %q, %q", first, second)
}
}
+119
View File
@@ -0,0 +1,119 @@
package main
import (
"errors"
"log"
"net/url"
"os"
"path/filepath"
"strings"
"time"
"pageswiki/internal/store"
)
const managedCampusReconcileInterval = 5 * time.Second
func startManagedCampusReconciler(dataDir string, st *store.Store) {
instanceFQDN := strings.TrimSpace(os.Getenv("PAGESWIKI_INSTANCE_FQDN"))
if instanceFQDN == "" {
return
}
go func() {
ticker := time.NewTicker(managedCampusReconcileInterval)
defer ticker.Stop()
for range ticker.C {
reconcileManagedCampuses(dataDir, st, instanceFQDN)
}
}()
}
func reconcileManagedCampuses(
dataDir string,
st *store.Store,
instanceFQDN string,
) {
campuses, err := st.ListCampuses()
if err != nil {
log.Printf("managed Campus reconciliation: %v", err)
return
}
for _, campus := range campuses {
if campus.AuthorityMode != "managed" || !campus.Enabled {
continue
}
if err := reconcileManagedCampus(
dataDir,
st,
campus,
instanceFQDN,
); err != nil {
_ = st.RecordCampusContact(campus.UUID, err.Error())
if campus.LastContactError != err.Error() {
log.Printf(
"managed Campus %s reconciliation failed: %v",
campus.FQDN,
err,
)
}
}
}
}
func reconcileManagedCampus(
dataDir string,
st *store.Store,
campus store.Campus,
instanceFQDN string,
) error {
tokenDir := filepath.Join(dataDir, "federation")
token, err := os.ReadFile(
filepath.Join(tokenDir, campus.UUID+".token"),
)
if err != nil {
return err
}
controllerAddress := campus.Transport + "://" + campus.FQDN
endpoint, err := os.ReadFile(
filepath.Join(tokenDir, campus.UUID+".endpoint"),
)
if err == nil {
controllerAddress = strings.TrimSpace(string(endpoint))
} else if !errors.Is(err, os.ErrNotExist) {
return err
}
parsed, err := url.Parse(controllerAddress)
if err != nil || parsed.Host == "" {
return errors.New("invalid saved Campus controller endpoint")
}
enrollment := &externalCampusEnrollment{
FQDN: parsed.Host,
Token: strings.TrimSpace(string(token)),
HTTP: parsed.Scheme == "http",
}
snapshot, scheme, err := fetchCampusSnapshot(
dataDir,
enrollment,
instanceFQDN,
)
if err != nil {
return err
}
if snapshot.Campus.UUID != campus.UUID && snapshot.Manifest.Campus.UUID != campus.UUID {
return errors.New("Campus controller identity changed")
}
if err := applyEnrollmentSnapshot(st, snapshot, campus.UUID, instanceFQDN, scheme); err != nil {
return err
}
markerPath := filepath.Join(
tokenDir,
campus.UUID+".no-default-campus",
)
if _, err := os.Stat(markerPath); err == nil {
_, err = st.RemoveUnusedBootstrapCampus()
return err
} else if !errors.Is(err, os.ErrNotExist) {
return err
}
return nil
}
+118
View File
@@ -0,0 +1,118 @@
package main
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"pageswiki/internal/db"
"pageswiki/internal/store"
)
func TestManagedCampusReconciliationDeploysAssignedDomain(t *testing.T) {
dataDir := t.TempDir()
database, err := db.Open(dataDir + "/pages.db")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = database.Close() })
if err := db.Migrate(database); err != nil {
t.Fatal(err)
}
st := store.New(database)
if err := st.EnsureConfigTable(); err != nil {
t.Fatal(err)
}
const (
campusUUID = "1140999c-b690-4c38-850d-8e04572e489e"
domainUUID = "2251000d-c7a1-4d49-961e-9f15683f590f"
instanceFQDN = "node.example.test"
)
campus := store.Campus{
UUID: campusUUID,
Name: "external",
Title: "External Campus",
DefaultLang: "en",
AuthorityMode: "controller",
FQDN: "campus.example.test",
Enabled: true,
}
domain := store.Domain{
UUID: domainUUID,
Name: "deployed",
Title: "Deployed Domain",
Domain: "deployed.example.test",
Enabled: true,
CampusUUID: campusUUID,
Placement: "remote",
InstanceFQDN: instanceFQDN,
InstanceFingerprint: "sha256:test",
}
controller := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/_/api/v1/federation/enroll" {
http.NotFound(w, r)
return
}
_ = json.NewEncoder(w).Encode(map[string]any{
"campus": campus,
"domains": []store.Domain{domain},
})
},
))
t.Cleanup(controller.Close)
if err := st.ApplyManagedCampusSnapshot(
campus,
nil,
instanceFQDN,
"http",
); err != nil {
t.Fatal(err)
}
enrollment := &externalCampusEnrollment{
FQDN: strings.TrimPrefix(controller.URL, "http://"),
Token: "test-token",
HTTP: true,
NoDefaultCampus: true,
}
if err := saveCampusEnrollment(
dataDir,
campusUUID,
enrollment,
); err != nil {
t.Fatal(err)
}
managed, err := st.GetCampusByUUID(campusUUID)
if err != nil {
t.Fatal(err)
}
if err := reconcileManagedCampus(
dataDir,
st,
*managed,
instanceFQDN,
); err != nil {
t.Fatal(err)
}
deployed, err := st.GetDomainByUUID(domainUUID)
if err != nil {
t.Fatal(err)
}
if deployed.Placement != "local" || !deployed.Enabled {
t.Fatalf("deployed Domain = %#v", deployed)
}
campuses, err := st.ListCampuses()
if err != nil {
t.Fatal(err)
}
if len(campuses) != 1 ||
campuses[0].UUID != campusUUID ||
campuses[0].AuthorityMode != "managed" {
t.Fatalf("Campuses after reconciliation = %#v", campuses)
}
}
+79 -3
View File
@@ -1,15 +1,38 @@
package main
import (
"errors"
"log"
"os"
"path/filepath"
"pageswiki/internal/db"
"pageswiki/internal/dr"
"pageswiki/internal/server"
"pageswiki/internal/store"
"path/filepath"
)
func main() {
if len(os.Args) > 1 && os.Args[1] == "schema" {
if err := runSchema(os.Args[2:], os.Stdout); err != nil {
log.Print(err)
if errors.Is(err, errSchemaPending) {
os.Exit(2)
}
os.Exit(1)
}
return
}
if len(os.Args) > 1 && os.Args[1] == "dr" {
if err := runDR(os.Args[2:], os.Stdout); err != nil {
log.Fatal(err)
}
return
}
enrollment, err := parseExternalCampusEnrollment(os.Args[1:])
if err != nil {
log.Fatal(err)
}
dataDir :=
os.Getenv(
@@ -20,7 +43,7 @@ func main() {
dataDir = "./data"
}
err := os.MkdirAll(
err = os.MkdirAll(
dataDir,
0755,
)
@@ -29,6 +52,39 @@ func main() {
log.Fatal(err)
}
instanceLock, err := dr.Lock(dataDir, false)
if err != nil {
log.Fatal(err)
}
defer instanceLock.Close()
node, err := dr.Load(dataDir)
if err != nil {
log.Fatalf("invalid DR configuration (startup refused): %v", err)
}
if err = dr.RequireRole(node, os.Getenv("PAGESWIKI_DR_EXPECT_ROLE")); err != nil {
log.Fatal(err)
}
if node != nil && node.Role == "standby" {
if len(os.Args) > 1 {
log.Fatal("standby does not accept enrollment/startup mutation flags")
}
database, err := db.OpenReadOnly(filepath.Join(dataDir, "pages.db"))
if err != nil {
log.Fatal(err)
}
defer database.Close()
st, err := store.NewReadOnlyWithDocumentRoot(database, filepath.Join(dataDir, "documents"))
if err != nil {
log.Fatal(err)
}
st.StandbyReadUntil = node.ReadUntil()
log.Printf("DR standby: migrations, federation reconciliation and background workers disabled; snapshot=%s", node.SnapshotID)
if err := server.New(st).Run(":3000"); err != nil {
log.Fatal(err)
}
return
}
dbPath :=
filepath.Join(
dataDir,
@@ -53,7 +109,20 @@ func main() {
log.Fatal(err)
}
st := store.New(database)
documentRoot, err :=
store.MigrateLegacyDocumentRoot(
dataDir,
)
if err != nil {
log.Fatal(err)
}
st :=
store.NewWithDocumentRoot(
database,
documentRoot,
)
err = st.EnsureConfigTable()
@@ -66,6 +135,13 @@ func main() {
if err != nil {
log.Fatal(err)
}
if err := enrollExternalCampus(dataDir, st, enrollment); err != nil {
log.Fatal(err)
}
st.BackgroundJobsDisabled = node != nil && !node.WorkersEnabled
if !st.BackgroundJobsDisabled {
startManagedCampusReconciler(dataDir, st)
}
srv := server.New(st)
+114
View File
@@ -0,0 +1,114 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"os"
"path/filepath"
"time"
"pageswiki/internal/db"
"pageswiki/internal/store"
)
var errSchemaPending = errors.New("schema migrations are pending; no changes were applied")
var errSchemaBlocked = errors.New("schema history or contract is incompatible; no changes were applied")
func runSchema(args []string, out io.Writer) error {
const usage = "pageswiki schema {status|check} [--db FILE | --data-dir DIR] [--json] [--timeout 30s]\nRead-only inspection of managed components; legacy migrations and pending DDL are not rehearsed.\ncheck exits 2 for pending migrations, 1 for incompatible state, 0 when managed components are current."
if len(args) == 0 || args[0] == "--help" {
_, err := fmt.Fprintln(out, usage)
return err
}
if args[0] != "status" && args[0] != "check" {
return errors.New(usage)
}
f := flag.NewFlagSet("schema "+args[0], flag.ContinueOnError)
f.SetOutput(out)
defaultDir := os.Getenv("PAGESWIKI_DATA_DIR")
if defaultDir == "" {
defaultDir = "./data"
}
dir := f.String("data-dir", defaultDir, "instance data directory (not created)")
path := f.String("db", "", "explicit existing SQLite database path")
jsonOutput := f.Bool("json", false, "JSON report")
timeout := f.Duration("timeout", 30*time.Second, "inspection timeout")
if err := f.Parse(args[1:]); err != nil {
if errors.Is(err, flag.ErrHelp) {
return nil
}
return err
}
if f.NArg() != 0 || *timeout <= 0 {
return errors.New("unexpected arguments or invalid timeout")
}
explicitDir := false
f.Visit(func(flag *flag.Flag) {
if flag.Name == "data-dir" {
explicitDir = true
}
})
if *path != "" && explicitDir {
return errors.New("choose --db or --data-dir, not both")
}
if *path == "" {
*path = filepath.Join(*dir, "pages.db")
}
database, err := db.OpenReadOnly(*path)
if err != nil {
return err
}
defer database.Close()
ctx, cancel := context.WithTimeout(context.Background(), *timeout)
defer cancel()
report, err := store.InspectSchema(ctx, database)
if err != nil {
return err
}
if *jsonOutput {
encoder := json.NewEncoder(out)
encoder.SetIndent("", " ")
err = encoder.Encode(report)
} else {
_, err = fmt.Fprintf(out, "Coverage: %s (not a full startup or data migration rehearsal)\n", report.Coverage)
for _, component := range report.Components {
if err != nil {
break
}
_, err = fmt.Fprintf(out, "%-24s %-8s %d -> %d\n", component.Component, component.State, component.AppliedVersion, component.LatestVersion)
for _, step := range component.Pending {
if err != nil {
break
}
_, err = fmt.Fprintf(out, " pending v%d %s (%d SQL statements, %d conditional columns)\n", step.Version, step.Name, step.Statements, len(step.AddColumns))
}
for _, issue := range component.Issues {
if err != nil {
break
}
_, err = fmt.Fprintf(out, " %s\n", issue)
}
}
if err == nil {
_, err = fmt.Fprintf(out, "Legacy areas not covered: %v\n", report.LegacyAreas)
}
}
if err != nil || args[0] == "status" {
return err
}
pending := false
for _, component := range report.Components {
if component.State == "blocked" || component.State == "unknown" {
return errSchemaBlocked
}
pending = pending || component.State == "pending"
}
if pending {
return errSchemaPending
}
return nil
}
+106
View File
@@ -0,0 +1,106 @@
package main
import (
"bytes"
"encoding/json"
"errors"
"os"
"path/filepath"
"testing"
"pageswiki/internal/db"
"pageswiki/internal/store"
)
func TestSchemaCommandMissingDatabaseDoesNotInitialize(t *testing.T) {
dir := filepath.Join(t.TempDir(), "missing")
var out bytes.Buffer
if err := runSchema([]string{"status", "--data-dir", dir}, &out); err == nil {
t.Fatal("missing DB accepted")
}
if _, err := os.Stat(dir); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("directory created: %v", err)
}
}
func TestSchemaCommandPendingAndBlocked(t *testing.T) {
dir := t.TempDir()
database, err := db.Open(filepath.Join(dir, "pages.db"))
if err != nil {
t.Fatal(err)
}
defer database.Close()
if err := database.Ping(); err != nil {
t.Fatal(err)
}
var out bytes.Buffer
if err := runSchema([]string{"status", "--data-dir", dir, "--json"}, &out); err != nil {
t.Fatal(err)
}
var report store.SchemaInspection
if err := json.Unmarshal(out.Bytes(), &report); err != nil {
t.Fatal(err)
}
if report.Coverage != "managed-components-only" || len(report.Components) == 0 || len(report.LegacyAreas) == 0 {
t.Fatalf("report=%+v", report)
}
out.Reset()
if err := runSchema([]string{"check", "--db", filepath.Join(dir, "pages.db")}, &out); !errors.Is(err, errSchemaPending) {
t.Fatalf("pending: %v", err)
}
if _, err := database.Exec(`CREATE TABLE schema_migration_steps(component TEXT,version INTEGER,name TEXT,checksum TEXT,applied_at_ms INTEGER); INSERT INTO schema_migration_steps VALUES('future-component',1,'future','unknown',0)`); err != nil {
t.Fatal(err)
}
out.Reset()
if err := runSchema([]string{"check", "--data-dir", dir}, &out); !errors.Is(err, errSchemaBlocked) {
t.Fatalf("unknown component: %v", err)
}
var count int
if err := database.QueryRow(`SELECT count(*) FROM sqlite_master WHERE type='table'`).Scan(&count); err != nil || count != 1 {
t.Fatalf("command initialized DB: %d %v", count, err)
}
}
func TestSchemaCommandArguments(t *testing.T) {
for _, args := range [][]string{{"apply"}, {"check", "--timeout", "0s"}, {"check", "extra"}, {"check", "--db", "a", "--data-dir", "b"}} {
if err := runSchema(args, &bytes.Buffer{}); err == nil {
t.Fatalf("accepted %v", args)
}
}
for _, args := range [][]string{nil, {"--help"}, {"check", "--help"}} {
if err := runSchema(args, &bytes.Buffer{}); err != nil {
t.Fatalf("help %v: %v", args, err)
}
}
}
func TestSchemaCommandCurrentApplicationDatabase(t *testing.T) {
dir := t.TempDir()
database, err := db.Open(filepath.Join(dir, "pages.db"))
if err != nil {
t.Fatal(err)
}
defer database.Close()
if err := db.Init(database); err != nil {
t.Fatal(err)
}
if err := db.Migrate(database); err != nil {
t.Fatal(err)
}
// Initialize only this synthetic fixture. runSchema must not run the
// constructor itself, even for an already initialized application DB.
store.NewWithDocumentRoot(database, filepath.Join(dir, "documents"))
var out bytes.Buffer
if err := runSchema([]string{"check", "--data-dir", dir, "--json"}, &out); err != nil {
t.Fatalf("current database: %v\n%s", err, out.String())
}
var report store.SchemaInspection
if err := json.Unmarshal(out.Bytes(), &report); err != nil {
t.Fatal(err)
}
for _, component := range report.Components {
if component.State != "current" {
t.Fatalf("component=%+v", component)
}
}
}
+1659
View File
File diff suppressed because it is too large. Load diff
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env python3
"""Compatibility launcher for the unreleased prototype. Prefer pages-cli/pages."""
from pathlib import Path
import runpy
import sys
print('Renamed to pages (pages-cli). Use pages configure, then pages login. See pages-cli/README.md.', file=sys.stderr)
runpy.run_path(str(Path(__file__).resolve().parent.parent / 'pages-cli' / 'pages'), run_name='__main__')
+69
View File
@@ -0,0 +1,69 @@
import importlib.util
from pathlib import Path
import unittest
import sqlite3
import io
from contextlib import redirect_stdout
from types import SimpleNamespace
SCRIPT = Path(__file__).with_name("pageswiki-cli")
SPEC = importlib.util.spec_from_file_location("pageswiki_cli", SCRIPT)
if SPEC is None or SPEC.loader is None:
from importlib.machinery import SourceFileLoader
SPEC = importlib.util.spec_from_loader(
"pageswiki_cli",
SourceFileLoader("pageswiki_cli", str(SCRIPT)),
)
assert SPEC is not None and SPEC.loader is not None
pageswiki_cli = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(pageswiki_cli)
class ContentTypeDetectionTest(unittest.TestCase):
def test_extensionless_shell(self) -> None:
content_type = pageswiki_cli.detect_content_type(
b"#!/usr/bin/env bash\necho hello\n",
)
self.assertEqual(content_type, "application/x-sh; charset=utf-8")
def test_extensionless_utf8_text(self) -> None:
content_type = pageswiki_cli.detect_content_type(
"拡張子なしの文書\n".encode(),
)
self.assertEqual(content_type, "text/plain; charset=utf-8")
def test_magic_number_wins_over_text_analysis(self) -> None:
content_type = pageswiki_cli.detect_content_type(
b"%PDF-1.7\nsample",
)
self.assertEqual(content_type, "application/pdf")
def test_shift_jis_text(self) -> None:
content_type = pageswiki_cli.detect_content_type(
"日本語の文書\n".encode("shift_jis"),
)
self.assertEqual(content_type, "text/plain; charset=shift_jis")
class ExtensionDefaultTest(unittest.TestCase):
def test_gecko_is_opt_in(self) -> None:
db = sqlite3.connect(":memory:")
self.addCleanup(db.close)
db.execute("CREATE TABLE config(domain_uuid TEXT,key TEXT,value TEXT,updated_at INTEGER,UNIQUE(domain_uuid,key))")
# Exercise the CLI command without reading any deployment database.
owner = next(value for value in vars(pageswiki_cli).values() if isinstance(value, type) and hasattr(value, "command_extension"))
fixture = SimpleNamespace(db=db, domain_uuid="synthetic")
output = io.StringIO()
with redirect_stdout(output):
owner.command_extension(fixture, ["viewer.gecko"])
owner.command_extension(fixture, ["viewer.gecko", "on"])
owner.command_extension(fixture, ["viewer.gecko"])
owner.command_extension(fixture, ["viewer.gecko", "off"])
owner.command_extension(fixture, ["viewer.gecko"])
self.assertEqual(output.getvalue().splitlines(), ["viewer.gecko\tdisabled", "viewer.gecko\ton", "viewer.gecko\tenabled", "viewer.gecko\toff", "viewer.gecko\tdisabled"])
if __name__ == "__main__":
unittest.main()
+43
View File
@@ -0,0 +1,43 @@
import runpy
import sqlite3
import tempfile
import unittest
from pathlib import Path
CLI = runpy.run_path(str(Path(__file__).with_name("pageswiki-cli")))["PagesWikiCLI"]
class AssetMetadataTest(unittest.TestCase):
def test_head_metadata_and_history_follow_cli_uploads(self):
with tempfile.TemporaryDirectory() as directory:
cli = CLI.__new__(CLI)
cli.documents = Path(directory) / "documents"
cli.domain_uuid = "test-domain"
cli.db = sqlite3.connect(":memory:")
self.addCleanup(cli.db.close)
cli.db.row_factory = sqlite3.Row
cli.db.executescript("""
CREATE TABLE objects(uuid TEXT PRIMARY KEY,kind TEXT,created_at INTEGER);
CREATE TABLE assets(domain_uuid TEXT,address TEXT,object_uuid TEXT,
head_document_hash TEXT,content_type TEXT,filename TEXT,size_bytes INTEGER,
created_at_ms INTEGER,updated_at_ms INTEGER,PRIMARY KEY(domain_uuid,address));
""")
address = "asset:image:logo"
payload = b"\x89PNG\r\n\x1a\nfixture"
self.assertEqual(cli.put_asset(address, payload, "logo.png"), "updated")
self.assertEqual(cli.put_asset(address, payload, "logo.png"), "skipped")
self.assertEqual(cli.put_asset(address, payload, "renamed.png"), "updated")
row = cli.db.execute("SELECT * FROM assets").fetchone()
self.assertEqual(row["content_type"], "image/png")
self.assertEqual(row["filename"], "renamed.png")
self.assertEqual(row["size_bytes"], len(payload))
self.assertEqual(cli.document_path(row["head_document_hash"]).read_bytes(), payload)
history = cli.db.execute("SELECT * FROM asset_head_history ORDER BY id").fetchall()
self.assertEqual(len(history), 2)
self.assertEqual(history[-1]["document_hash"], row["head_document_hash"])
self.assertEqual(history[-1]["filename"], "renamed.png")
if __name__ == "__main__":
unittest.main()
+34
View File
@@ -0,0 +1,34 @@
import fcntl
import json
import os
from pathlib import Path
import tempfile
import unittest
from unittest.mock import patch
from pageswiki_cli_test import pageswiki_cli
class DRGuardTest(unittest.TestCase):
def test_standby_rejected_before_database_open(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "pages.db").write_bytes(b"not a database; must not be opened")
(root / "dr-node.json").write_text(json.dumps({"version": 1, "role": "standby"}))
with patch.dict(os.environ, {"PAGESWIKI_DATA_DIR": directory, "PAGESWIKI_DB": str(root / "pages.db"), "PAGESWIKI_DOCUMENTS_DIR": str(root / "documents")}):
with self.assertRaisesRegex(pageswiki_cli.CLIError, "standby"):
pageswiki_cli.PagesWikiCLI()
def test_offline_maintenance_lock_blocks_cli(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "pages.db").write_bytes(b"not a database")
with (root / ".pageswiki-instance.lock").open("wb") as lock:
fcntl.flock(lock.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
with patch.dict(os.environ, {"PAGESWIKI_DATA_DIR": directory, "PAGESWIKI_DB": str(root / "pages.db")}):
with self.assertRaisesRegex(pageswiki_cli.CLIError, "maintenance guard"):
pageswiki_cli.PagesWikiCLI()
if __name__ == "__main__":
unittest.main()
+47
View File
@@ -0,0 +1,47 @@
# Optional development/pilot overlay, never selected by the production recipe.
# Requires host egress rules; an isolated bridge alone is not an egress firewall.
services:
app:
environment:
PAGESWIKI_GECKO_RELAY: http://gecko-relay:3012
PAGESWIKI_GECKO_KEY_FILE: /run/secrets/gecko-key
volumes:
- ${PAGESWIKI_GECKO_KEY_PATH:?Set absolute relay key path}:/run/secrets/gecko-key:ro
networks:
default: {}
gecko-front: {}
gecko-relay:
build:
context: .
dockerfile: Dockerfile.gecko-relay
image: pageswiki-gecko-relay:dev
environment:
PAGESWIKI_GECKO_KEY_FILE: /run/secrets/gecko-key
volumes:
- ${PAGESWIKI_GECKO_KEY_PATH:?Set absolute relay key path}:/run/secrets/gecko-key:ro
- ${PAGESWIKI_GECKO_ASSETS_DIR:?Set verified Gecko dist directory}:/gecko:ro
networks:
gecko-front: {}
gecko-egress: {}
read_only: true
cap_drop: [ALL]
security_opt: [no-new-privileges:true]
mem_limit: 256m
cpus: 0.5
pids_limit: 64
restart: unless-stopped
healthcheck:
test: [CMD, /usr/local/bin/gecko-relay, --status]
interval: 30s
timeout: 3s
retries: 3
networks:
gecko-front:
internal: true
ipam:
config:
- subnet: ${PAGESWIKI_GECKO_FRONT_SUBNET:-172.24.0.0/27}
gecko-egress:
ipam:
config:
- subnet: ${PAGESWIKI_GECKO_EGRESS_SUBNET:-172.24.0.32/27}
+11
View File
@@ -0,0 +1,11 @@
# Optional overlay. Never builds locally when used with `up --no-build`.
# Use all three images from the same successful workflow, identified by SHA.
services:
app:
image: ${PAGESWIKI_IMAGE_PREFIX:-git.anya.jp/anya}/pageswiki-app:${PAGESWIKI_IMAGE_TAG:?Set the image tag from a successful Gitea workflow}
app2:
image: ${PAGESWIKI_IMAGE_PREFIX:-git.anya.jp/anya}/pageswiki-app:${PAGESWIKI_IMAGE_TAG:?Set the image tag from a successful Gitea workflow}
adaptor-gateway:
image: ${PAGESWIKI_IMAGE_PREFIX:-git.anya.jp/anya}/pageswiki-adaptor-gateway:${PAGESWIKI_IMAGE_TAG:?Set the image tag from a successful Gitea workflow}
excalidraw-viewer:
image: ${PAGESWIKI_IMAGE_PREFIX:-git.anya.jp/anya}/pageswiki-excalidraw-viewer:${PAGESWIKI_IMAGE_TAG:?Set the image tag from a successful Gitea workflow}
+110 -11
View File
@@ -1,20 +1,119 @@
services:
nginx:
container_name: pageswiki-proxy
image: nginx:1.27-alpine
restart: unless-stopped
ports:
- "8080:80"
volumes:
- ./deploy/nginx.conf:/etc/nginx/conf.d/default.conf:ro
depends_on:
- app
app:
container_name: pageswiki
build: .
image: pageswiki-app
networks:
default:
aliases:
- extpages.anya.jp
volumes:
- ./data:/data
ports:
- "3000:3000"
environment:
PAGESWIKI_DATA_DIR: /data
KROKI_URL: http://kroki:8000
PAGESWIKI_ADAPTER_GATEWAY_ORIGIN: http://adaptor-gateway:3000
extra_hosts:
- "id.anya.jp:172.18.0.10"
healthcheck:
test: ["CMD", "wget", "-q", "-O", "-", "http://localhost:3000/health"]
interval: 2s
timeout: 1s
retries: 15
restart: unless-stopped
app2:
container_name: pageswiki-ext
image: pageswiki-app
command: >
/pageswiki --no-default-campus --add-external-campus-http extpages.anya.jp:3000 ${PAGESWIKI_EXTERNAL_CAMPUS_TOKEN:?Set PAGESWIKI_EXTERNAL_CAMPUS_TOKEN in .env}
volumes:
- ./data2:/data
ports:
- "3001:3000"
environment:
PAGESWIKI_INSTANCE_FQDN: ext-node-pages.anya.jp
PAGESWIKI_DATA_DIR: /data
KROKI_URL: http://kroki:8000
extra_hosts:
- "id.anya.jp:172.18.0.10"
depends_on:
app:
condition: service_healthy
restart: unless-stopped
adaptor-gateway:
container_name: pageswiki-adaptor-gateway
build:
context: .
dockerfile: Dockerfile.adapter-gateway
environment:
PAGESWIKI_ORIGIN: http://app:3000
CODE_SERVER_ORIGIN: http://code-server:8080
EXCALIDRAW_VIEWER_ORIGIN: http://excalidraw-viewer:8080
volumes:
- adapter-workspaces:/workspaces
expose:
- "3000"
depends_on:
app:
condition: service_healthy
code-server:
condition: service_started
excalidraw-viewer:
condition: service_healthy
restart: unless-stopped
code-server:
container_name: pageswiki-code-server
image: codercom/code-server:latest
command:
- --bind-addr
- 0.0.0.0:8080
- --auth
- none
- --disable-telemetry
- --disable-proxy
- --disable-getting-started-override
- /workspaces
expose:
- "8080"
volumes:
- adapter-workspaces:/workspaces
restart: unless-stopped
excalidraw-viewer:
container_name: pageswiki-excalidraw-viewer
build:
context: .
dockerfile: Dockerfile.excalidraw-viewer
expose:
- "8080"
healthcheck:
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8080/health"]
interval: 5s
timeout: 2s
retries: 12
restart: unless-stopped
kroki:
image: yuzutech/kroki
environment:
KROKI_MERMAID_HOST: kroki-mermaid
KROKI_MERMAID_PORT: 8002
restart: unless-stopped
depends_on:
- kroki-mermaid
kroki-mermaid:
image: yuzutech/kroki-mermaid
restart: unless-stopped
expose:
- "8002"
volumes:
adapter-workspaces:
networks:
default:
ipam:
config:
- subnet: 172.23.0.0/27
+92
View File
@@ -0,0 +1,92 @@
# PagesWiki design documentation
## Current reading order
The reorganized Japanese handbook is [ja/index.md](ja/index.md), published under
`https://pages.anya.jp/lab/pageswiki-dev/`. It separates implemented slices,
proposals, unresolved decisions and delivery gates. Its chapter identifiers are
language-neutral; existing English source documents remain here for traceability
and future aligned translations, not as automatically current English editions.
- Concepts and boundaries: chapters 1–6.
- [Organizer navigation and personal dashboard](ja/organizer-dashboard.md): shared navigation,
personal settings, resizable widgets, owner-scoped layout storage and extension boundaries.
- [Task capture, work reporting and private address book](ja/task-productivity-contacts.md):
shared Web/CLI API, neomutt, defaults, daily digest and home search commands.
- Existing product and runtime: chapters 7–11.
- Organizer / task / calendar / RSS / mail / HTML editing proposals: chapters 12–17.
- Operations, i18n, ordered work, decisions and sources: chapters 18–22.
- [Source-to-chapter inventory](ja/sources.md).
- [Decision and inconsistency register](ja/decisions.md).
- [Component names and responsibilities](ja/component-names.md): adopted subsystem names,
all eight initial extractions, compatibility boundaries and the remaining component work.
- [Dregarnuhr scheduler contract](ja/dregarnuhr.md): application registration, queue persistence,
concurrency/retry tests and the remaining lease/HA/shutdown work.
- [Geduldh document storage contract](ja/geduldh.md): immutable local payloads, bounded scratch copies,
compatibility facades, Range/backfill/DR verification and remaining backend work.
- [Mestionora search and index contract](ja/mestionora.md): SQLite/City lookup, explicit read policies,
bounded enrichment, query-plan compatibility and remaining indexing/policy work.
- [Ordoschnellic notification delivery contract](ja/ordoschnellic.md): bounded Web Push dispatch,
transactional Inbox enqueue, privacy, retry/lease compatibility and remaining fencing/observability work.
- [Schutzaria authorization and publication contract](ja/schutzaria.md): pure policy evaluation,
immutable request snapshots, retained application checks and remaining consistency/reviewer work.
- [Kunstzeal projection and renderer contract](ja/kunstzeal.md): stable adapter selection, bounded
Kroki execution, source authorization, atomic derived cache and remaining provider/runtime work.
- [Seheweit observation contract](ja/seheweit.md): bounded local collection, compatible snapshots,
history windows/retention, cancellation and remaining availability/rollup/provider metrics work.
- [Editor resilience subproject](ja/editor-resilience.md): drafts, conditional saves,
conflict resolution, mobile authoring, language/history, round-trip preservation and collaboration gates.
- [Storage portability and operations](storage-portability-and-operations.md): backend contracts,
migration/backfill, backup/export/import and setup phases;
[read-only schema preflight and rollout](../operations/storage-schema-rollout.md).
## Document roles
DRの詳細設計・初版CLI・復旧手順は [単一writerのDR](ja/disaster-recovery.md) を参照。
インスタンス単位の停止snapshotと手動昇格を先行し、自動切替・オンライン同期は未実装。
次段階のDR-D1は**詳細設計完了・実装未着手**:
[役割分離と再参加](ja/dr-authority-and-rejoin.md)、
[protocol・同期・切替](ja/dr-protocol-and-sync.md)、
[実施計画と受入試験](ja/dr-delivery-plan.md)。DB別件と独立して文書化し、
保存側の能力契約が未提供の場合は自動切替を有効化しない。
`arch.md`, `address-model.md`, `access-model.md`, `auth.md` and `i18n-delivery.md`
include original conceptual drafts. Detailed topic documents describe contracts
and implementation slices. `implementation-status.md` contains dated progress;
older paragraphs are historical and may be superseded. The roadmap is a task
inventory; the execution ledger is verification/deployment history, not a blanket
feature-completeness claim.
For new work, update the relevant chapter and detailed contract together. Keep
stable identity, authorization and bounded processing constraints. Do not convert
design proposals into active features or public grants merely by documenting them.
## Publication
Japanese Markdown under `ja/` is the editing source. The documentation export
tool renders canonical HTML with the application's Markdown renderer, validates
chapter links, and emits a manifest. Upload through the trusted data CLI with an
explicit Domain and `PAGESWIKI_LANG=ja`; no authentication/publication policy is
changed. Record resulting Page/Revision identities separately from source text.
English publication is deferred until an aligned translation and language UI
are ready. Never overwrite Japanese with English at language `und`.
Generate a fresh, empty staging directory, then run:
```sh
go run ./tools/design-export -out /tmp/pageswiki-design-staging
go test ./tools/design-export
```
The tool does not access the database or network. Review `manifest.json`, check
destination collisions, and select the correct database/Domain explicitly before
using `data/pageswiki-cli put` for each rendered HTML file and manifest Path.
The index maps to `/lab/pageswiki-dev`, not `/lab/pageswiki-dev/index`.
Do not blindly replace existing Wiki edits on a later publication: compare the
recorded Revision UUID/hash with the current head and resolve divergence first.
The initial upload receipt is [publication-ja-2026-09-19.json](publication-ja-2026-09-19.json).
All 23 canonical payloads were read back and compared. Source hashes and stored
Page/Revision identities support later language alignment. Anonymous HTTPS access
still redirects to login; this upload did not change public access settings.
+665
View File
@@ -0,0 +1,665 @@
# Access Model draft
## Goal
Separate:
```text
knowledge navigation
projection rendering
editing surfaces
object retrieval
native execution
```
into independent runtime layers.
The system treats:
```
content != rendering != editor != storage
```
as a core architectural principle.
# Core Principle
Pages Wiki does NOT treat editors as canonical reality.
Editors are:
```text
authoring projections
```
onto canonical semantic object structures.
Canonical storage is:
```text
DOM/object graph centered
```
depending on object class.
# High-Level Access Architecture
```mermaid
graph TD
KA[Knowledge Access]
PA[Projection Access]
EA[Editing Access]
OA[Object Access]
NA[Native Access]
KA --> PA
PA --> EA
PA --> OA
EA --> OA
PA --> NA
EA --> NA
```
* * *
# 1. Knowledge Access
## Purpose
Semantic navigation for humans.
# Example
```text
/research/network/bgp
/dev/api/auth
/hr/onboarding
```
# Responsibilities
- semantic routing
- hierarchy traversal
- language negotiation
- capability evaluation
- object resolution
# Output
Usually:
```text
rendered semantic projection
```
# Resolution Pipeline
```text
path
↓
uuid
↓
revision
↓
projection
```
# Important Principle
Knowledge access is:
```text
human semantic navigation
```
NOT raw object retrieval.
# 2. Projection Access
## Purpose
Interpret semantic objects into runtime surfaces.
# Example
```text
/_/view/flow/uuid:abcd
/_/view/slides/uuid:abcd
/_/view/sheet/uuid:abcd
/_/view/print/uuid:abcd
/_/view/canvas/uuid:abcd
```
# Projection Examples
| Projection | Meaning |
| --- | --- |
| `flow` | Flowing article |
| `slides` | Staged presentation |
| `print` | Printable layout |
| `sheet` | Spreadsheet runtime |
| `canvas` | Whiteboard/canvas |
| `topology` | Graph topology |
| `preview` | Lightweight rendering |
# Important Principle
Projection is:
```text
runtime interpretation
```
NOT canonical storage.
# Same Object, Multiple Projections
```mermaid
graph TD
OBJ[Semantic Object]
FLOW[Flow Projection]
SLIDE[Slides Projection]
PRINT[Print Projection]
SHEET[Sheet Projection]
OBJ --> FLOW
OBJ --> SLIDE
OBJ --> PRINT
OBJ --> SHEET
```
* * *
# 3. Editing Access
## Purpose
Provide authoring surfaces.
* * *
# Important Principle
Editors are:
```text
projection-specific authoring surfaces
```
NOT canonical storage owners.
* * *
# Editor Pipeline
```mermaid
graph TD
EDITOR[Editor]
AST[Semantic Object Graph]
REV[Revision Snapshot]
EDITOR --> AST
AST --> REV
```
* * *
# Supported Editors
| Editor | Purpose |
| --- | --- |
| Markdown | Lightweight flow editing |
| WYSIWYG | Visual article editing |
| Raw HTML | Direct DOM editing |
| Slide Editor | Presentation authoring |
| Sheet Editor | Spreadsheet runtime editing |
| Whiteboard Editor | Canvas editing |
| DrawIO-like Editor | Topology editing |
* * *
# Important Principle
All editors ultimately produce:
```text
semantic object graph
```
or:
```text
canonical DOM representation
```
depending on object class.
* * *
# Editor Negotiation
Runtime selects editor based on:
- object class
- supported projection types
- runtime capabilities
- user preferences
- available native integrations
* * *
# Example
```text
HTML article
```
Available editors:
- Markdown
- WYSIWYG
- Raw HTML
* * *
# Example
```text
Presentation object
```
Available editors:
- Slide editor
- Raw DOM editor
* * *
# Example
```text
Spreadsheet object
```
Available editors:
- Sheet runtime editor
- Native Excel integration
* * *
# 4. Object Access
## Purpose
Raw immutable object retrieval.
* * *
# Example
```text
/_/obj/sha256:abcd
```
* * *
# Typical Objects
- PDF
- PPTX
- XLSX
- PNG
- SVG
- JS
- CSS
- DrawIO payload
- Excalidraw payload
* * *
# Important Principle
Object access is:
```text
transport/storage retrieval
```
NOT semantic navigation.
* * *
# Object Access Pipeline
```mermaid
graph TD
OBJREF[Object Reference]
FETCH[Object Fetch]
CACHE[Cache/CDN]
RETURN[Runtime Consumer]
OBJREF --> FETCH
FETCH --> CACHE
CACHE --> RETURN
```
* * *
# Important Principle
Object retrieval supports:
- deduplication
- CDN caching
- immutable references
- snapshot reproducibility
- derived object generation
* * *
# 5. Native Access
## Purpose
Bridge semantic objects into local/native applications.
> Implementation status: TODO. The current integration-adapter runtime only
> permits hosted HTTP(S) viewers and editors. Direct executable launch requires
> a separately designed local helper, temporary workspace, credential store,
> executable allowlist, conflict detection, and Revision write-back protocol.
* * *
# Example
| Semantic Link | Native Projection |
| --- | --- |
| `teams.office.com` | `msteams://` |
| `mailto:` | Mail client |
| `tel:` | Dialer |
| `vscode://` | VS Code |
| `figma://` | Figma |
| `slack://` | Slack |
* * *
# Native Resolution Pipeline
```mermaid
graph TD
LINK[Semantic Link]
RES[Resolver]
NATIVE[Native Handler]
WEB[Web Fallback]
LINK --> RES
RES --> NATIVE
RES --> WEB
```
* * *
# Important Principle
Links represent:
```text
semantic execution intent
```
NOT fixed transport mechanisms.
* * *
# Runtime chooses:
- native app
- browser fallback
- embedded projection
- external mounted surface
depending on environment.
* * *
# 6. Workspace Plane vs Revision Plane
## Workspace Plane
Mutable editing world.
Contains:
- path
- alias
- asset
- live editors
* * *
# Revision Plane
Immutable historical world.
Contains:
- revision
- sha256 snapshots
- derived projections
* * *
# Important Principle
Editors operate on:
```text
workspace objects
```
Revisions store:
```text
immutable semantic snapshots
```
* * *
# 7. Canonical Storage Principle
Canonical storage depends on object class.
* * *
# Flow Articles
Canonical form:
```text
HTML DOM
```
* * *
# Presentation Objects
Canonical form:
```text
presentation object graph
```
Projected into:
- SVG sections
- HTML slides
- print layouts
* * *
# Spreadsheet Objects
Canonical form:
```text
structured semantic table graph
```
Projected into:
- HTML tables
- spreadsheet runtime
- CSV
- print layouts
* * *
# Canvas/Whiteboard Objects
Canonical form:
```text
spatial object graph
```
Projected into:
- SVG
- interactive canvas
- thumbnails
* * *
# Important Principle
Projection-specific formats are:
```text
runtime surfaces
```
NOT canonical storage truth.
* * *
# 8. Embedded Object Model
Embedded objects may be:
| Type | Meaning |
| --- | --- |
| Inline | Embedded payload |
| Linked | Immutable external object |
| Mounted | Live external application |
| Derived | Generated projection |
| Lazy | Runtime-fetched object |
* * *
# Example
```html
<embedded-object
kind="drawio"
immutable-ref="sha256:abcd"
></embedded-object>
```
* * *
# Important Principle
Large payloads should generally use:
```text
external immutable object references
```
instead of giant inline blobs.
* * *
# 9. Semantic Link Model
Semantic links may resolve into:
- knowledge paths
- projections
- native applications
- external services
- embedded runtimes
* * *
# Example
```text
/research/network/bgp
```
may contain:
```text
teams://
mailto:
vscode://
figma://
tel:
```
runtime-aware semantic links.
* * *
# 10. Final Architectural Direction
Pages Wiki evolves toward:
```text
semantic knowledge runtime environment
```
rather than:
```text
filesystem-oriented wiki
```
* * *
# Conceptual Summary
| Layer | Responsibility |
| --- | --- |
| Knowledge Access | Semantic navigation |
| Projection Access | Runtime interpretation |
| Editing Access | Authoring surfaces |
| Object Access | Immutable retrieval |
| Native Access | Local runtime execution |
* * *
# Final Principle
Pages Wiki separates:
```text
meaning
identity
projection
editing
storage
runtime execution
```
into independent architectural planes.
+644
View File
@@ -0,0 +1,644 @@
# Address Model Draft
## Goal
Separate:
```text
identity
reachability
semantic meaning
projection
mutability
```
into independent address layers.
The addressing model is designed to support:
- semantic navigation
- immutable revisions
- mutable editing workspaces
- object storage
- projection rendering
- delegated sharing
- graph-capable knowledge topology
## Core Principle
```text
Address != Meaning
```
Addresses are:
```text
reachable locators
```
NOT semantic meaning themselves.
* * *
# Address Layer Overview
| Scheme | Purpose | Mutable | Human-oriented |
| --- | --- | --- | --- |
| `path:` | Semantic navigation | yes | yes |
| `uuid:` | Stable Page/Object identity | no | partially |
| `rev:` | Immutable historical snapshot | no | no |
| `sha256:` | Immutable content identity | no | no |
| `asset:` | Mutable editable workspace | yes | partially |
| `alias:` | Semantic pointer | yes | yes |
| `share:` | Delegated capability projection | caveat-bound | yes |
| `derived:` | Generated object | preferably no | no |
| `external:` | Foreign mounted object | provider-defined | partially |
* * *
# 1. path:
## Purpose
Human semantic routing namespace.
* * *
# Example
```text
/research/network/bgp
/dev/api/auth
/hr/onboarding
```
* * *
# Properties
Property Value mutable yes human-readable yes semantic yes canonical identity no
* * *
# Important Principle
Paths are:
```text
semantic navigation surfaces
```
NOT canonical object identities.
* * *
# Resolution Flow
```text
path
↓
uuid
↓
revision
↓
projection
```
* * *
# 2. uuid:
## Purpose
Stable semantic object identity.
* * *
# Example
```text
uuid:550e8400-e29b-41d4-a716-446655440000
```
* * *
# Properties
Property Value mutable no semantic no canonical identity yes stable yes
* * *
# Important Principle
UUIDs are:
```text
opaque reachable locators
```
NOT semantic meaning.
* * *
# UUID Targets
UUID may identify:
- HTML document object
- PDF semantic object
- Spreadsheet object
- DrawIO semantic object
- Presentation object
- Embedded semantic object
* * *
# Important Principle
UUID does NOT imply rendering format.
Projection decides interpretation.
* * *
# 3. rev:
## Purpose
Immutable historical semantic snapshot.
* * *
# Example
```text
rev:018f6b5d...
```
* * *
# Properties
Property Value immutable yes historical yes reproducible yes
* * *
# Revision Responsibilities
A revision freezes:
- HTML structure
- embedded object references
- semantic state
- projection state
A Revision has its own UUID. Immutable payload identity is represented
separately by `sha256:`. A Revision record links a Page UUID, language tag,
Document hash, authoring metadata, and creation time.
* * *
# Important Principle
Revisions MUST reference:
```text
immutable object references only
```
* * *
# Forbidden
```text
revision -> mutable asset HEAD
```
* * *
# Allowed
```text
revision -> sha256 immutable snapshot
```
* * *
# 4. sha256:
## Purpose
Immutable content-addressed object identity.
* * *
# Example
```text
sha256:deadbeef...
```
* * *
# Properties
Property Value immutable yes content-addressed yes deduplicatable yes cacheable yes
* * *
# Typical Targets
- DrawIO payload
- Excalidraw payload
- PDF binary
- Image
- CSS snapshot
- JS snapshot
- PPTX binary
- XLSX binary
* * *
# Important Principle
sha256 objects are:
```text
historical truth
```
* * *
# 5. asset:
The normative lifecycle, freezing, authorization, and retention rules are in
[`asset-model.md`](asset-model.md).
## Purpose
Mutable editable workspace object.
* * *
# Example
```text
asset:drawio:network-core
asset:xlsx:q1-budget
asset:slide:roadmap
```
* * *
# Properties
Property Value mutable yes editable yes workspace-oriented yes
* * *
# Important Principle
Assets represent:
```text
current editable HEAD
```
* * *
# Save Flow
```text
asset mutable HEAD
↓
save
↓
snapshot
↓
sha256 immutable object
```
For HTML Revisions, supported embedding attributes containing an `asset:`
address are resolved at save time. Unresolved Asset HEADs reject the save.
* * *
# Important Principle
Asset identity represents:
```text
semantic continuity of editing
```
NOT immutable historical truth.
* * *
# 6. alias:
## Purpose
Semantic indirection pointer.
* * *
# Example
```text
alias:latest-network-design
```
* * *
# Properties
Property Value mutable yes semantic yes canonical no
* * *
# Typical Uses
- latest stable design
- current handbook
- canonical public URL
- migration pointer
* * *
# 7. share:
## Purpose
Delegated capability projection.
* * *
# Example
```text
share:abcdef
```
* * *
# Typical Projection
```text
/s/abcdef
```
* * *
# Properties
Property Value temporary usually delegated yes capability-bound yes
* * *
# Internal Model
Share addresses usually resolve to:
```text
macaroon delegated capability
```
* * *
# 8. derived:
## Purpose
Generated/transformed object identity.
* * *
# Example
```text
derived:thumbnail:sha256:abcd
derived:ocr:sha256:abcd
derived:preview-svg:sha256:abcd
```
* * *
# Properties
Property Value generated yes reproducible usually immutable preferred
* * *
# Typical Derived Objects
- thumbnails
- OCR text
- SVG previews
- rendered PDF pages
- AI summaries
- embeddings
* * *
# 9. external:
## Purpose
Mounted foreign object reference.
* * *
# Example
```text
external:gdrive:file123
external:figma:abcd
external:slack:thread456
```
* * *
# Properties
Property Value foreign ownership yes mutability external-defined projection-capable yes
* * *
# Runtime Resolution Pipeline
```text
path
↓
uuid
↓
revision
↓
projection
↓
embedded object resolution
↓
sha256 / asset / external
↓
renderer
```
* * *
# Projection Principle
## Address != Rendering
Rendering is determined by:
- mime type
- object class
- projection adapter
- user context
- runtime capability
* * *
# Example
```text
uuid:abcd
```
may render as:
- HTML article
- PDF viewer
- slide deck
- spreadsheet runtime
- whiteboard canvas
depending on projection adapter.
The adapter contract, bounded-delivery rules, and structured-format roadmap
are defined in [projection-adapters.md](projection-adapters.md).
* * *
## Raw Document Projection
Raw payload delivery is a projection, not another identity namespace. The
HTTP form therefore carries an explicit internal address:
```text
/v/raw/path:/manual/report
/v/raw/uuid:550e8400-e29b-41d4-a716-446655440000
/v/raw/rev:018f6b5d...
```
Viewer embeds `path:` when displaying the current semantic projection.
Immutable Revision Viewer embeds `rev:`. A bare `/raw/<semantic-path>` is
forbidden because it conflates projection, reachability, and identity.
## Path Rename
Rename changes reachability, not Page identity. It is committed atomically as:
```text
link(new-path, existing-page-uuid)
unlink(old-path, existing-page-uuid)
```
Recursive rename applies the same mapping to every descendant in one database
transaction. Destination collisions abort the entire operation. When the
basename of an extension-bearing file changes, the latest language projections
receive a new Revision carrying the new `filename`; older Revisions remain
immutable.
* * *
# Mutability Model
| Layer | Mutable |
| --- | --- |
| `path:` | yes |
| `alias:` | yes |
| `asset:` | yes |
| `uuid:` | no |
| `rev:` | no |
| `sha256:` | no |
| `derived:` | preferably no |
* * *
# Important Separation
## Workspace Plane
Mutable editing world.
Contains:
- asset:
- path:
- alias:
* * *
# Revision Plane
Immutable historical world.
Contains:
- rev:
- sha256:
- derived:
* * *
# Core Architectural Principle
Pages Wiki separates:
```text
semantic navigation
identity
content storage
projection
editing
history
```
into independent planes.
* * *
# Conceptual Summary
| Layer | Responsibility |
| --- | --- |
| `path:` | Semantic routing |
| `uuid:` | Stable Page/Object identity |
| `rev:` | Historical snapshot identity |
| `sha256:` | Immutable content reality |
| `asset:` | Mutable editing continuity |
| Projection | Runtime interpretation |
| Capability | Authorized reachability |
* * *
# Final Conceptual Direction
Pages Wiki evolves toward:
```text
semantic knowledge routing
+
immutable object graph
+
projection-oriented rendering
+
capability-based reachability
```
rather than:
```text
filesystem-oriented page hierarchy
```
+463
View File
@@ -0,0 +1,463 @@
# Overview
## Concept
Pages Wiki is designed as **semantic locality driven knowledge system** rather than a traditional filesystem wiki.
### Key properties
- Hierarchy-oriented UI
- Graph-capable Internal model
- Language-neutral Semantic paths
- Capability-centric authorization
- Delegated access via macaroons xxx
## Implementation component names
The adopted subsystem names and boundaries are defined in
[Component names and responsibilities](ja/component-names.md).
Grammaratur owns document-format conversion and authoring fidelity, Dregarnuhr
owns in-process scheduling, and Geduldh owns immutable payload storage and bounded
materialization. Mestionora owns SQLite search/index projections with caller-supplied
read policies. Ordoschnellic owns bounded Web Push delivery and transport policy,
with recipient checks and transactional Inbox enqueue retained in the application.
Schutzaria evaluates trusted Path/locality/group/publication policy facts without I/O;
authentication, fact loading and transactional authority remain application-owned.
Kunstzeal owns built-in projection selection and bounded diagram-provider execution;
source authorization and sandbox delivery remain in the application.
Seheweit owns local resource collection and history-window/retention policy;
authorization, queue control and SQL persistence remain in the application.
All eight initial extractions preserve compatibility; their remaining capabilities
are tracked separately. These names do not rename Knowledge
City Model entities, public URIs, stored identifiers or deployment services.
## Knowledge City Model
Pages Wiki don’t only comprises Machine friendly but also Human-Facing friendly with Knowledge City Model.
Knowledge City Model constitute five administrative division unit called ‘domain’, ‘quartier’, ‘street’, ‘spot’, ‘page’.
- ‘page’ is minimal knowledge unit.
- ‘spot’ constitute multiple pages. Namely, spot is minimal knowledge group.
- An overgrown knowledge spot can’t be upgraded to a street.
- ‘zone’ is managed unit in data structure.
- ‘street’ is group of spot. ‘street’ used minimal knowledge city block.
- If existed mixed spot and various page, it is unconsolidated street.
- ‘quartier’ can be called if it consists of an aggregation of streets delineated through urban planning.
- An overgrown knowledge street can be upgraded to a quartier.
- ‘Domain’ is really site group. For example, ’Domain’ may indicate division and department group, project, and team. ‘Domain’ used as subdomain space in URI.
- ‘Domain’ managed by one ‘Campus’.
```mermaid
graph TD
Cm[Campus]
DmA[Domain: Sales Unit]
DmB[Domain: Technical Unit]
DmC[Domain: Management Unit]
%% Campus から Domain への接続
Cm-..->DmA
Cm-..->DmB
Cm-..->DmC
%% Zone定義(Quartier と Street を内包し、縦に並ぶよう調整)
subgraph Zone ["Zone"]
direction TB
%% Quartier レ法
QtA[Quartier: Network]
QtB[Quartier: Server]
QtC[Quartier: Application]
%% Street レ法
StA[Street: DNS]
StB[Street: Web]
StC[Street: Mail]
%% Zone内での上下関係を固定するための接続
QtB-->StA
QtB-->StB
StB-.if it is overgrown, upgrade to 'quartier'.->QtB
QtB-->StC
end
%% Domain から Zone(Quartier) への接続
DmB-->QtA
DmB-->QtB
DmB-->QtC
%% Zone(Street) から下位レイヤーへの接続
SpA[Spot: Web Infrastructure]
SpB[Spot: Wiki Application]
SpC[Spot: Search Engine]
StB-->SpA
StB-->SpB
StB-->SpC
%% Spot から Page への接続
PgA[Page: Access Control Model]
PgB[Page: Addressing Model]
PgC[Page: Authenticating Model]
SpB-->PgA
SpB-->PgB
SpB-->PgC
```
## Internal Semantic Structure
Pages Wiki is arranged by machine-friendly semantic structure.
The detailed responsibilities, invariants, schema, projections, API surface,
and implementation order for Quartier, Street, Spot, and MagicPage are defined
in [Knowledge City Model Detailed Design](knowledge-city-model.md).
Collaboration, notification, delegated sharing, extension providers, external
storage, malware inspection, controller DR, and multi-node HA must preserve
these identities through the contracts in
[Platform Extension Architecture](platform-extension-architecture.md). These
features extend Page/Revision/City projections; they do not introduce alternate
Page identity, mutable Revision payloads, or filesystem-style ownership.
- ‘ZoneTree’ is arranged tree-structured model for control ‘quartier’ and ‘street’, and so ‘spot’ as external pointer. ZoneTree referenced from domain.
- ‘SpotGraph’ is page pointer graph from spot.
- SpotGraph point page ID. Multiple spot can any page with page ID(called ‘MagicPage’).
- Revision
- A Page UUID owns an ordered Revision history.
- Each immutable Revision has a Revision UUID and references an immutable
Document by SHA-256.
- Revision identity and Document content identity are distinct.
- Revisions carry a BCP 47 language tag; `und` represents legacy or
language-unspecified content.
```mermaid
graph LR
Dm[Domain]
ZT[Zone Tree]
SG[Spot Graph]
PG[Pages]
RV[Revisions]
Dm--Domain refer to ZoneTree-->ZT
ZT--ZoneTree refer to spot with SpotGraph-->SG
SG--SpotGraph refer to page-->PG
PG--page refer to revisions-->RV
```
### MagicPage
The MagicPage can reach from multiple spot without depending quartier and street. So, A single Page may appear in multiple Spots.
This creates: rooted multitree
instead of strict filesystem hierarchy.
```mermaid
graph LR
SPA[Spot A]
SPB[Spot B]
SPC[Spot C]
P1[Page UUID=P1]
P2[Page UUID=P2]
SPA --> P1
SPB --> P1
SPC --> P2
```
## Language Projection Model
```mermaid
graph LR
REQ[Request]
AL[Accept-Language]
P[Page UUID]
JAJP[Revision ja-JP]
JA[Revision ja]
EN[Revision en]
REQ --> AL
AL --> P
P --> JAJP
JAJP -->|not found| JA
JA -->|not found| EN
```
### Example
```http
GET /dev/api/auth
Accept-Language: ja-JP,ja;q=0.9,en;q=0.8
```
# Canonical Identity Model
```mermaid
graph TD
DOMAIN[Domain UUID]
PATH[Semantic Path]
UUID[Page UUID]
REV[Revision]
DOMAIN --> PATH
PATH --> UUID
UUID --> REV
```
### Important Principle
```text
path != object
representation != object
(domain_uuid, path) = semantic address
object UUID = instance-global identity
```
A Domain owns a semantic path namespace, not the Page Object itself. The same
normalized path may exist in multiple Domains without conflict, and multiple
Domains may bind different paths to the same Page UUID. Cross-Domain links use
an absolute URL containing the destination FQDN. See
`domain-namespace-federation.md` for local/remote placement and Campus
authority modes.
## Authentication / Authorization Flow
```mermaid
graph TD
OIDC[OIDC Provider]
PR[Principal]
MB[Membership]
RL[Role]
CAP[Capability Graph]
OIDC --> PR
PR --> MB
MB --> RL
RL --> CAP
```
### Important Separation
| Layer | Responsibility |
| --- | --- |
| OIDC | Identity proof |
| Principal | Canonical internal identity |
| Membership | Organizational mapping |
| Role | Semantic grouping |
| Capability | Actual authority |
## Capability Model
```mermaid
graph TD
PR[Principal]
CAP[Capabilities]
SP[Spot]
PAGE[Page]
PR --> CAP
CAP --> SP
SP --> PAGE
```
## Example
```text
Alice
└─ auth-api:read
```
Spot:
```text
Auth API
└─ requires auth-api:read
```
## Capability Evaluation
```mermaid
graph TD
REQ[Read Request]
PR[Principal]
SP[Spot]
CAP[Capability Check]
ALLOW[Allow]
DENY[Implicit Deny]
REQ --> PR
REQ --> SP
PR --> CAP
SP --> CAP
CAP -->|has capability| ALLOW
CAP -->|missing capability| DENY
```
### Important Principle
```text
default deny
positive grants only
```
## Macaroon Delegation
```mermaid
graph TD
PR[Principal]
CAP[Capability]
MAC[Macaroon]
CAV[Caveats]
USR[Temporary User]
PR --> CAP
CAP --> MAC
MAC --> CAV
MAC --> USR
```
### Caveat Examples
- expires
- readonly
- network restriction
- lang restriction
- path restriction
## Share Link Model
```mermaid
graph TD
SHARE["/s/abcdef"]
MAC[Anonymous Macaroon]
SP[Spot]
VIEW[Read-only Projection]
SHARE --> MAC
MAC --> SP
SP --> VIEW
```
## Storage Model
```mermaid
graph TD
PAGE[Page UUID]
REV1[Revision 1]
REV2[Revision 2]
REV3[Revision 3]
PAGE --> REV1
PAGE --> REV2
PAGE --> REV3
```
The staged evolution to PostgreSQL/MySQL, Redis caches, S3 and network file
stores, together with backup/export/import and first-run setup, is defined in
[Storage Portability and Operations](storage-portability-and-operations.md).
The current supported runtime remains SQLite and local storage; the design
does not imply that remote backends are already selectable.
Metadata is stored in the database. Immutable Document payloads use a global
content-addressed filesystem:
```text
documents/
└─ sha256/
└─ ab/
└─ abcdef...
```
The payload extension is not used as identity. MIME type and canonical format
are metadata. A Revision row references the hash and never references a
mutable asset HEAD.
## Runtime Rendering Pipeline
```mermaid
graph TD
REQ[HTTP Request]
PATH[Semantic Path]
UUID[Page UUID]
LANG[Language Resolution]
REV[Revision]
AUTH[Capability Evaluation]
RENDER[Render]
REQ --> PATH
PATH --> UUID
UUID --> LANG
LANG --> REV
REV --> AUTH
AUTH --> RENDER
```
## Overall Architectural Direction
```mermaid
graph TD
UI[Hierarchy UI]
TREE[Rooted Tree]
GRAPH[Semantic Graph]
UUID[UUID Identity]
LANG[Language Projection]
CAP[Capability Access]
MAC[Macaroon Delegation]
UI --> TREE
TREE --> GRAPH
GRAPH --> UUID
UUID --> LANG
LANG --> CAP
CAP --> MAC
```
## Final Summary
### Pages Wiki is:
- hierarchy-oriented for humans
- graph-capable internally
- semantic-locality-driven
- UUID identity based
- language-neutral
- capability-centric
- projection-oriented
### NOT:
- filesystem ACL centric
- path identity centric
- language hierarchy centric
+34
View File
@@ -0,0 +1,34 @@
# Archive Import and Knowledge City Planning
Archive import is a staged, auditable operation. Uploading an archive never
writes Pages directly.
## Pipeline
1. Store the uploaded ZIP or tar-family archive in private import storage.
2. Queue `import.analyze`; foreground requests always take scheduler priority.
3. Validate entry names, link types, file count and expanded byte limits.
4. Produce an immutable dry-run manifest containing source path, destination
Path, detected content type and initial Quartier/Street/Spot suggestions.
5. Allow a curator to edit, merge, exclude and approve suggestions.
6. Queue import execution in bounded batches. Markdown is transpiled to the
canonical HTML Document while its source representation remains available.
7. Record created UUIDs and topology operations in an import journal so a
session can be reviewed, retried or rolled back.
## Safety limits
- archive upload: 512 MiB;
- expanded content: 2 GiB;
- entries: 10,000;
- absolute paths, `..`, NUL, hard links and symbolic links are rejected;
- supported staging formats: ZIP, tar, tar.gz/tgz and tar.xz/txz.
## Initial suggestion rules
The first directory component proposes a Quartier, the second proposes a
Street, and the nearest containing directory proposes a Spot. Markdown receives
higher initial confidence than opaque assets. Later scoring incorporates
front matter, headings, README/index portals, internal links, vocabulary and
existing City boundaries. Suggestions never mutate City topology until a
curator confirms the plan.
+247
View File
@@ -0,0 +1,247 @@
# Asset Model
## 1. Purpose
An Asset is a mutable authoring identity whose HEAD points to one immutable
Document. It represents continuity of editing for binary or application-native
material such as an image, DrawIO scene, Excalidraw scene, spreadsheet, slide
deck, audio file, or video file.
An Asset is not:
- a Semantic Path or a Viewer URL;
- a Page Revision;
- immutable historical truth;
- an authorization capability;
- a remote-instance address.
The identity layers remain distinct:
| Identity | Meaning | Mutable |
| --- | --- | --- |
| `asset:<type>:<name>` | Editable Asset identity and current HEAD | yes |
| `sha256:<hash>` | Exact immutable Document bytes | no |
| `rev:<uuid>` | Immutable Page Revision that may refer to Documents | no |
| `path:/...` | Domain-scoped semantic reachability | yes |
## 2. Address and scope
The canonical local address is:
```text
asset:<type>:<name>
```
Examples:
```text
asset:image:knowledge-city-logo
asset:drawio:network-core
asset:xlsx:q1-budget
```
`type` is a projection/editor hint, not trusted MIME metadata. `name` is the
stable authoring name. The complete address is Domain-local, case-sensitive,
and unique by `(domain_uuid, address)` in the `assets` table.
Asset authoring identity is Domain-scoped so separate Domains may independently
own `asset:image:logo`. Immutable Documents remain content-addressed and may be
deduplicated across Domains. A local `asset:` address must never be resolved on
another instance. A remote resource requires an absolute HTTPS URL and an
explicit federation or external-object contract.
## 3. Stored state
The current additive schema is:
| Column | Meaning |
| --- | --- |
| `domain_uuid` | Owning Domain; part of the composite primary key |
| `address` | Canonical `asset:` identity; part of the composite primary key |
| `object_uuid` | Stable Object UUID with kind `asset` |
| `head_document_hash` | Current immutable Document SHA-256; empty means no HEAD |
| `created_at_ms` | Asset creation time |
| `content_type` | Mutable MIME metadata for the current HEAD |
| `filename` | Mutable filename metadata for the current HEAD |
| `size_bytes` | Current HEAD byte length |
| `updated_at_ms` | Last HEAD update time |
The bytes are stored in the Document Store under their SHA-256. The Asset row
contains no duplicate body. Filename, MIME type, and size describe the mutable
HEAD but do not participate in Document identity. Immutable delivery uses
bounded content detection when historical metadata is unavailable.
## 4. State transitions
### Create or update HEAD
```text
input bytes
-> calculate SHA-256
-> atomically persist immutable Document
-> compare with Asset HEAD
-> create Asset or move HEAD
```
Writing identical bytes with unchanged MIME and filename is a no-op. Correcting
MIME or filename records a metadata/history update without changing the hash.
Moving HEAD never mutates or overwrites the
old Document. The local CLI currently implements this operation through
`put LOCAL asset:<type>:<name>` and `edit asset:<type>:<name>`.
Asset HEAD updates do not create Page Revisions. An old Document remains live
when a Page Revision has frozen a reference to it; otherwise it may become
garbage after it is no longer an Asset HEAD. Garbage collection must treat
current Asset HEADs and `revision_document_refs` as roots.
### Read
`view`, `cat`, and `get` resolve the Asset HEAD at operation time. These are
mutable reads and must not be used for reproducible rendering or historical
export. Reproducible consumers use the resolved `sha256:` identity.
### Delete
Administrator HTTP deletion removes the mutable Asset identity only; it does
not delete a Document still
referenced by a Revision, another Asset HEAD, export, or retention root.
Automatic retention and physical Document collection policy remain pending.
## 5. Revision save and freezing
A saved Page Revision must never use a live `asset:` dependency as its rendered
payload. It does retain the authoring address as provenance so a later edit can
resolve the then-current HEAD. For HTML payloads, the server resolves `asset:`
values at save time:
| Authoring input | Stored Revision |
| --- | --- |
| `asset-ref="asset:…"` | Retain `asset-ref`; add/update `immutable-ref="sha256:<hash>"` |
| `immutable-ref="asset:…"` | Normalize to retained `asset-ref` plus fixed `immutable-ref` |
| `src="asset:…"` | Fixed `src="/_/obj/sha256:<hash>"` plus `data-pageswiki-asset-src="asset:…"` |
| `href="asset:…"` | Fixed `href` plus `data-pageswiki-asset-href` provenance |
| `poster="asset:…"` | Fixed `poster` plus `data-pageswiki-asset-poster` provenance |
| `data="asset:…"` | Fixed `data` plus `data-pageswiki-asset-data` provenance |
Only HTML attribute values are transformed. Explanatory prose, code blocks,
and non-HTML payloads are not rewritten. A missing Asset or an Asset without a
valid HEAD rejects the save with `ErrEmbeddedAssetUnavailable`; silently
leaving a mutable dependency is forbidden.
On a later save, the provenance address is resolved again and the fixed hash is
updated to the current Asset HEAD. Merely viewing an old Revision never resolves
the mutable address and therefore remains reproducible.
### Authoring forms
Markdown may use ordinary resource and link syntax because compilation produces
the supported HTML attributes:
```markdown
![Knowledge City logo](asset:image:knowledge-city-logo)
[Download the workbook](asset:xlsx:q1-budget)
```
Equivalent HTML examples are:
```html
<img src="asset:image:knowledge-city-logo" alt="Knowledge City logo">
<a href="asset:xlsx:q1-budget">Download the workbook</a>
<video poster="asset:image:intro-poster">
<source src="asset:video:intro">
</video>
<embedded-object
kind="drawio"
asset-ref="asset:drawio:network-core">
</embedded-object>
```
`asset:` is accepted only in `asset-ref`, legacy `immutable-ref`, `src`, `href`,
`poster`, and `data`. CSS `url()`, `srcset`, inline scripts, arbitrary
`data-*` attributes, and plain text are not resolvers. They require a future
typed parser rather than unsafe textual replacement.
### Access surfaces
| Surface | Address | Semantics |
| --- | --- | --- |
| CLI authoring | `asset:<type>:<name>` | Mutable HEAD read/write in selected Domain |
| Administrator HEAD URL | `/v/raw/asset:<type>:<name>` | Resolves current HEAD on every request; private and non-canonical |
| HTML/Markdown authoring | `asset:<type>:<name>` | Resolved only while saving a Revision |
| Stored semantic reference | `asset-ref="asset:…"` | Provenance for later editing; never rendered directly |
| Stored immutable reference | `sha256:<hash>` | Exact dependency identity |
| Browser delivery | `/_/obj/sha256:<hash>` | Authorized immutable GET/HEAD/Range |
| Management API | `/_/api/v1/assets?address=asset:…` | Domain-admin metadata and mutation surface |
The mutable Asset URL is an authenticated administrator authoring projection;
it is never emitted into published Pages or historical Revisions. It uses a
private revalidation cache policy and reports both the Asset address and
resolved HEAD hash in response headers. Published Pages always load the frozen
Document URL.
Every frozen dependency is recorded as:
```text
revision_document_refs(revision_uuid, document_hash)
```
This relation is part of Revision truth. It enables dependency inspection,
authorization, export, and garbage-collection reachability without reparsing
historical HTML.
## 6. Delivery and authorization
Immutable bytes are delivered through:
```text
/_/obj/sha256:<hash>
```
Knowing a hash is not authorization. Delivery succeeds only when the Document
is either the body of, or a frozen dependency of, a Revision whose Object has a
readable Path in the current Domain. The response exposes Object, Revision, and
Document identities in the PagesWiki metadata headers and supports ETag, HEAD,
and byte ranges.
The current HEAD of a Domain Asset may also be inspected through the same
immutable URL by a Domain or Campus administrator, even before any Revision has
frozen it. This administrative exception is for Asset management and does not
make the mutable HEAD anonymously readable. Once frozen into a Page Revision,
ordinary access follows that Page's read authorization.
The trusted local CLI bypasses HTTP Capability evaluation and is controlled by
database and filesystem permissions. The administrator HTTP API is restricted
to Domain or Campus administrators and requires an Asset HEAD ETag for updates
and deletion, or `If-None-Match: *` for creation. It records immutable HEAD
history with actor identity. Moving this endpoint into the typed policy matrix
and general audit outbox remains pending. Last-writer-wins HTTP updates are not
accepted.
## 7. Consistency rules
1. `head_document_hash` and every frozen dependency must be a lowercase,
64-character SHA-256.
2. Asset HEAD movement never changes existing Documents or Revisions.
3. Revision save resolves all supported embedded Asset references or fails.
4. `asset:` is authoring identity; `sha256:` is delivery/storage identity.
5. A hash, Asset address, or Object UUID never grants access by itself.
6. Cross-instance Asset resolution is forbidden without an explicit origin.
7. GC must preserve Documents reachable from Revisions, frozen references,
current Asset HEADs, and configured retention roots.
## 8. Implementation status
| Capability | Status |
| --- | --- |
| Local CLI create/update/read/edit | active |
| Content-addressed Document persistence | active |
| HTML save-time freezing | active |
| Revision-to-Document dependency records | active |
| Authorized immutable HTTP delivery | active |
| Asset owner excluded from Zombie Object GC | active |
| Asset MIME/filename/size metadata | active |
| Administrator HTTP CRUD and ETag/CAS updates | active |
| Initial Domain Asset management UI | active |
| History/restore and dependency detail UI | active |
| Delete/retention UI and audit trail | designed, not implemented |
| Remote/federated Asset synchronization | not designed as implicit lookup |
+159
View File
@@ -0,0 +1,159 @@
# Authentication and Authorization Backend
## Boundary
Authentication proves an identity. Every authenticator produces the same
internal `principal_uuid`; authenticators never grant document access directly.
Authorization is Domain-scoped and evaluates positive grants only:
```text
request -> session -> principal -> Domain membership -> role -> capability
```
Campus owns trusted identity-provider configuration. A Domain owns membership,
roles, grants, session policy, and the default authorization mode. Managed
Campus provider metadata is inherited from its controller; secrets remain on
the instance that performs authentication and are never federated.
## Authentication methods
| Method | Stable external key | Secret location |
| --- | --- | --- |
| password | normalized username | Argon2id verifier in credential table |
| OIDC | exact `(issuer, subject)` | encrypted client secret/config |
| LDAP | `(provider UUID, immutable directory attribute)` | encrypted bind secret |
| TOTP | principal + factor UUID | encrypted seed; recovery codes hashed |
| magic link | normalized verified email | one-use hashed challenge |
| device approval | requesting-device challenge UUID | one-use hashed challenge |
OIDC groups and LDAP groups are attributes. Explicit mapping rules may create
or remove Membership/Role bindings, but group names are not capabilities.
The route policy, internal group schema, `/g/` namespace, and endpoint matrix
are specified in [Endpoint Authorization and Groups](authorization-groups.md).
## Core records
- `principals`: canonical person/service identity and lifecycle state.
- `principal_identities`: unique provider subjects attached to a Principal.
- `auth_credentials`: password/TOTP/recovery material and version metadata.
- `domain_memberships`: Principal membership in one Domain.
- `auth_roles`, `auth_role_bindings`: Domain roles and membership assignment.
- `groups`, `group_members`, `auth_group_role_bindings`: Domain-scoped
Principal collections and their Role assignment.
- `auth_capability_grants`: positive operation grants with typed targets.
- `auth_sessions`: opaque, hashed, rotating browser sessions.
- `auth_challenges`: short-lived one-use login/MFA/device challenges.
- `auth_audit_log`: append-only authentication and policy decisions.
Capabilities use verbs `read`, `create`, `upload`, `write`, `rename`, `delete`,
`link`, `manage`, and `delegate`. Targets are `domain`, `quartier`, `street`, `spot`, `path`, or
`object`. Locality inheritance is resolved by the address model; an exceptional
Page/Object grant does not rewrite the path hierarchy.
## Evaluation
1. Resolve Domain before reading the session.
2. Hash the opaque cookie and load an unexpired, unrevoked session for that
Domain. Reject suspended/quarantined Principals.
3. Resolve Membership and its Role bindings.
4. Match the requested verb and target against positive grants. `manage`
includes administrative operations but does not imply `delegate`.
5. Record denied sensitive operations and every policy/configuration mutation.
No matching grant means deny. During migration, `authorization_mode=legacy`
leaves existing anonymous behavior intact. Administrators bootstrap an owner,
verify access, then switch a Domain to `enforcing`. New Domains should default
to `enforcing` once the bootstrap flow is complete.
## HTTP flows
Implementation clarification (2026-11-01): clients requesting JSON from
`POST /_/auth/login` (or `/totp`) receive an opaque Bearer session rather than
extracting a browser cookie. The remote CLI follows this existing flow and the
same Domain membership/capability checks; see [remote CLI](ja/remote-cli.md).
It does not implement an OIDC device handoff yet. The cookie descriptions below
apply to browser sessions, not a requirement for all HTTP clients.
- `/_/auth/login`, `/_/auth/logout`, `/_/auth/callback/oidc/{provider}`
- `/u/{username}` is the Domain-scoped profile and user-Wiki root.
`/u/{username}/_/edit` is its owner-only profile editor and
`/_/auth/profile` redirects there. Other descendants are ordinary semantic
Wiki paths. Usernames exclude dots so the public handle can be derived as
`{username}.{domain FQDN}` without ambiguity.
- `/_/auth/challenge/password`, `/totp`, `/magic-link`, `/device`
- `/_/api/v1/auth/session`, `/principals`, `/roles`, `/grants`, `/audit`
Cookies are `Secure`, `HttpOnly`, `SameSite=Lax`, host-only, and contain only a
random secret whose SHA-256 hash is stored. Login rotates the session; password
or factor changes revoke all Principal sessions. State/nonce/PKCE are mandatory
for OIDC. Magic links and device approvals store only a token hash, are
single-use, expire quickly, and are bound to Domain and intended action.
TOTP seeds are encrypted with AES-GCM using a 256-bit instance key stored as
`auth.key` beside the Document Store (`0600`), or derived from
`PAGESWIKI_AUTH_KEY`. Enrollment remains disabled until a valid six-digit code
is confirmed. Login challenges expire after five minutes, allow five attempts,
and reject reuse of an accepted 30-second counter.
OIDC providers are configured per Domain from the authentication panel. The
flow uses discovery, Authorization Code, PKCE S256, state, and nonce, then
resolves the subject through UserInfo. A first login may bind `(issuer, sub)`
only to an existing Principal with the same verified email; it never creates a
role or capability from provider claims. Client secrets are encrypted with the
instance authentication key.
LDAP providers are also Domain-scoped. The implemented login sequence follows
the same separation used by Wiki.js' LDAP authenticator:
1. connect to an `ldap://` or `ldaps://` endpoint with an 8-second socket
deadline; optionally upgrade an `ldap://` connection with StartTLS;
2. bind the configured search service account, or use an anonymous search when
both bind fields are empty;
3. insert the RFC 4515-escaped login into a configured filter containing
`{{username}}`, search below the configured Base DN, and require exactly one
result (server size limit 2, time limit 6 seconds);
4. read an explicitly configured immutable ID, local username, email, and
display-name attribute;
5. bind as the returned user DN with the submitted password. A search result by
itself is never accepted as proof of authentication;
6. resolve the first login by an exact unique local username or email, then
persist `(provider UUID, immutable ID)`. Following logins use only that
immutable binding. Optional automatic provisioning creates an ordinary
`member` with no local password credential.
The immutable attribute should be `entryUUID` for common LDAP servers or the
binary `objectGUID` for Active Directory. DN, email, display name, and account
name are mutable attributes and therefore are not stable subjects. Bind
passwords use the instance authentication-key encryption; leaving the password
blank in the settings form preserves the existing encrypted value. TLS
certificate validation is on by default, a private CA can be appended as PEM,
and disabling validation is an explicit per-Domain administrator action.
The settings panel exposes a saved-configuration login test. User passwords are
used only for the user-DN bind and are neither persisted nor logged. Connection
admission is bounded to four process-wide LDAP operations. The browser receives
the same generic invalid-credentials response for lookup and password failures;
the server log retains the provider-side error without the submitted username
or password.
SMTP magic links target existing active/read-only Principals by normalized
email. The public request response never reveals whether an address exists.
Links contain a random token whose hash alone is stored, expire after fifteen
minutes, and are consumed atomically once. SMTP credentials use the same
encrypted provider-secret storage. Delivery is admitted through a bounded
background slot so an SMTP server cannot hold the login request open.
LDAP and SMTP run through bounded worker queues with timeouts; request handlers
do not wait indefinitely on an external directory or mail server. A durable
authentication-attempt quota keyed by Domain, normalized account identifier,
source prefix, and challenge kind remains required before an Internet-facing
multi-tenant deployment; bounded provider admission is not a rate limiter.
## Delivery order
1. Common schema, Principal normalization, sessions, RBAC evaluation, audit.
2. Bootstrap owner plus password/Argon2id and optional TOTP.
3. OIDC Authorization Code + PKCE.
4. SMTP magic link and device approval challenges.
5. LDAP bind/search with immutable identity mapping.
6. Delegated macaroon capabilities after canonical RBAC is enforced.
+642
View File
@@ -0,0 +1,642 @@
# Pages Wiki vNext Authentication / Authorization Draft
## Goal
Separate:
```text
Authentication
Identity
Capability
Delegation
Projection
```
into independent layers.
Avoid tightly coupling:
```text
OIDC groups
filesystem ACLs
path hierarchy
```
to the knowledge topology.
* * *
# Core Philosophy
## Authentication != Authorization
Authentication answers:
```text
Who are you?
```
Authorization answers:
```text
What may you do?
```
These are intentionally separated.
* * *
# High-Level Architecture
```text
OIDC
↓
Principal
↓
Membership / Roles
↓
Capability Graph
↓
Macaroon Delegation
↓
Runtime Policy Evaluation
```
* * *
# Layer 1: Authentication
## Responsibility
Identity proof only.
* * *
# Recommended Mechanism
OIDC / OpenID Connect.
Examples:
- Keycloak
- Authentik
- Azure AD
- Google Workspace
- GitHub OIDC
* * *
# OIDC Output
Example claims:
```json
{
"iss": "...",
"sub": "...",
"email": "...",
"groups": [...]
}
```
* * *
# Important Principle
OIDC groups are NOT canonical authorization objects.
They are only:
```text
external identity attributes
```
* * *
# Layer 2: Principal
## Responsibility
Canonical internal identity normalization.
* * *
# Principal Model
```text
principal_id
issuer
subject
display_name
email
```
* * *
# Important Principle
The canonical identity inside Pages is:
```text
principal_id
```
NOT:
```text
OIDC group
email address
username
```
* * *
# Layer 3: Capability Model
## Responsibility
Represent authority within the knowledge topology.
* * *
# Capability Philosophy
Capabilities are attached primarily to:
```text
Zone
Spot
```
rather than individual Pages.
* * *
# Why
Knowledge naturally clusters around semantic localities.
ACLs scale better when attached to locality.
* * *
# Example
```text
Spot(Auth API)
allow:
engineering-reader
api-editor
```
Pages inherit effective visibility from their locality.
* * *
# Capability Model
```text
principal
↓
membership
↓
role
↓
capability
```
* * *
# Example
```text
engineering-team
↓
api-editor
↓
write Spot(Auth API)
```
* * *
# Important Principle
Roles are internal semantic policy objects.
They are NOT identical to OIDC groups.
* * *
# OIDC Mapping
```text
OIDC group
↓
Pages membership
↓
Pages role
↓
Capability
```
* * *
# Capability Target Types
Capabilities may target:
| Object Type | Meaning |
| --- | --- |
| Zone | Containment locality |
| Spot | Semantic locality |
| Page | Exceptional override only |
| Binding | Projection-level visibility |
* * *
# Default Authorization Strategy
## Default Deny
No capability means:
```text
access denied
```
* * *
# Positive Grant Only
Avoid large explicit deny graphs.
Prefer:
```text
grant omission
```
instead of:
```text
explicit deny
```
* * *
# Why
Explicit deny causes:
- precedence complexity
- inheritance conflicts
- ACL explosion
- runtime ambiguity
* * *
# Recommended Model
```text
allow-only locality policies
```
plus:
```text
principal-side overlays
```
for exceptional cases.
* * *
# Exceptional Restriction Handling
Instead of:
```text
deny alice
```
prefer:
```text
alice.status = suspended
```
or:
```text
alice lacks capability
```
* * *
# Principal States
Examples:
```text
active
readonly
suspended
quarantined
audit-only
```
* * *
# Layer 4: Macaroon
## Responsibility
Portable delegated constrained capability.
* * *
# Important Principle
Macaroon is NOT the canonical ACL database.
Macaroon is:
```text
authority artifact
delegated capability token
portable constrained access proof
```
* * *
# Macaroon Role
Macaroons transport delegated authority.
* * *
# Example
Canonical server-side capability:
```text
Alice
└─ read Spot(Auth API)
```
Delegated macaroon:
```text
read-only
spot=auth-api
expires<2026-02-01
office-network-only
```
* * *
# Caveats
Macaroon caveats attenuate authority.
Examples:
Caveat Meaning expires time restriction network source network restriction readonly capability narrowing path locality restriction lang projection restriction method operation restriction
* * *
# Important Property
Macaroons can only:
```text
reduce authority
```
never expand it.
* * *
# Runtime Validation
```text
1. Verify macaroon signature
2. Verify caveats
3. Verify delegated capability validity
4. Apply runtime policy
```
* * *
# Share Links
Example:
```text
/s/abcdef
```
may internally represent:
```text
anonymous delegated macaroon capability
```
* * *
# Important Property
Share links may work without OIDC login.
This enables:
```text
identity-less delegated access
```
* * *
# Capability vs ACL
## Traditional ACL
```text
Object
└─ Subjects
```
* * *
# Capability-Oriented Model
```text
Principal
└─ Possessed Authorities
```
* * *
# Real-World Analogy
Pages authorization behaves more like:
```text
building access control
```
than filesystem ACLs.
* * *
# Example
```text
Zone = building
Spot = room
Capability = key
Macaroon = temporary visitor pass
```
* * *
# Locality Unlock Model
## Spot
```text
requires capability:
auth-api:read
```
* * *
# Principal
```text
possesses:
auth-api:read
```
* * *
# Runtime
```text
Does principal possess required capability?
```
* * *
# Capability Delegation
Example:
```text
Alice delegates:
read-only access
to Spot(Auth API)
valid for 24 hours
```
This maps naturally to:
```text
temporary visitor badge
```
* * *
# Runtime Authorization Evaluation
## Without Macaroon
```text
principal session
↓
effective capability lookup
```
* * *
# With Macaroon
```text
verify macaroon
↓
apply caveats
↓
treat as delegated capability
```
* * *
# Runtime Evaluation Components
| Component | Responsibility |
| --- | --- |
| Principal | Identity |
| Capability Graph | Canonical authority |
| Macaroon | Delegated authority |
| Runtime Policy | Dynamic constraints |
| Effective Cache | Acceleration |
* * *
# Runtime Constraints
Examples:
```text
time
network
risk
device
lang
projection
```
* * *
# Effective Capability Cache
Recommended for scalability.
Example:
```text
principal_effective_capabilities
```
* * *
# Important Principle
The source of truth is:
```text
server-side capability graph
```
NOT macaroon tokens.
* * *
# Architectural Direction
Layer Model Authentication OIDC Identity Principal Authorization Capability graph Delegation Macaroon Constraints Caveats Runtime Effective policy evaluation
* * *
# Conceptual Summary
Pages Wiki authorization is evolving toward:
```text
semantic locality capability system
```
rather than:
```text
filesystem ACL system
```
+410
View File
@@ -0,0 +1,410 @@
# Endpoint Authorization and Groups
## Scope
2026-09-14 extension: [locality access and publication](locality-access-publication.md)
adds restrictive named user/group layers over existing grants and a separate
explicitly approved anonymous semantic/raw path. For Objects, all local
restrictive alias caps apply before the original positive-grant/any-readable-
alias rule below. Existing group profiles and administrative roles are unchanged.
This document completes the backend design between authentication and the
existing capability graph. It specifies route-level enforcement, target
resolution, groups, and the `/g/` namespace. Authentication providers prove a
Principal; neither an OIDC/LDAP group nor an HTTP route grants authority by
itself.
## Request authorization pipeline
Every registered route has a static `RoutePolicy`. Authorization must not infer
authority from arbitrary URL substring matching.
```go
type RoutePolicy struct {
Name string
Methods map[string]MethodPolicy
Authentication AuthenticationPolicy // public, optional, principal, service
CSRF bool
}
type MethodPolicy struct {
Capability string // read, write, rename, delete, link, manage, delegate
Resolve TargetResolver
}
type AuthorizationTarget struct {
DomainUUID string
Type string // domain, quartier, street, spot, path, object, group
ID string // canonical UUID where one exists
}
```
The request sequence is fixed:
1. Normalize the request and attach a request ID.
2. Resolve the local Domain from the effective host. Reject remote placement.
3. Match a registered route and method. An unregistered method returns `405`.
4. Resolve credentials into an optional Principal or a service/delegation
context. Domain browser sessions and Bearer tokens use the same stored
session record; adapter, federation, and future share tokens are distinct
credential types and are never promoted to Principal sessions.
5. Guard browser mutations with same-origin Fetch Metadata/Origin before any
authorization bypass. Cookie requests missing that evidence require a
matching Referer; otherwise deny. Explicit Bearer/service clients may omit
browser metadata, but actual cross-origin requests are still rejected. See
[browser request boundaries](browser-request-security.md).
6. Resolve the target without performing the operation. Path input is
normalized and resolved in the current Domain; Object and Revision routes
resolve back to their owning Object and all currently relevant bindings.
7. Evaluate Principal state, Domain membership, direct Role bindings, group
Role bindings, and positive grants. No match means deny.
8. Invoke the handler only after authorization. Record sensitive allows,
administrative mutations, and all denies in `auth_audit_log`.
Target resolvers return `404` only after the caller has enough `read` authority
to distinguish absence. Otherwise they return `403`, preventing private-object
enumeration. Authentication failures are `401` JSON for APIs and a login
redirect for browser navigation. Authorization failures are `403` in both.
For a path, candidate grants are the exact Object/Path exceptional grant, its
Spot/Street/Quartier ancestry once ZoneTree exists, and finally the Domain.
Evaluation stops at no artificial path segment: only stored topology records
participate. Until those locality records exist, the safe fallback is the
Domain grant plus exact Object/Path grants, not string-prefix inheritance.
An Object with several semantic bindings is readable through its UUID when an
exact Object grant applies or at least one live binding is readable. Responses
then include only bindings separately readable by the caller. A Revision and
its raw/derived projections inherit this Object decision; knowing a Revision
UUID never supplies authority. Broad Domain grants require a live local binding
or recorded creation provenance while the Object has no bindings anywhere.
Other unbound/foreign-only Objects require an exact Object grant, never a broad
Domain or Object-wildcard grant. See [Object authority scope](object-authority-scope.md).
## Route policy inventory
The first implementation slice uses the following policies. A more specific
locality target replaces `domain` as Quartier, Street, and Spot records become
available.
| Surface | Methods | Authentication | Capability and target |
| --- | --- | --- | --- |
| `/health`, static theme/CSS needed by login | GET/HEAD | public | none; health exposes no internals |
| `/_/auth/login`, setup, OIDC callback, magic-link request/verify | declared flow methods | public challenge | rate-limited, challenge-bound |
| `/_/auth/logout` | GET/HEAD confirmation; POST revocation | principal | revoke the supplied session only; browser origin guard |
| `/u/{username}`, `/g/{group}` and semantic Viewer paths | GET/HEAD | optional in legacy, principal in enforcing | `read` resolved path/domain |
| `/e/-/...`, `/:/new`, upload and compile write surfaces | GET plus declared writes | principal | `write` resolved path/locality |
| `/o/uuid:*`, `/o/path:*`, `/r/view/*`, `/r/history/*`, `/r/diff/*` | GET/HEAD | policy-dependent principal | `read` owning object/path; immutable IDs do not bypass policy |
| `/v/raw/*`, previews, archive/email/PDF children | GET/HEAD | policy-dependent principal | `read` source object/revision; cached derivations inherit source policy |
| `/v/search`, tree, browser and children | GET/HEAD | optional/principal | filter every result by `read`; never reveal hidden counts or ancestors |
| `/l/uuid:*` | GET | principal | `read`; POST/DELETE require `link` on object and affected paths |
| object/revision/tag APIs | exact route method | principal | list/read=`read`, create/save/tag=`write`, restore=`write`, purge=`delete` |
| path bind/copy/rename/remove APIs | POST | principal | `link`, `write`, `rename`, or `delete` on both source and destination as applicable |
| GC zombie list/purge | GET/POST | principal | Domain `manage` |
| `/-/sys/config*`, auth administration | GET/POST | principal | Domain `manage`; provider secrets never returned |
| event scheduler and queue APIs | GET or mutation method | principal | GET Domain `manage`; mutation Domain `manage` |
| adapter registry/open-with | GET/POST | principal | read source plus Domain `manage` for registry mutation |
| adapter session exchange/read/write | declared ticket | service | ticket scope, object UUID, base Revision, method, expiry, one use |
| federation enrollment | POST | service | Campus enrollment token, rate limit, controller mode; not Domain RBAC |
| future `/s/*` | GET/HEAD | delegated | macaroon caveats and underlying still-valid delegated grant |
Handlers must declare supported methods; a mutating operation is never exposed
through GET. Search indexes, recent-object lists, children projection caches,
preview caches, archive entries, and adapter workspaces are derived data and
must apply the source object's current authorization on every read.
The server keeps these declarations in a central typed endpoint policy
classifier. Exact API routes are method-closed, unknown mutations below a
reserved namespace fail closed, and semantic Page mutations use a Path target.
Handler-local checks remain authoritative where the target is carried in JSON,
multipart data, or an adapter ticket; those checks must be replaced only by a
typed target resolver with equivalent source and destination coverage.
For immutable addresses, authorization resolves back to mutable authority:
a Revision is authorized through its Object, and an Object is readable through
an exact Object grant or at least one readable live Path binding in the current
Domain. Raw bytes, bounded text, generated PDF pages, cached thumbnails, print,
history, and diff projections all reuse that decision. Their responses use
private caches varying on credentials even when the underlying Revision is
content-addressed. Directory/search projections authorize their requested scope
and filter every returned Path; hidden entries are excluded from displayed
counts.
Recursive namespace mutations expand the complete stored source set before the
transaction. Rename requires `rename` on every source and `create` on every
mapped destination; copy requires `read` and `create` respectively; recursive
delete requires `delete` on every affected Path. Link management additionally
requires `link` authority on the Object and affected Path, and only readable
bindings are returned by its UI.
## Content permission profiles
The group UI exposes profiles rather than ambiguous collections of checkboxes:
| Profile | Effective capabilities | Explicit exclusions |
| --- | --- | --- |
| `read` | `read` | every mutation |
| `read_write_once` | `read`, `create`, `upload` for a previously absent target | overwrite, later Revision save, rename, unlink, delete |
| `read_write` | `read`, `create`, `upload`, `write`, `rename`, `link` | delete |
| `upload_only` | `upload` for a previously absent target | list/read, overwrite, rename, unlink, delete |
`write` implies `create` and `upload` for canonical direct Role grants, while a
group profile stores its exact set above. `create` means create a new semantic
binding/Object and its first Revision. `upload` is checked only after the final
filename and destination path are normalized. Both fail closed if the target
already exists unless `write` is also granted. Deletion is never inferred from
write and requires an explicit `delete` grant outside these four profiles.
`group admin` and `group user` are membership administration roles, not content
profiles. Owner and manager are presented as group-admin variants; member is
presented as group user. Their Wiki access still comes only from a content
permission profile or another Role binding.
## Group model
A Pages group is a Domain-scoped collection of Domain memberships. It is not an
authentication provider group, a Role, a Capability, a Campus object, or a
semantic path. The group UUID is stable while its name and Wiki binding may
change.
```sql
CREATE TABLE groups (
uuid TEXT PRIMARY KEY,
domain_uuid TEXT NOT NULL,
name TEXT NOT NULL,
display_name TEXT NOT NULL DEFAULT '',
description TEXT NOT NULL DEFAULT '',
visibility TEXT NOT NULL DEFAULT 'members', -- public, domain, members
state TEXT NOT NULL DEFAULT 'active', -- active, readonly, archived
created_at_ms INTEGER NOT NULL,
updated_at_ms INTEGER NOT NULL,
UNIQUE(domain_uuid, name)
);
CREATE TABLE group_members (
group_uuid TEXT NOT NULL,
membership_uuid TEXT NOT NULL,
member_role TEXT NOT NULL DEFAULT 'member', -- owner, manager, member
source TEXT NOT NULL DEFAULT 'local', -- local or mapping UUID
created_at_ms INTEGER NOT NULL,
PRIMARY KEY(group_uuid, membership_uuid)
);
CREATE TABLE auth_group_role_bindings (
group_uuid TEXT NOT NULL,
role_uuid TEXT NOT NULL,
created_at_ms INTEGER NOT NULL,
PRIMARY KEY(group_uuid, role_uuid)
);
CREATE TABLE auth_group_mappings (
uuid TEXT PRIMARY KEY,
domain_uuid TEXT NOT NULL,
provider_type TEXT NOT NULL,
provider_uuid TEXT NOT NULL,
external_group TEXT NOT NULL,
group_uuid TEXT NOT NULL,
mode TEXT NOT NULL DEFAULT 'additive', -- additive or authoritative
enabled INTEGER NOT NULL DEFAULT 1,
UNIQUE(domain_uuid, provider_type, provider_uuid, external_group)
);
```
Names use `^[a-z0-9][a-z0-9_-]{1,63}$`; dots and the reserved names `_`, `new`,
and `sys` are rejected. Display names are unrestricted presentation metadata.
Initial implementation permits direct Principal membership only. Nested groups
are deliberately deferred to avoid membership cycles and unclear ownership.
Deleting a group is a soft archive followed by explicit purge. Archive removes
effective group Role bindings immediately but retains audit history and its Wiki
objects. Purge requires Domain `manage`, fails while bindings/mappings remain,
and never deletes Wiki Objects or Revisions implicitly.
### Effective capabilities
The evaluator unions positive grants from:
```text
Domain membership -> direct Role bindings -> grants
Domain membership -> group membership -> group Role bindings -> grants
```
Then it applies Principal, Domain membership, group, and Role lifecycle state.
There are no negative group grants. A Principal who belongs to the same Role
both directly and through groups receives one effective grant. Cache keys include
`domain_uuid`, `principal_uuid`, and a monotonically increasing policy version;
membership, binding, role, grant, or state mutation increments that version.
`owner`, `manager`, and `member` in `group_members.member_role` govern group
administration only:
- owner: metadata, managers/members, Role bindings, external mappings, archive;
- manager: metadata and member list, but not owners, Role bindings, or mappings;
- member: view the group according to visibility and use granted Roles.
Domain `manage` bypasses group-administration roles. Group ownership itself does
not imply Wiki `read` or `write`; those remain capabilities. At least one local
owner must remain while a group is active.
Group `visibility` controls only profile metadata and discoverability. `public`
does not make the group Wiki public, `domain` does not grant Wiki `read`, and
`members` does not make every member a Wiki editor. The normal capability
decision always protects the Wiki subtree.
## `/g/` user interface namespace
`/g/` is reserved before the semantic Viewer catch-all, parallel to `/u/`:
| Route | Purpose | Required authority |
| --- | --- | --- |
| `/g/{groupname}` | Group profile followed by the group Wiki root document | visibility plus Wiki `read` |
| `/g/{groupname}/_/edit` | Metadata, owners/managers/members, Role bindings and mappings | group manager/owner or Domain `manage` |
| `/g/{groupname}/_/avatar` | Mutable non-revision group image | GET by visibility; PUT/POST owner/manager |
| `/g/{groupname}/...` | Ordinary group Wiki semantic subtree | normal path capability |
| `/e/-/g/{groupname}/...` | Canonical editor route for group Wiki documents | normal `write` capability |
The reserved `/_/` segment is intercepted only immediately below the group
name. It never creates semantic children named `_`. `/g/` is excluded from the
normal root `children` projection just as `/u/` is; group discovery is provided
by an explicit group directory UI subject to group visibility.
The profile shell and Wiki document are separate projections. Absence of a Wiki
root document does not make the group absent. Renaming a group changes its
`/g/{name}` binding transactionally, preserves the group UUID and Objects, and
leaves an optional time-bounded redirect alias. It must check destination path
conflicts before mutation.
## Group API
HTML forms call versioned JSON endpoints; handlers may render errors for normal
browser submission but share the same service layer.
| Endpoint | Methods | Operation |
| --- | --- | --- |
| `/_/api/v1/groups` | GET, POST | visible group list; create group |
| `/_/api/v1/groups/{uuid}` | GET, PATCH, DELETE | detail; metadata/state; archive |
| `/_/api/v1/groups/{uuid}/members` | GET, POST | list; add Domain membership |
| `/_/api/v1/groups/{uuid}/members/{membership_uuid}` | PATCH, DELETE | change administrative role; remove |
| `/_/api/v1/groups/{uuid}/roles` | GET, PUT, DELETE | effective Role bindings |
| `/_/api/v1/groups/{uuid}/mappings` | GET, POST, DELETE | external group mappings |
All mutations require CSRF for cookie sessions, use optimistic `updated_at_ms`
or an ETag, and append actor, request ID, old/new values, and outcome to the
audit log. Bulk membership updates are transactional and bounded. List APIs use
cursor pagination and do not disclose invisible groups.
## Implementation boundaries
Route policies live beside route registration in `internal/server/routes` so a
new handler cannot silently inherit a broad default. Startup validates that
every non-catch-all route has policies for every accepted method. Typed target
resolvers live in `internal/server/authorization_targets.go`; they perform only
bounded reads. `internal/store/authorization.go` owns the effective direct and
group grant query, while `internal/store/groups.go` owns transactional group
mutations. HTTP handlers do not issue authorization SQL directly.
The semantic Viewer catch-all has one explicit policy whose resolver uses the
normalized semantic path. Reserved `/u/` and `/g/` handlers are registered
before it and delegate their Wiki descendants back to that same semantic
service after resolving the profile shell.
Minimum audit events are `auth.login.*`, `authz.denied`, `role.binding.*`,
`grant.*`, `group.created`, `group.updated`, `group.member.*`,
`group.role_binding.*`, `group.mapping.*`, and `group.archived`. Details contain
stable UUIDs and changed field names, but never passwords, provider secrets,
TOTP seeds, session/challenge tokens, email magic links, or raw OIDC tokens.
Required integration tests enumerate every registered route/method in the
policy registry and cover anonymous, authenticated-but-ungranted, direct Role,
group Role, readonly, suspended, cross-Domain, cookie-without-CSRF, Bearer,
service-ticket, renamed-path, multiple-binding, and derived-cache access.
## External group synchronization
OIDC/LDAP claims are normalized attributes, then matched only against explicit
`auth_group_mappings`. No mapping means no change. `additive` adds mapped
memberships but never removes local membership. `authoritative` may remove only
memberships previously created by that same mapping UUID; it cannot remove a
local owner or membership from another source. Every synchronization is audited.
OIDC synchronization occurs after successful subject verification and before
session issuance. Existing sessions observe policy-version changes on their next
request. Controller-managed Campus manifests may distribute provider metadata
and mapping definitions, but client/bind/SMTP secrets and live group membership
are not federated unless a future signed policy explicitly says so.
## Built-in administrator groups
Every Domain has reserved `campus_admin` and `domain_admin` groups. They cannot
be created, renamed, archived, or repurposed as ordinary groups. Existing owner
or Domain-manage bindings seed both groups during migration to avoid lockout.
- `campus_admin` controls Campus records and tokens, Domain creation,
placement, update, and deletion for that Campus. It has an owner-controlled
option to project its membership into every Domain belonging to the Campus.
- `domain_admin` controls Domain-local configuration, accounts, sessions,
groups, Roles, and grants. Only a `domain_admin` may create an ordinary
group. It is never inherited merely because two Domains share a Campus.
- ordinary group admins control only their own group and cannot grant
themselves either built-in administrative membership.
Campus projection records each independent origin in `campus_admin_grants`;
`group_members` is its strongest-role projection. Disabling inheritance or
removing a source revokes derived grants transactionally, without promoting
inherited entries into new roots. New projection-owned Domain memberships are
removed, and their sessions revoked, after the last source disappears unless
an independent group/role grant has adopted that membership. Local and other
inherited sources are preserved. Legacy memberships with unknown provenance
remain inert rather than being guessed to be safe to delete. See
[Campus administrator revocation](campus-admin-revocation.md) for the migration
boundary, transaction hooks, regression coverage, and remaining protocol work.
## Cross-Domain identity inheritance
Cross-instance inheritance uses a Domain-pair PSK, not a browser session or a
Campus enrollment token. The source stores only the PSK hash; the receiver
stores the PSK encrypted by its instance authentication key. A peer record pins
source/target Domain UUIDs, source FQDN, instance fingerprint, enabled datasets,
and last successful sequence. TLS remains mandatory except for an explicitly
named development-only HTTP setup path.
The signed snapshot/delta contains Principal UUID plus non-secret profile
metadata, stable external identities, groups, administrative membership role,
and content permission profiles. It never contains password verifiers, TOTP
seeds, recovery codes, sessions, challenges, provider client secrets, SMTP
credentials, avatars unless separately enabled, or audit-log secrets.
Imported rows retain `(source_instance_fingerprint, source_domain_uuid,
source_uuid)` as their stable key. Local UUIDs remain local. Each imported
membership records its peer UUID as source, so disabling a peer removes only
that peer's effective membership. Groups with the same display/name but a
different source identity are not merged implicitly.
The receiver pulls monotonically sequenced deltas and applies each sequence in
one transaction. Replay is idempotent; gaps trigger a full snapshot. A changed
source fingerprint stops synchronization and emits a warning until a Campus
administrator approves the replacement. `campus_admin` may opt into fan-out to
all Domains in its Campus; `domain_admin` and ordinary groups are inherited only
when explicitly selected by the peer policy.
## Migration and implementation slices
1. Replace `requestedCapability` with route policy registration, exact methods,
typed target resolvers, and deny/audit tests. Preserve `legacy` only for
semantic read routes; administration and mutation are never anonymous.
2. Add CSRF for cookie mutations and separate Principal, adapter, federation,
and future delegation credential resolvers.
3. Add group schema/service, `/g/` routing, profile/edit UI, and i18n. Migrate no
existing Role binding destructively; group Role bindings use a new table.
4. Extend effective authorization with group bindings and policy-version cache;
add direct/group union and state-invalidation tests.
5. Add external group mapping and audited OIDC/LDAP group synchronization. LDAP
authentication is implemented; directory-group lookup and explicit mapping
reconciliation remain separate authorization work and must not be inferred
from the successful login attributes.
6. Switch new Domains to enforcing after owner bootstrap; retire the broad
`legacy` bypass only after the endpoint matrix has integration coverage.
Rollback disables group-derived evaluation and route-policy enforcement behind
separate migration flags; additive tables remain readable. It never drops
Principals, sessions, existing direct Role bindings, Objects, or Revisions.
+70
View File
@@ -0,0 +1,70 @@
# Scoped automatic publication
Implemented 2026-09-19. Explicit opt-in, local Domain policy; no existing scope is
enabled by migration or deployment. This is a live-head publication mechanism,
not the reviewer/Inbox workflow in `publication-approval-inbox.md`.
## Scope and controls
- City detail → governance dialog → Automatic publication → preview current impact.
- `inherit`: no local automatic rule; ancestor automatic rules may apply.
- `required`: manual publication only, including descendants. A descendant cannot
override this floor by selecting automatic. Supported at Domain/Quartier/Street/Spot.
- `automatic`: supported only at Street/Spot, with an effective public audience.
Existing eligible Pages, future descendants (including moved-in Pages) and all
current language heads are public immediately. Empty scopes can opt in too.
- Existing public audience ceilings and user/group restrictions remain authoritative.
Aliased Objects, reserved Paths and restricted descendant scopes remain excluded.
- Automatic and inherited activation require confirmation of current impact and
future publication. Preview binds actor/scope/governance/topology and all automatic
rule versions. CAS plus SQLite writer reservation reject stale or concurrent edits.
- Stopping automatic publication does not revoke separately frozen approvals. Use
audience restriction for complete privacy. Existing publication-revoke also sets
this scope to manual-only to stop automatic publication inherited from an ancestor.
## Authority, data and read behavior
`locality_auto_publication` stores Domain, node kind/UUID, mode, pinned Path boundary,
enabling Principal and version. The enabling Principal must remain an active member
with current governance and scoped read authority. Existing owner/admin delegation
is reused; ownership, membership and role revocation suspend the corresponding grant.
Audit records use `city_governance_events`; no silent new grants or source-content edits.
Automatic publication uses a read transaction, not materialized approval entries or
an asynchronous job. This gives read-after-write publication for authorized posts,
CLI writes and edits without a stale scheduler window. Anonymous body/raw reads,
public navigation, projected blog/diary titles, RSS and share readiness use the same
rules. Manual exact-Revision approvals remain the default and remain separate.
The boundary is pinned to the configured Path; renaming the Street/Spot boundary
suspends its grant until explicitly reconfirmed. Moving a Page outside the boundary
does not carry publication with its UUID. Audience restrictions are checked live.
Domain/Campus disablement and authorization-mode changes fail closed. Enforcing
authorization cannot be switched off while an automatic rule remains configured.
Public-navigation generations include automatic rules and their authority sources.
Role/member invalidation is conditional on the existence of automatic rules, keeping
the existing invariant that ordinary authenticated navigation never caches ACLs.
## API and budgets
`GET/PATCH /_/api/v1/city/{zones|spots|domains}/{uuid}/governance/auto-publication`
uses the existing authenticated handler-owned governance authorization. PATCH fields:
`mode`, `version`, `epoch`, `topology_version`, `confirm`, `impact_token`.
GET never grants publication. Strict JSON and 4KiB input limits apply.
Impact previews retain the existing 512-Path limit; a too-large preview still permits
manual-only shutdown, but not activation. Live navigation is metadata-only and bounded
by 20,000 entries / 4MiB with a 2-second caller deadline. Exact body lookup restricts
the query to the requested Path. Policy enumeration is capped at 128 local rows.
Errors/budget exhaustion do not fall back to public/unfiltered content.
## Remaining work
- Independent reviewer policies, requests, quorum, Inbox and signed Campus inheritance.
A future `approval=required` floor must also block this live-head publication mode.
- Federation of these policies is not implemented; no automatic-policy fields are
added to Domain provisioning manifests.
- Larger activation previews need paginated impact review/token binding rather than
silently lifting the current limit. Policy tombstone/archived-node cleanup is pending.
- External caches, feed-reader copies and already-posted social links cannot be recalled.
+91
View File
@@ -0,0 +1,91 @@
# Browser request and session boundaries
Implemented 2026-09-14. All requests pass the browser mutation guard before
authorization's legacy-mode or handler-owned route branches. Same-origin forms
and fetch calls continue working without page-specific hidden-token injection.
## CSRF
The guard uses Go's `http.CrossOriginProtection` (available since Go 1.25) and
has no trusted cross-origin list or route-prefix exceptions. Fetch Metadata and
Origin distinguish same-origin requests from unrelated or sibling origins;
`Origin: null` is rejected. Host resolution never trusts `X-Forwarded-Host`.
For legacy clients without same-origin Fetch Metadata, Origin must also use the
request's scheme; an HTTP origin cannot target an HTTPS browser session merely
because the host matches. TLS-terminating proxies must preserve forwarded scheme.
See the [Go 1.25 HTTP release notes](https://go.dev/doc/go1.25#net/http).
Unsafe requests carrying an ambient session/MFA cookie must additionally have
validated origin metadata or a matching HTTP(S) Referer. Missing browser-origin
evidence fails closed. Headerless server-to-server clients remain supported;
API clients should use explicit Bearer authentication. Cross-origin login,
profile/group/configuration/upload and API mutations are covered, not just
document save routes. API denials return JSON with status 403.
Logout is POST-only for revocation; GET/HEAD render confirmation without
changing session state. Profile menus submit a native same-origin POST form.
Bearer logout revokes exactly the supplied token, not an unrelated cookie.
Legacy semantic editor GET entry can still allocate an empty Page. Its entry
routes are therefore checked as mutations too; direct address-bar/bookmark
navigation with browser `Sec-Fetch-Site: none` remains allowed. Moving creation
fully into the save transaction remains a separate editor API correction.
OIDC/magic-link callbacks are one-use, purpose-bound login flows, not generic
state-changing GET endpoints. OIDC additionally has browser-bound state.
## Credential purpose
New API tokens begin with `pw_api_`; new cookie sessions begin with
`pw_browser_`. The complete prefix plus random secret is SHA-256 hashed in the
session store. Changing/removing a prefix cannot convert its purpose.
Browser tokens are rejected in Authorization; API tokens are rejected in the
session cookie. Adapter/federation/challenge tokens use independent validation
and tables and are not accepted as login sessions.
Unprefixed pre-upgrade sessions retain compatibility until their recorded
expiry (normally 24 hours). No live session is forcibly migrated or broadened.
Once the old rows expire, new sessions enforce transport separation throughout.
Store helpers without HTTP context remain trusted internal APIs.
Session insertion itself requires an active/readonly Principal and active Domain
membership in the same SQL statement. Each session lookup rechecks membership;
disabling/removing it invalidates the next request even if a session row remains.
Dynamic responses default to `private, no-store` with credential variation;
individual immutable projections can opt into a specific private cache policy.
An explicitly supplied invalid/unknown/ambiguous Authorization header never
falls back to a cookie. Duplicate session cookies are rejected. Login body
size and local return URLs are validated; slash/backslash/control-character
redirect variants cannot redirect to another authority.
## Request-scoped read authorization
Domain lookup is cached only in one request. A bounded single-query snapshot
loads at most 4096 direct-role/group Path read grants, honoring principal and
membership states, exact direct grants and literal group subtree boundaries.
It is immutable, contains no list of all document Paths and is discarded at
request end. Errors/oversized policies fail closed. Tree/children/search/City
read filtering reuses this snapshot; mutation authorization does not.
The next request rechecks sessions, membership and grants. There is no TTL
allow-cache that outlives revocation. A change during an already-rendering
request takes effect on the next request; transactional mutation authorization
and streaming revocation have separate contracts.
## City projection
Map/detail/Spot Page APIs, City macros and Viewer locality badges filter by
readable locality boundaries and Page Paths. Counts and portal choices are
recomputed from visible Pages; nearby relations require both visible ends.
No Domain-only topology ETag is reused across callers (`private, no-store`).
Parents without readable boundaries are not exposed just because a descendant
or MagicPage happens to be readable. Unbound/hidden aliases are not substituted
with UUID links to bypass Path visibility. The complete topology is still loaded
before projection; lazy server-side City pagination remains outstanding.
Regression tests cover cross-origin/missing-metadata denials, login/logout,
purpose confusion, exact/wildcard grant semantics, principal states, next-request
revocation, hidden City metadata/counts/portals/relations and macro projection.
An opt-in Chromium regression additionally submits the real login and logout
forms using HttpOnly cookies and actual Fetch Metadata, with no injected Bearer
credential, and verifies logout leaves a separate API session intact.
+99
View File
@@ -0,0 +1,99 @@
# Campus administrator inheritance: revocation and provenance
Implemented: 2026-09-14. This note supersedes the former insert-only Campus
projection described in `authorization-groups.md`. Cross-instance Domain-pair
PSK synchronization is still a separate, unimplemented protocol.
## Authority and data ownership
`campus_admin_grants` is the authoritative set of independent and inherited
grants for the built-in `campus_admin` group only. Its key is
`(group_uuid, membership_uuid, source)`. An independent grant uses `local`,
`migration`, or an existing non-Campus source identifier. Derived grants use
`campus_admin_inherit:<origin group UUID>`.
`group_members` remains the public/effective membership table. The strongest
remaining role wins (`owner > manager > member`), with a direct source winning
ties. A weaker local grant is kept in provenance, not overwritten by a stronger
inherited role, and becomes effective again when that inherited role expires.
Ordinary groups and `domain_admin` retain their existing storage and behavior.
Projection roots must have an active source group, active principal, active
source Domain membership, enabled source Domain, and enabled inheritance flag.
The target must be an enabled Domain in the same Campus with an active built-in
group. Only independent grants are roots. Derived rows cannot retransmit
authority, become new roots, or keep one another alive in a cycle. Multiple
independent origins are recorded separately; removing one does not remove the
others. Changing a principal to readonly also withdraws inherited administrator
membership; changing it back to active can restore the projection but does not
revive revoked sessions.
## Transaction boundary
The following write paths reconcile within the same SQLite transaction:
- enabling/disabling Campus inheritance;
- adding, changing, or removing an independent Campus group member;
- changing a principal's state and revoking its sessions;
- updating/deleting a Domain, including moves between Campuses;
- applying a managed-Campus topology snapshot or deleting a managed Campus;
- built-in group creation/bootstrap migration.
The projection rebuilds the effective Campus group membership from authoritative
grants. Any SQL failure rolls back both the source mutation and the projection.
No asynchronous queue or stale authorization cache sits between revocation and
the next authorization decision. Direct SQL administration must explicitly run
`ProjectCampusAdminMemberships`; deleting only the materialized `group_members`
row is not a revocation API. Future LDAP/OIDC mappings and PSK importers must
write provenance and reconcile in their own transaction, not edit the cache.
Inherited-only members cannot be removed at a destination: revoke their origin
or disable its inheritance. Removing an independent owner requires another
independent owner at that source; an owner inherited from somewhere else is
not sufficient. Bootstrap migration records each migrated source binding once
in `campus_admin_migrations`, so a later explicit Campus-group removal is not
undone by a restart or the creation of another Domain.
## Domain membership and session cleanup
`campus_admin_membership_origins` records exactly which Domain memberships the
projector created. After the last derived grant disappears, a provenance-owned
membership with no other group membership or role binding is removed and that
principal's sessions in that Domain are revoked atomically.
Existing local memberships are never reactivated, relabeled as inherited, or
deleted by projection. Adding an independent group grant adopts an inherited
membership as local. An independent role binding likewise prevents cleanup and
preserves its session. No role binding or capability grant is deleted: the old
Campus projection never created `auth_role_bindings`, and revocation of a group
membership already removes access through that group's permission profiles.
Independent `domain_admin` and direct role capabilities remain unchanged.
## Upgrade behavior and limits
Older installations recorded only one source per effective group member and
did not record the origin of a Domain membership. On the first schema upgrade,
only stored non-inherited Campus group memberships seed authoritative grants;
old inherited chains are discarded and regenerated from those roots. A legacy
Domain membership with ambiguous origin is retained, while its inherited group
authority is revoked normally. Do not guess that an untagged membership belongs
to the projector and delete a potentially independent local identity record.
An old local grant previously overwritten or mislabeled by the single-source
implementation cannot be reconstructed; an administrator must explicitly grant
that local role again if needed. Newly recorded grants do not have this loss.
Reconciliation is set-based and limited to Campus administrator memberships;
it runs on administrative changes, not on ordinary reads or session lookups.
It currently recomputes all Campus administrator projections in one transaction.
Incremental Campus-scoped reconciliation and sequenced cross-instance deltas are
future scaling work; do not advertise distributed revocation guarantees before
the PSK protocol and its source-disappearance policy exist.
## Regression coverage
`groups_inheritance_test.go` covers disable/re-enable, repeated reconciliation,
source removal, suspension/quarantine/readonly, removed/disabled/moved source
Domains, multiple roots, prevention of transitive amplification, weaker local
role restoration, independent role/session preservation, legacy-chain cleanup,
restart-safe bootstrap removal, and injected mid-transaction failures for
disable/remove/suspend. Existing concurrent owner-integrity tests remain green.
+99
View File
@@ -0,0 +1,99 @@
# Campus and Domain Model
## Purpose
Campus is the top-level administrative and language-default boundary.
Domain is the host-selected subdomain space presented to a request. Every
Domain belongs to exactly one Campus.
This slice intentionally stops before ZoneTree. Quartier, Street, Spot, and
SpotGraph remain later topology work.
## Identity
Both Campus and Domain have:
- an integer SQLite row ID for local administration;
- a stable UUID for relationships;
- a normalized, unique machine name;
- a human-readable title;
- an enabled flag;
- a creation timestamp.
Campus additionally has a BCP 47 default language, authority mode, and FQDN.
Domain additionally has a host selector. A selector can be a full host
(`docs.example.test`), a subdomain label (`docs`), or empty to select the
Campus FQDN itself.
## Ownership invariants
- Every Domain `campus_uuid` resolves to an existing Campus.
- At least one Campus exists and at least one Campus remains enabled.
- A Campus referenced by a Domain cannot be deleted.
- A Campus referenced by a Domain cannot be disabled.
- Creating or moving a Domain requires an enabled target Campus.
- Disabled Domains do not participate in request-host selection.
- Request-host resolution is exact and has no last-resort Domain fallback.
- An empty Domain host matches only its owning Campus FQDN.
- The request Campus is derived from the resolved Domain; it is not selected
independently.
SQLite does not currently rebuild legacy tables to add a foreign-key
constraint. These invariants are enforced by store operations so existing
installations can migrate without destructive table replacement.
## Schema
```sql
CREATE TABLE campuses (
id INTEGER PRIMARY KEY AUTOINCREMENT,
uuid TEXT NOT NULL UNIQUE,
name TEXT NOT NULL UNIQUE,
title TEXT NOT NULL,
default_lang TEXT NOT NULL DEFAULT 'en',
authority_mode TEXT NOT NULL DEFAULT 'standalone',
fqdn TEXT NOT NULL,
enabled INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL
);
-- Added to domains during initialization:
ALTER TABLE domains ADD COLUMN campus_uuid TEXT;
CREATE INDEX idx_domains_campus_uuid ON domains(campus_uuid);
```
## Startup migration
Initialization is idempotent:
1. The existing District-to-Domain compatibility migration runs.
2. If no Campus exists, `Default Campus` is created.
3. The nullable `domains.campus_uuid` column is added when absent.
4. Empty, unknown, and legacy Campus references are assigned to the enabled
default Campus.
UUIDs, Domain host selectors, and existing paths are preserved.
## Request context
The server resolves one Domain from the HTTP Host and then resolves its
owning Campus. Both values are exposed as `RequestContext.Domain` and
`RequestContext.Campus`.
Revision language selection uses this order:
1. explicit language request;
2. request language preferences;
3. Campus default language;
4. system locale;
5. language-neutral revision;
6. latest revision.
## Administration
The Config view supports Campus create, update, and delete, as well as Domain
assignment to a Campus. The last enabled Campus cannot be disabled. A Campus
must first have its Domains moved or deleted before it can be deleted.
The data CLI remains object/path/revision/asset oriented. Campus and Domain
administration is therefore not added to the CLI in this slice.
+93
View File
@@ -0,0 +1,93 @@
# City courtyard projection and immersive navigation
Implemented UI slice, 2026-09-14. Complements
[City organization](city-organization-ui.md) and the larger
[City review](city-review-and-reconciliation.md); not a replacement for its
authorization, reconciliation or Worker hardening work.
## Interaction
The browsing sequence is `Domain → Quartier → Street → Spot → Page`.
Clicking a Spot now explores its courtyard **inside the map**, rather than
immediately navigating to its portal document. The central building represents
the Spot; surrounding smaller buildings are its Page placements. The focused
Spot offers a separate portal Page link. An actual boundary/portal Page is not
repeated as its own surrounding Page building. A descendant suggested by the
API as a fallback portal remains independently visible/selectable. Undefined
placements use a fountain and retain
yellow-green Page markers.
Page buildings link to their semantic Path, falling back to `/o/uuid:{uuid}`.
These are rendering changes only: no document Paths, memberships, UUIDs,
revisions or contents are changed. The displayed set follows existing SpotGraph
membership, including explicit links; it is not a new filesystem hierarchy.
Breadcrumbs and clicking outside the courtyard ascend within the map. Map
exploration does not call pushState or add browsing history. Actual Page links
still navigate normally (from a map popup, in the containing window).
Ctrl/Command selection and the existing merge/rename/move dialogs work for
whole Spots and individual Page buildings, without following their links.
## Layout and visual vocabulary
- Domain/Quartier retain the single generated city fabric and its allocations.
- Street close-ups use a deterministic courtyard layout, with generated road
segments projected behind it and paths connecting courtyard gates to roads.
This is a readable close-up schematic, not geometric continuity at every zoom
level. Connected road allocation and natural parcel packing remain future work.
- Nonoverlapping plots reserve space for the Spot, six nearby Pages and gates.
At most six Spots are rendered per Street screen; further Spots are paginated.
- A focused Spot renders at most 24 surrounding Pages per screen, also paginated.
Page UUID deduplication prevents multiple graph edges creating duplicate houses.
- Labels remain horizontal and visible at rest, with bounded two-line wrapping
and full-name/Path tooltips. The existing label toggle controls the new labels.
- Native SVG motifs include stucco/terracotta houses, a pediment and colonnade,
stone paving, balustrades, cypresses, fountains and ornamental iron gates.
Theme-derived stone/brass colors, the academic crest and compass carry the
same visual language into the map frame and menu. No external images/fonts
or raster-generation service are needed for these additions.
The new `city-estates.js` module owns only close-up layout and rendering.
`city-map.js` retains topology, navigation, generated fabric and API interaction;
`city-organizer.js` retains selection and mutations. The obsolete geometry
script and mutation-observer Spot repositioning no longer run on the map.
Pointer dragging and wheel zoom use SVG coordinates with a screen-pixel drag
threshold, including differently sized close-up viewBoxes.
## Utility menu
The initial normal and popup screens show the map, breadcrumb, menu launcher,
small navigation hint, compass and (only when needed) pagination/selection bar.
The permanent inspector, directory, planner and toolbar are moved into a themed
native modal (`city-menu.js`):
- Guide: selected locality, details link and existing editing operations.
- Directory: named navigation to Quartier/Street/Spot views.
- Planning, for authorized planners only: URL planning and one creation form at
a time, selected by kind.
- Settings: zoom/reset/labels, topology status, scoped search and account links.
The dialog uses native modal focus containment and Escape dismissal. Its Escape
handler does not propagate to the containing map popup. There is no permanent
outer-page scrolling; menu content can scroll independently. Mobile CSS reduces
the frame and changes the menu navigation to two columns.
## Verification and limits
Opt-in Chromium fixtures cover normal/popup modes: resting labels, Ctrl
selection, title/group/merge/move regression, plot nonoverlap, six-Spot and
24-Page bounds, full pagination reachability, correct Page/portal links,
unchanged map URL/history, menu panel switching, creation-form visibility,
directory navigation and background ascent. Screenshots cover sparse Street,
dense Spot and menu scenes, including a 390px viewport with Index White. Run:
```sh
PAGESWIKI_BROWSER_TEST=1 go test ./internal/server -run 'TestCity(Labels|Organizer|Immersive)Chromium' -count=1
```
These are DOM-rendering bounds, **not** server/API pagination. The map still
loads topology and Page placements eagerly; the directory and some inspector
option lists still traverse that loaded topology. Per-result authorization,
bounded/lazy server projection, connected roads, Worker generation, keyboard
directional navigation and auto-reconciliation remain open in CITY-A–D.
This slice does not claim to solve those scaling/security gaps.
+81
View File
@@ -0,0 +1,81 @@
# City organization UI
Implemented follow-up, 2026-09-13. This is a bounded manual editing slice of
[the City review](city-review-and-reconciliation.md), not the automatic outbox
or Worker pipeline.
## User model
Hierarchy remains `Quartier → Street → Spot → Page`; a Spot cannot own a Spot.
Yellow-green points are unclassified Pages, not nested Spot objects.
- Context menu or inspector pencil: edit a locality's **display title**. UUID,
slug/name and Path remain unchanged. Japanese titles are accepted.
- Ctrl/Command-click toggles selection of Page buildings/points or whole Spots.
Selection persists while navigating between Streets. As of 2026-09-14, ordinary
Spot clicks explore a courtyard; Page buildings and the separate portal link
navigate to documents. See [immersive map projection](city-immersive-map.md).
- The floating icon toolbar/context menu can combine the selection into an
existing Spot, or create one Spot containing the selected Pages.
- Whole-Spot merge archives the sources, transfers Page placements and
incoming/outgoing relations, deduplicates relations and drops self-relations.
Source descriptions/charters remain on archived rows; they are not concatenated
into the destination. Existing destination labels take precedence over duplicates.
- Move relocates whole Spots to a Street, retaining UUIDs. Path boundaries
change atomically with Street placement. Single moves allow a custom boundary;
bulk moves use the old boundary's final segment under the new Street.
Conflicts or invalid ancestry reject the entire operation.
These operations **preserve document URLs, Page UUIDs, Revisions and content**.
Moving a Spot captures its existing inferred members as explicit memberships;
the moved boundary may infer new Pages too. This is a City/SpotGraph operation,
not recursive filesystem/URL rename. The modal explains the distinction before
submission. Changing document URLs remains a separate Path operation.
## Backend contract
- `POST /_/api/v1/city/{zones|spots}/{uuid}/title`: title and mandatory
`base_version`; zones additionally supply `kind=quartier|street`.
- `POST /_/api/v1/city/organize`: mandatory `base_version`, `action=merge|move`,
whole-Spot UUIDs and/or `{spot_uuid,page_uuid}` selections. Merge chooses
exactly one of `target_spot_uuid` and a `new_spot` definition. Move accepts
whole Spots and a destination Street only.
- Both routes require existing Domain administrator/manage authorization.
Targets, sources, parents and Page membership are revalidated inside the
transaction against the current Domain. Stale topology returns 409.
- Selection is limited to 200 entries. Expansion is capped at 2,000 Page-edge
placements and 2,000 affected relations; larger operations fail without
partial application. This is not a background bulk-import API.
- Transfer, archive/move, overrides and one topology event/version increment
commit together. Title no-ops do not increment versions. Validation/DB failures
roll back everything; retries using an old version cannot reapply.
- Audit events store requests, source placements and previous Path/Street
boundaries. No automatic undo UI or exact reverse-operation endpoint exists
yet; this is not a complete reversible merge implementation.
`city_page_overrides(domain_uuid,page_uuid,spot_uuid)` records manual placement.
Active overrides prevent autoplan reconstructing old assignments and suppress
inferred membership in other Spots. Other explicit MagicPage edges remain valid.
An archived destination no longer suppresses inference; overrides are carried
forward when its whole Spot is merged again. This is not the future full
generation-provenance/outbox design or a general exclusion-management UI.
Normal and popup views share a themed native modal and selection toolbar;
Ctrl/Command-click in a popup must not trigger top-window navigation.
The generator seed no longer includes topology version, and its in-page fabric
is reused on reload so renaming does not randomly rebuild the town. Allocation
can still change with membership weights; Worker generation and persistent
geometry caching remain open.
## Verification and remaining work
Store/API tests cover Unicode titles, identity/URL preservation, stale requests,
admin denial, invalid/cross-Domain selections, relation transfer/deduplication,
inference preservation, and rollback after injected mid-write failure.
Chromium tests exercise Ctrl selection/deselection, grouping undefined Pages,
rename, whole-Spot merge, relocation and autoplan overrides in normal/popup modes.
Remaining: editable slug aliases and redirects for merged Spot detail URLs;
per-Page override management/undo; URL-renaming with explicit preview;
bounded automatic reconciliation; projection authorization/read scaling;
Worker/geometry capacity fixes and directional keyboard navigation.
+91
View File
@@ -0,0 +1,91 @@
# Bounded City read snapshots and literal Path boundaries
Implemented 2026-09-14. This is a bounded repair of CITY-04/CITY-06, not completion
of the [City review](city-review-and-reconciliation.md) or its lazy projection plan.
## Read contract
`GetCityMapContext` uses one read-only SQLite transaction with a two-second
deadline, shortened by request cancellation. The topology version, Quartiers,
Streets, Spots, Page placements and relations all come from that same snapshot.
Each cursor is fully scanned, checked and closed before the next query; a
one-connection pool therefore works without nested-connection deadlock.
Quartiers, Streets, Spots and relations use four bulk cursors. Page placements
still use one bounded query per Spot, on the **same transaction/connection**.
Parent joins require current-Domain active Quartiers/Streets/Spots. Foreign or
archived parents cannot introduce children into the model. Explicit Page edges
require a current-Domain Path binding, including edges left behind after moving
the last binding to another Domain. Relations require two visible active Spots.
The projection fails with `ErrCityMapBudget` rather than silently truncating:
| Budget | Limit per complete map |
| --- | ---: |
| Quartier + Street + Spot nodes | 4,096 |
| Page placements, including multiple semantic appearances | 20,000 |
| Relations examined | 8,192 |
| Database snapshot / connection wait | 2 seconds |
Errors return an empty model, not partial topology with apparently successful
counts. Spot counts deduplicate Page UUIDs while preserving distinct typed edges.
Exact boundary Pages remain preferred portals. Query/scan/version errors are
propagated, and all error/cancellation paths release cursors and connections.
`ListSpotPagesContext` applies the same two-second read transaction and 20,000
placement safety limit to an HTTP Spot projection. Trusted `ListSpotPages`
retains its complete-set contract. This is a safety ceiling, **not pagination**.
No Page bodies or revision histories are read for these projections.
HTTP map/detail/Page projections return `503` with `Retry-After: 5` when a
safety ceiling or request deadline is reached, rather than misreporting a
large locality as missing. Missing or unreadable localities remain `404`;
unexpected storage failures remain `500`.
The server's separate request-scoped read-policy filter remains mandatory: a
consistent Domain snapshot is not authorization to see every locality or Page.
No cross-principal model cache or topology-only public ETag is introduced.
## Literal boundary contract
All SQL containment/overlap matching in `city.go` and `city_operations.go` uses
case-sensitive binary segment ranges, not `LIKE`. A normalized non-root boundary
`/a%_` contains its exact Path and the range `[/a%_/, /a%_0)`; `%`, `_`, backslash
and Unicode are ordinary characters. `/A` does not contain `/a`, and `/life`
does not contain `/life2`. An empty legacy boundary contains nothing.
Root `/` covers every absolute Path. Creating it therefore conflicts with any
other active Quartier boundary, regardless of creation order. Streets and Spots
still require strict descendant Paths and their existing valid parent/Domain.
No raw Path or UUID is changed, decoded again, lowercased or escaped in storage.
Creation validation, import overlap lookup, Path-context resolution, inferred
Page membership, appearance lookup and health checks use these semantics.
Context and health use bounded read snapshots too. The `/u` health exclusion
includes the namespace root and descendants, not unrelated or differently-cased
Paths. Health is still a basic topology summary, not a full reconciliation of
manual membership overrides and all curation semantics.
## Verification and remaining work
`city_read_test.go` covers a one-connection pool, WAL concurrent commits between
snapshot pinning and topology loading, all three size ceilings, connection-wait
cancellation, nested scan errors, connection release, foreign/archived parent
edges, foreign explicit Pages, literal wildcard/case/Unicode boundaries, root
overlap, and health/context/overlap consistency. Existing City/import/organizer
tests remain regression gates.
2026-09-14 named references: `/v/city/s/{quartier}/{street}` resolves in one
bounded query with its current active parent. Legacy unique Street names and
explicit `uuid:` references still work; ambiguous legacy references fail closed
with the same 404 as missing/hidden nodes. Details check ancestor read access,
do not load the map, and propagate cancellation. Breadcrumb macros defer map
loading until `nearby` actually needs it. Literal Page links encode `?/#/%`.
Regressions: `city_reference_test.go`, `city_links_test.go`, and Chromium
`city-links.js` (normal, popup and all three initial-focus levels).
Still pending: lazy paginated locality/Page APIs; removal of per-Spot Page
queries; size limits for individual legacy metadata fields; context propagation
to remaining health/macro callers; full URL-autoplan provenance/atomic adoption. Large City data
can currently exceed a safety ceiling and show an error until lazy projections
are implemented. Raising global limits is not a replacement for that work.
@@ -0,0 +1,261 @@
# City module review and URL-driven reconciliation
Reviewed: 2026-09-13; implementation follow-up 2026-09-14.
Status: **incremental repairs landed; automatic reconciliation remains design-only**.
Scope: City stores/APIs, URL autoplan, SVG UI, the bundled gen-city 1.4.6
adapter, and scheduler integration. This supplements
[Knowledge City Model](knowledge-city-model.md) and the
[platform outbox contract](platform-extension-architecture.md#event-and-outbox-contract).
Phase 2 checkboxes represent first slices, not completion of these repairs.
## Landed follow-ups
- [x] CITY-05 named-route slice: parent-qualified Streets, bounded cancellable
detail lookup with active same-Domain ancestors and ambiguity detection;
shared links across search, map, inspector, context menu and macros.
Detail views no longer load the whole City; map focus and locality search
support all three levels. Spot autoplan name collision handling remains open.
- [x] Per-result map/detail/Page/macro/badge authorization and private responses;
visible portal/count/relation recomputation.
- [x] One-connection-safe bounded City/Spot read transactions and binary literal
Path containment/overlap/context/health matching; see
[read integrity and remaining limits](city-read-integrity.md).
- [x] Stable Domain fabric seed, local Worker generation, bounded geometry cache,
deadline/cancellation and directional Arrow/WASD/HJKL navigation; see
[Worker contract](city-worker-navigation.md).
- [x] Horizontal/resting labels, manual organization, nonoverlapping bounded
Spot courtyards with Page buildings and a modal utility menu; see
[immersive projection](city-immersive-map.md).
Lazy Page APIs, connected road allocation, deterministic generated Spot names and the
outbox/provenance/idempotent inference pipeline are not implied by these checks.
## Original findings, in repair order
| ID / priority | Evidence at audit | Correction |
| --- | --- | --- |
| CITY-01 / P0 | `city_handler.go` returns `GetCityMap` and `ListSpotPages` without per-member authorization. Middleware gates the endpoint, not each returned Page/locality. | Batch-authorize nodes, Page links, portals, relations and counts. Audit details/macros too; Domain read must not expose denied descendants. |
| CITY-02 / P1 | `buildCityAutoPlan` ignores Paths shorter than three segments despite its UI's shallow-Page description. `AddSpotPage` increments topology version even on identical repeated UPSERTs. | Explicit root/portal/leaf rules and no-op reconciliation for unchanged inputs. Separate inferred membership from manual edges. |
| CITY-03 / P1 | POST `/city/autoplan` synchronously scans all Paths and applies separate node/edge transactions; the first error leaves a partially applied plan. No generation ownership/checkpoint exists. | Preview/adoption, bounded resumable batches, source provenance and itemized conflicts before automatic jobs. |
| CITY-04 / P1 | `CityNodeOverlappingPath` merges exact/ancestor/descendant candidates with `LIMIT 1`; it and `ListSpotPages` use unescaped LIKE patterns. Autoplan treats lookup errors as absence. | Literal normalized Path segments, compatible parent/identity matching, explicit ambiguity and DB errors. Test `%`, `_`, case and non-ASCII Paths. |
| CITY-05 / P1 | Streets allow repeated names under different parents but `CityNode`/detail URLs resolve by Domain/name. Spots require Domain-unique names, while autoplan derives only a URL segment. | Server-provided unambiguous hrefs, parent-qualified readable Street routes, existing UUID fallback, deterministic Spot name collision handling without altering display titles. |
| CITY-06 / P1 | `GetCityMap` nests open cursors and queries per Quartier/Street/Spot, returning every Spot's entire Page list. Autoplan also queries while the Paths cursor is open. No consistent read snapshot. | Bounded bulk queries, snapshots, closed cursors, cancellation and lazy locality/Page APIs. Test with a one-connection pool. |
| CITY-07 / P1 | `render()` seeds gen-city with `domain_uuid:topology_version` on every reload. Cached `city-geometry.js` output is assigned to `geometry` but never used by this renderer. | Stable base-fabric seed and one real geometry cache; remove obsolete generation after migration/tests. |
| CITY-08 / P1 | gen-city's `async generate()` runs synchronous matrix initialization, recursive path generation and building generation. Layout/SVG construction also runs on the main thread. | Worker generation/layout with cancellation and budgets; separately bound DOM work. `await` alone is not CPU isolation. |
| CITY-09 / P1 | `cityLayout` sorts/slices roads by coordinates, not connectivity; early Streets can consume all roads. Spots without buildings are omitted. Proximity uses infinite-line rather than segment distance. | Connected/fair allocation, finite segment projection and fallback picking targets. Geometry scarcity must never hide a logical node. |
| CITY-10 / P1 | Street weights sum Spot weights (duplicate Page counting), unlike Quartier weights. Allocation repeats on navigation; a MutationObserver repairs Spot positions afterward. | Deduplicate at every level, cache allocation, place directly, manage label collisions and remove post-render repair. |
| CITY-11 / P2 | Three embedded creation panels plus inspector move/binding/relation forms; many dynamic English literals. | Contextual icon actions, shared modal forms, localized errors and progressive disclosure. |
| CITY-12 / P2 | Only Enter/Space node keys; no map navigation stack. `showSpot` can append a late response to a different selection. Deleted current nodes can leave an invalid view. | Directional focus, explicit view state, stale-response guards, valid-parent fallback and separate in-map history. |
CITY-01 describes a confirmed missing check in the inspected call chain, not a
production exploit test. CITY-06 is a code-level scaling/pool risk, not a
measurement of a production DB lock. Additional correctness requirements:
- Path/Revision/policy changes can change map output without changing its
topology-only ETag. Model and authorization generations belong in validators.
- Explicit edges resolve a Page's primary Path, which may be outside the
inferred locality or unreadable. Preserve locality-specific safe portal links.
- Undefined classification currently guesses from name/path substrings. Add an
explicit kind instead of interpreting a user's spelling as system metadata.
- Deleting a generated node requires an exclusion/tombstone; otherwise the next
sweep recreates it. Offer explicit re-enablement.
- Recursive rename already updates City prefixes transactionally. Reuse this
identity-preserving behavior rather than recreating locality UUIDs.
## Component boundaries
1. **URL topology reconciler**: canonical Path bindings → derived memberships
and generated node proposals, running in the existing event scheduler.
2. **Authorized City projection**: topology + metadata → a caller-specific,
bounded navigation model. Geometry never grants access.
3. **Geometry adapter**: stable fabric + permitted model → blocks, connected
Streets, anchors, hit targets and labels. gen-city does not own identity.
Retain gen-city for the first repair and evaluate continuity/capacity/performance
fixtures before replacing it or maintaining a fork. Preserve its license and
rebuild the committed bundle from `assets/city-generator`, not vendor edits.
## URL inference and preservation
- Infer from canonical Domain-scoped Paths, never display slugs, host strings
or repeated URL decoding. Use shared reserved-address rules to exclude user,
group and system routes instead of an incomplete local exclusion list.
- First directory → Quartier; second → Street; third → Spot; deeper descendants
→ that Spot. A directory has descendants and may be virtual (no own Page).
- A locality-boundary Page is its portal, not a child of itself. For example,
`/life/home/kitchen` with descendants is the Kitchen Spot portal;
`/life/home/readme` without descendants is an undefined Page on Home Street.
- A leaf gaining descendants may gain a locality without changing Page UUID.
Change only derived membership, not curated semantic edges.
- `/`, `/life`, `/life/home` remain Domain/Quartier/Street portal Pages. Show
shallow/unassigned Pages at their nearest valid level; do not manufacture
strict-descendant boundaries to force them into a Spot.
- Undefined Pages stay individual yellow-green points. An internal bucket must
not become a fabricated public `/_undefined` portal.
- Store source/rule version and manual overrides separately from descriptions.
Existing nodes and `source=local` edges start as manual/unknown: never adopt
automatically by matching “Generated from URL Paths” text.
- Preview adoption. Reuse exact compatible boundaries; ambiguous ancestors or
descendants need explicit mapping. Keep exclusions, UUIDs and Page content.
- Aliases may infer multiple memberships, but count each Page UUID once in each
aggregate; select readable portal Paths within the locality.
## Event/queue design
Extend the shared transactional outbox; do not install expensive callbacks in
individual HTTP handlers or build a second scheduler. Cover WebUI, CLI, import,
hosted editors, restore, link/unlink, rename and deletion through store writes.
| Committed change | City invalidation |
| --- | --- |
| Path bind/create/unlink/delete; Object deletion | Domain/prefix membership and metadata. |
| Rename/move, including recursive operations | Old and new prefixes; retain locality UUIDs. |
| Revision/edit/restore; filename, MIME or title change | Metadata in Domains binding this Page; unchanged Paths do not regenerate fabric. |
| Topology/manual rules | Affected mapping/layout. |
| Permission/group inheritance change | Authorized projections, not canonical geometry seed. |
| Periodic or manual repair | Same bounded reconciler/checkpoint. |
Proposed persistent state, **not yet migrated**:
- Shared `activity_outbox`: instance/event UUID, Domain sequence, event version,
actor/source, typed Page/Revision target and old/new Path data. Atomic with the
mutation; no document bodies or credentials.
- `city_reconcile_state`: Domain, desired/applied sequence, rule version,
debounce deadline, continuation cursor and last error/job UUID.
- Generated node/membership provenance: stable source key, rule version and
manual override/exclusion; independent of explicit SpotGraph edges.
- Completed model generation: publish finished snapshots, not half-built
reconciliation as a completed map.
Delivery protocol:
1. Commit change and outbox event together; rollback emits no work. SQLite
invalidation triggers may cover legacy writers, but cannot replace actor-aware
events. Unsupported raw SQL edits rely on periodic repair.
2. Consume bounded batches, advance desired sequence and coalesce import/rename
bursts by prefix/Domain. Do not create a full-scan job for every file.
3. Enqueue `city.reconcile`, keyed by Domain. Initial proposed debounce: 2 seconds,
maximum coalescing delay: 10 seconds, configurable to avoid starvation under
continuous editing. These are starting values, not measured tuning results.
4. Capture watermark N and process bounded batches (initial target: 200 Paths,
short transactions). Persist derived changes and progress together. Reconcile
current state; do not replay stale rename payloads as commands. Recheck source
generation before publishing a completed snapshot.
5. Acknowledge only processed N. If desired advanced while queued/running, enqueue
a successor **after the active job finishes**, or let the dispatcher do so.
Current active-key queue deduplication must not acknowledge/drop newer edits.
6. Retries are idempotent; unchanged data does not increment topology version.
Keep itemized conflicts/cursors. Queue-full, pause, crash and cancellation
preserve pending invalidations and unprocessed events.
7. `city.reconcile.sweep` is a fair, bounded recovery pass across Domains using
the same algorithm. Compact outbox only after consumers' checkpoints/retention
requirements allow it.
Use existing `/_/api/v1/event/` and `/_/api/v1/queue` definitions, trigger, job and
log mechanisms. Proposed City event types are **not registered yet**. When POST
`/city/autoplan` becomes asynchronous, return `202` with a job UUID and update
its UI simultaneously. Dry-run needs bounded results and a base/rule-version
token; stale approval must not silently apply a newly recomputed plan.
Settings are Domain-scoped. Existing Domains start manual; enable URL-auto after
an adoption preview. New-Domain URL-auto defaults can follow once gates pass.
Manual/scheduled triggers share settings and permission checks. Workers may
mutate derived City state only, not arbitrary Pages using an administrator token.
Use low priority, one City writer per Domain, shared preview/import resource
budgets, cancellation and backoff. Keep geometry out of write transactions and
request handling. This is single-writer operation: current startup requeue of
all running jobs is not an HA lease; multi-instance workers require fencing.
## Geometry and read scalability
- Fabric key = Domain UUID + explicit seed + generator/version/options. Ordinary
edits and topology increments must not change the seed.
- Layout key = fabric + structural/membership generation + layout version;
metadata/policy generations are separate. Preserve prior assignments with
bounded growth/shrink hysteresis to avoid global rearrangement for one Page.
- Build a connected road graph, clip at Quartier boundaries, allocate fairly,
project anchors onto bounded segments. Keep every locality pickable with
explicit fallback/capacity warnings when geometry is scarce.
- Deduplicate weights at every level; leave unoccupied space when membership
is small. Use zoom-dependent labels/aggregation for dense points.
- Worker generates/allocates with size/time/cancellation limits; viewport/zoom
bounds main-thread rendering separately. Superseded results cannot replace
newer views. Remove observer-driven placement repair after regression tests.
- Top-level API returns Quartier summaries, not every descendant Page. Lazy
bounded Street/Spot/Page APIs use bulk queries, snapshots and cancellation.
- Cache authorized models by Domain, principal/policy and data generation with
private HTTP validators. Invalidate on logout/policy change; don't put restricted
labels/coordinates in a shared origin-wide cache. Topology-independent fabric
can be cached separately. Remove unused `city-geometry.js` storage work.
- During regeneration keep the last valid authorized map, with a small status
icon. Failures provide a directory fallback/retry, never block Page navigation.
## Compact UI and keyboard contract
- Toolbar: breadcrumbs, search, zoom/reset, add icon, settings icon. Generation
status tooltip links to details and existing job monitor, not verbose inline text.
- Add opens one modal for Quartier/Street/Spot with contextual parent/Path defaults
and ancestry/conflict validation. Page bindings/relations use the same modal
shell. Preserve inputs on failure, prevent duplicate submits and scope parents.
- Settings modal: manual/URL-auto, adoption preview, rules/exclusions, event timing
and explicit regeneration. Changing fabric seed is a separate confirmed action.
- Accessible icon names/tooltips, adequate hit areas, visible focus, Japanese and
English strings, trapped/restored modal focus. Escape closes innermost overlay
before outer map popup. No global form-input shortcuts.
| Key, only while map navigation is focused | Action |
| --- | --- |
| Arrows; W/A/S/D; K/H/J/L | Focus nearest visible selectable node up/left/down/right, without activation. |
| Enter / Space | Open locality or activate Spot portal/Page, matching click behavior. |
| Backspace | Parent City level, without leaving the page. |
| `+` / `-`; `0` | Zoom in/out; reset viewport. |
| Shift+F10 / Context Menu | Context menu on focused node. |
| Escape | Close menu/modal/map popup from inside outward. |
| Tab / Shift+Tab | Native accessible traversal, roving tab stop within map nodes. |
Do not intercept input/textarea/select/contenteditable, IME or Ctrl/Alt/Meta
shortcuts. Pointer drag pans; directional keys navigate nodes, with defined
offscreen/empty-view fallback. Refresh preserves selection/zoom; deletion falls
back to nearest valid parent. Guard async inspector/load responses by selection
and request generation, and cancel obsolete requests.
Locality history is an in-memory stack, not a browser entry for each movement.
Existing back/forward buttons invoke browser history; they cannot provide
per-locality traversal without entries. Provide distinct map back/forward
controls instead of pretending these two requirements are equivalent.
## Delivery and acceptance checklist
1. [ ] **CITY-A — correctness gate**: denied descendants, literal Paths, scoped
names/links, shallow/portal rules, source-aware idempotent plan/apply.
2. [ ] **CITY-B — durable events**: outbox and watermarks, consumer, CLI/import/
adapter coverage, concurrent-edit/rollback/retry tests, manual/scheduled/auto UI.
3. [ ] **CITY-C — responsive map**: stable seed/cache, Worker, connected allocation,
capacity fallbacks, lazy authorized reads and continuity/stale-result tests.
4. [ ] **CITY-D — compact navigation**: icons/modals, keyboard/i18n/focus/history,
popup and error recovery browser tests.
CITY-C/D may proceed independently of CITY-B once CITY-A's read/identity
contracts are established. Do not enable automatic production reconciliation
before CITY-A/B pass. Main-roadmap P0 authentication work remains open.
Fixtures: 500 / 5,000 / 50,000 Paths, aliases, many tiny Streets, one crowded
Street, virtual folders, repeated names, `%`/`_`/Unicode, manual/archived mappings,
denied descendants, rename/edit during generation, crash/queue-full/retry/cancel.
Record API p50/p95, response size, query count, SQLite write-lock duration,
generation CPU/memory and browser long tasks before fixing production budgets.
Audit checks performed locally:
- Passed existing tests: `GOCACHE=/tmp/pageswiki-go-cache go test
./internal/store ./internal/server -run 'TestCity|Test.*City|Test.*Topology'
-count=1`. These do not cover all findings above.
- Ran the checked-in gen-city bundle in a Node VM three times, seed key
`audit-city:1`, dimensions 240×160. Synchronous call times: about 39/32/29 ms;
259 roads and 279 buildings each. A zero-delay timer did not run before
generation completed. This supports the source finding of synchronous work,
not a browser/production latency benchmark.
- No live DB mutation, schema migration, runtime/UI change or deployment in
this audit.
+76
View File
@@ -0,0 +1,76 @@
# City generation Worker and directional navigation
Implemented bounded CITY-C/D slice, 2026-09-14. Preserves the generated fabric
and [courtyard projection](city-immersive-map.md); this does not replace the
server topology/reconciliation work in [the City audit](city-review-and-reconciliation.md).
## Generation and lifecycle
`city-generator-client.js` retains the asynchronous `PagesWikiGenCity` facade,
but the generator bundle is no longer loaded into the main document. A local
`city-generator-worker.js` imports the rebuilt gen-city 1.4.6 bundle and returns
only numeric roads/building polygons. Its synchronous internals therefore cannot
block browser input or navigation. The Worker is terminated when finished, on
abort/supersession, on page suspension, or after a 5-second deadline. There is
one active generation job per map/client. Same-key requests share the job.
Inputs are limited to a 256-character stable Domain key and integer dimensions
64–320 by 64–240 (the map requests 240 by 160). The adapter and Worker enforce
these bounds; returned geometry is limited to 4,096 roads, 4,096 buildings and
16 finite vertices per building. The main-thread deadline can kill runaway
synchronous Worker code without waiting for it to process a cancellation message.
The base seed is the Domain UUID, not a topology version or membership count.
Geometry-only caches use the generator/layout version, Domain UUID and dimensions
as their key. Memory and sessionStorage each retain at most three fabrics; a
serialized storage entry must fit 512 KiB. Storage denial, quota exhaustion and
corruption do not disable generation. No Page names, Paths, grants, topology
or principal information are cached here. Authorized API results remain separate.
The map aborts superseded fetches and discards late render responses. `pagehide`
cancels fetch/generation; BFCache `pageshow` reloads the authorized model and
reuses valid geometry. An unsupported/blocked Worker, timeout or malformed result
shows a retry action and error without executing gen-city synchronously as a
fallback. Utility menus remain usable; this is not a claim that the map works
without Workers. The SVG exposes its busy state to assistive technology.
## Keyboard interaction
`city-navigation.js` handles directional focus over currently visible Quartier,
Street, Spot entry, Page/portal links and overflow controls:
- Arrow keys, WASD and HJKL choose the nearest useful candidate in the requested
screen-space direction; there is no arbitrary wrap across the map.
- Enter/Space activates the focused element. Repeated activation keydown is
suppressed to avoid drilling through multiple levels while a key is held.
- Backspace moves to the parent locality and restores the former child's focus.
At the Domain level it stays in the map. It never calls browser back/history.
- Ctrl/Command/Alt shortcuts, IME composition, editable fields, controls outside
the map, open modals and context menus are not consumed. Ctrl selection and
existing dialogs continue to work. Escape remains available to the containing
popup/menu rather than being repurposed for hierarchy navigation.
The existing mouse, semantic Page hrefs, popup top-window Page navigation,
horizontal labels and courtyard design are unchanged. Navigation changes local
view state only; actual Page links still navigate normally.
## Verification and remaining limits
Opt-in Chromium tests run in normal and popup modes with the real Worker and
real gen-city output. They cover stable repeated seeds, a deliberately busy
Worker while main-thread timers keep running, timeout/abort/supersession,
stale messages, storage bounds, invalid results and pagehide/BFCache resumption.
Keyboard tests cover all key families, semantic Page activation, focus restoration,
modifier/IME/editing/modal exclusion and unchanged URL/history. Existing label,
organization, immersive/pagination and mobile-light-theme fixtures still pass:
```sh
PAGESWIKI_BROWSER_TEST=1 go test ./internal/server -run 'TestCity(Labels|Organizer|Immersive|Keyboard|Worker)Chromium' -count=1
```
This isolates base-fabric generation, not all City work. Topology-to-road
allocation, SVG DOM construction and loaded directory/inspector options remain
on the main thread. Connected road allocation, lazy locality projections and
larger-scale DOM virtualization remain separate tasks. A future server-side
geometry cache could share work across tabs/devices; no cross-tab worker pool
or server generation scheduler is introduced here.
+194
View File
@@ -0,0 +1,194 @@
# Configuration panels, bookmarks, and resource charts
Resource history is now collected by a built-in background event, not by an
open browser. The current persistence, retention and extension-catalog design
is specified in [System extensions](system-extensions.md); it supersedes the
earlier in-memory live-sampling description below.
Bookmark sections and direct home-page management are specified in
[Home navigation and hosted Vim](home-bookmark-sections-vim.md). The flat
bookmark representation below remains compatible; section rows extend it.
## 設定画面の構成
`/-/sys/config?panel=<id>` を共通シェルとする。`config_panels.go` の
サーバー所有レジストリがグループとパネルIDを定義する。
| グループ | パネルID |
| --- | --- |
| 表示・ナビゲーション | `home`, `editor`, `bookmarks`, `nav-home`, `nav-system` |
| コンテンツ | `objects`, `assets` |
| 認証・アクセス | `auth`(アカウント), `auth-providers`, `auth-sessions` |
| キャンパス・ドメイン | `campuses`, `domains` |
| 連携アダプタ | `adapters`, `adapters-builtin`, `adapter-<plugin-id>` |
| 運用 | `scheduler`(実行状態・リソース), `logs`(実行履歴・ログ), `events`, `queue` |
選択グループを展開し、他グループは折りたたむ。各summaryは手動でも開閉可能。
サイドバー全体を閉じた場合は本文の幅も広げる。小画面は縦並びとする。
パネル選択をURLに反映するため再読み込み/直接リンクで同じパネルを開ける。
既存の `panel=nav` は `nav-home` として扱う。
共通CSSは `/static/css/config.css`、遷移・APIエラー処理・グラフは
`/static/js/config-panels.js`。チェックボックスはネイティブinputを残した
`role=switch` とし、labelの `data-label` をCSSの `attr(data-label)` で表示する。
inputにも翻訳済み `aria-label` を付け、キーボード操作・disabled・フォーム所属・
送信値を保持する。秘密情報は既存のpassword入力と空欄時保持の契約に従う。
## スケジューラの情報分離
- 実行状態: 件数、直近25ジョブ、再試行/キャンセル、プレビュー生成の手動トリガー。
- イベント: 実施タイミング、次回実行、追加/実行/削除。周期と日時は選択した
スケジュール方式に対応する項目だけ表示。優先度・試行回数・JSONは詳細設定に置く。
- キュー: 一時停止、並列数、キュー深度、リトライ待ち時間、履歴保持。
- ログ: 直近50件の実行ログ。
- リソース: 下記のライブグラフ。
従来はスケジューラを開くと4種類のAPIをまとめて取得していた。現在は選択した
パネルで必要なAPIだけ取得し、遷移時に中止する。定期取得はリソースパネルに限定。
設定フォームを定期更新で上書きしない。APIの非JSONエラーも本文/ステータスとして
表示し、JSON構文エラーで本来のエラーを隠さない。
## アセット一覧の障害と是正
報告: `Unexpected token 'o', "forbidden: "... is not valid JSON`。
原因はGETがendpoint policyに登録されていなかったこと。POST/DELETEの登録により
同じエンドポイントがmethod-closedになり、一覧GETが認可ハンドラに到達する前に
拒否されていた。GET/POST/DELETEは型付きpolicyからアセット管理ハンドラへ渡し、
現行のキャンパス/ドメイン管理者権限をハンドラで検証する。
調査時の `asset:image:profile` は、HEAD、履歴、実ファイルSHA-256が一致し、
記録サイズ・実サイズともに2,627バイトだった。データを削除・再生成する必要はない。
あわせて以下を是正した。
- 参照数を「同一ドメインにPathを持つ参照元Revisionの異なり数」に統一。
過去HEADと現在HEADを含める。詳細はPath別の行を返すため、同じRevisionの
別名Pathがある場合の行数は参照数と一致するとは限らない。
- HEAD履歴がない旧データも、現在HEADから依存関係を取得可能にした。
- 更新/削除は書き込みSQLでも期待HEADを比較し、競合を検出する。
- 同じバイト列でもMIME/ファイル名を変えた場合はメタデータと履歴を更新する。
- CLIのアセット書き込みもMIME、ファイル名、サイズ、履歴を同一トランザクションで更新。
- 一覧APIは `no-store`。画面はテキスト形式の拒否応答も読み取れる。
HEADハッシュのETagはバイト列の識別子である。メタデータだけの同時更新を厳密に
区別するには将来、Asset状態バージョンを含めた条件付き更新契約が必要。
## ブックマークマネージャー
`panel=bookmarks` でドメイン共通のブックマークを最大64件管理する。
行には名前、指定方法(Path / URL)、宛先、上下移動、削除を用意する。
保存時に管理者権限を検証し、入力が不正なら保存済み一覧を変更しない。
既存の `home_bookmarks` JSONを継承し、`kind` を任意追加する。
```json
[{"label":"Docs","kind":"path","href":"/docs"},
{"label":"External","kind":"url","href":"https://example.com/"}]
```
Pathはドメインルート相対の意味を持ち、URL化するときに空白・%・Unicodeを
適切にエスケープする。URLはHTTP(S)またはルート相対URLを許可し、JavaScript、
プロトコル相対URL、資格情報を含むURLは拒否する。ローカルの宛先は表示時にも認可する。
ホーム設定の保存ではブックマークを更新せず、別パネルの保存で消える事故を防ぐ。
現在は「検索+ブックマーク」のホームに反映し、共通サイドバー実装時に同じデータを使う。
個人ブックマーク、フォルダ分類、UUID追従型ブックマークは将来の別拡張とする。
## 設定テーブルの表示密度
設定画面の `.recent-table` / `.config-form-table` に限定して行の余白を圧縮する。
セルの上下paddingは0.22rem、行内コントロールは高さ1.8rem、セル内フォームの外余白は0。
空のform要素は表示しないが、form IDを用いた行内入力・保存/削除との関連付けは維持する。
名前と識別子を1行にまとめ、通常セルは折り返さず、幅不足はテーブル領域内で横スクロール。
長いPath・ファイル名・URL・補足説明は全文をDOMとtitleに残した省略表示とする。
ログ展開行とその本文は例外として複数行表示を保つ。Viewer/Editorなど他画面のテーブルには適用しない。
## リソースグラフ
`scheduler` パネルはhtop型の監視画面とし、上部に現在値と4つのグラフ、下部に
実行中ジョブのみを表示する。旧 `panel=resources` は `scheduler` の別名として維持。
収集設定・表示期間・自動更新・指標の説明は、タイトル横の歯車から開くネイティブdialogに
まとめる。閉じるボタン・Escape・背景クリックで閉じ、歯車へフォーカスを戻す。
別パネルへの移動時も閉じる。表示設定は即時反映し、同一originのlocalStorage
`pageswiki.monitor.display.v1` に期間・自動更新ON/OFF・現在値とグラフの更新間隔を保存。
初期値は1時間・自動更新ON・現在値1秒・グラフ5秒。保存された期間があれば優先する。
不正な値は使わず、storage利用不可でも既定値で動作する。収集設定は保存操作でDBへ永続化。
時間範囲はタイトル直下に1つだけ表示し、各グラフの時間ラベル・表示点数は出さない。
更新操作はリロードアイコン、設定は歯車。履歴へのリンクはサイドバーに集約する。
グラフは常に1行4列。狭い画面ではグラフ領域内だけ横スクロールし、2列/1列へ折り返さない。
`GET /_/api/v1/queue/resources` の現在値と `GET /_/api/v1/queue/jobs?status=running&limit=100`
を既定1秒間隔で取得する(要求完了後に次を予約し、重複実行しない)。実行中の絞り込みはLIMITより前にDBで行い、最近終了した
ジョブに長時間実行ジョブが埋もれない。ワーカー上限32に対し表示上限100。
現在値のキュー統計はstatus索引で待機・実行中だけを集計し、完了履歴を全走査しない。
グラフは既定5秒ごとに、現在値から得たライブサンプルを履歴へ重ねて描画する。
両間隔は1〜60秒で設定可能。グラフ間隔は現在値の間隔以上に補正する。
永続履歴APIの再取得は30秒ごとに抑え、期間変更と手動更新時は即時再取得。
ライブバッファも描画も最大361点。I/O速度は間引き前に計算し、時間バケットの最終値を描く。
ブラウザ内ライブサンプルはDBへ書き戻さず、再起動後は永続サンプルから復元する。
バックグラウンド収集は引き続きビルトイン
`resource.sample` イベントで行い、画面を閉じても7日間保持する。
詳しい収集契約は [システム拡張](system-extensions.md) を参照。
パネルを離れる、タブが非表示になる、自動更新を止める場合は定期取得を停止する。
取得中リクエストはAbortControllerで中止し、古い結果が再表示しないよう世代を管理する。
グラフはテーマの色を使ったSVGで、外部ライブラリ/CDN不要。
### 実行履歴とログ
`logs` パネルにジョブと実行ログを統合。既定は終了状態
(succeeded/skipped/failed/cancelled)、待機・個別状態のフィルタも用意する。
`GET /_/api/v1/queue/jobs?status=history&limit=50&cursor=...` のkeysetページングで
50件ずつ置換し、DOMへ無制限に追記しない。順序は投入時刻降順+UUID降順。
完了状態用partial indexを指定し、SQLiteが状態別索引のIN検索後に保持履歴全体を
一時ソートする計画を避ける。ジョブpayloadは一覧SQLでも応答でも読み出さない。
新しいジョブへは更新ボタンで戻る。cursorは位置を表すのみで認可は毎要求再検査する。
ログはジョブの展開時だけ `/_/api/v1/event/logs?job_uuid=...&limit=100` で取得。
以前のログは `before_id` で100件ずつ置換する。内容はHTMLではなくテキストとして表示。
実行中のキャンセル、失敗等の再試行を維持する。終了した実行中行は次回更新で消える。
実行状態は秒数・試行数を示すが、ジョブ別のCPU/RSS計測や進捗率を捏造しない。
| 指標 | 意味 |
| --- | --- |
| CPU | ホストの1分ロードアベレージ。CPU使用率ではない |
| メモリ | このGoプロセスのheap bytes |
| I/O | PagesWikiプロセスの読み込み・書き込み速度(B/s)。2系列で表示 |
| キュー負荷 | 待機ジョブ数+実行ワーカー数 |
これらはドメイン別ではなくインスタンス/ホスト単位の指標。
ディスク容量は現在値カードに残し、グラフの1枠はI/Oに使う。I/Oの計測範囲と制約は
[システム拡張の監視仕様](system-extensions.md#ディスクio)を参照。
コンテナCPU制限・RSS・cgroup使用率・複数インスタンス・長期保管は将来の
監視プロバイダ拡張として別API/保持方針を設計する。
## アダプタ個別パネルの拡張契約
現行プラグインは `adapter-<id>` に個別パネルを持ち、共通partialで接続先・
対応MIME/拡張子、有効化、別タブ、既定エディタ、HTML/Markdown変換を編集できる。
旧 `/-/sys/config/adapters?id=...` のGETは統合パネルへリダイレクトする。
保存も同じパネルに戻り、ドメイン別の既存プラグイン設定へ保存する。
将来のアダプタ固有項目は次の契約で追加する。
1. サーバーが信頼するmanifestに `settings_schema_version` と複数のパネル定義を登録。
定義は `id`, `title_key`, `group`, `fields` とし、フィールドはboolean、text、
enum、number、secretの許可リストから選ぶ。現行partialがその最初の実装。
2. 保存先は `(domain_uuid, adapter_id, schema_version)` で区切る。バリデーションと
schema移行はサーバー所有。未定義キーは拒否し、秘密値は読み出しレスポンスに含めない。
3. 特殊UIが必要な場合のみ、サーバー登録済みの専用partial/モジュールを使う。
プラグインやドキュメントから任意HTML/JavaScriptを設定画面に挿入しない。
4. 外部設定画面の埋め込みが必要な場合は、別途sandbox iframeと固定originの
postMessage契約を設計する。通常ログインCookie/Bearerをiframeへ渡さない。
5. アダプタ固有の接続確認・秘密管理・長い検証処理は個別APIとイベントへ分離。
汎用schema rendererと外部設定iframeは未実装。今回の共通シェル・レジストリ・
アダプタpartialを拡張点として使用する。
## 検証
- 管理者のAsset GETが成功し、一般ユーザーのGETが拒否されるHTTP回帰テスト。
- 参照数のドメイン分離、旧HEAD参照、同一バイト列のメタデータ訂正。
- ブックマーク保存/不正URL拒否、アダプタ個別設定の保存と戻り先。
- 設定HTMLの重複ID、メニューの未定義パネル、inline JavaScriptの構文検証。
- Chromiumでassets/events/queue/resources/bookmarks/adapter個別パネルを確認。
`PAGESWIKI_BROWSER_TEST=1 go test ./internal/server -run TestConfigChromiumSmoke`
- CLI: `python3 -m unittest discover -s data -p test_cli_assets.py`。
+188
View File
@@ -0,0 +1,188 @@
# Data CLI
Remote HTTP access is a separate subproject, `pages-cli`, with the `pages`
command (`pages-cli/pages`); see the
[remote CLI contract and usage](ja/remote-cli.md). This local tool retains its
trusted administrative database access and does not acquire Web credentials.
`data/pageswiki-cli` is a local administrative data surface. It provides
rsync-like one-shot transfers and an interactive Semantic Path shell.
It accesses:
- metadata in `pages.db`;
- immutable payloads in `documents/sha256/`;
- mutable Asset HEAD metadata in `assets`.
The implementation uses Python's standard library only.
## Invocation modes
### One-shot
```bash
data/pageswiki-cli get path:/manual/start start.html
data/pageswiki-cli put start.html path:/manual/start
```
Transfers compare SHA-256 before writing. Identical content is reported as
`skipped` and does not create another Revision.
### Interactive shell
```bash
data/pageswiki-cli shell
```
Invoking without arguments also opens the shell when stdin is a terminal.
The shell maintains a Semantic Path working directory:
```text
pages:/> cd /manual
pages:/manual> ls
pages:/manual> view start
```
The shell uses `readline` when available:
- Up/Down traverse command history.
- Tab completes commands, Semantic Paths, UUIDs, Revisions, Assets, and local
transfer paths.
- History persists in `~/.pageswiki_history`, or the path specified by
`PAGESWIKI_HISTORY`.
## Commands
| Command | Behavior |
| --- | --- |
| `ls [PATH]` | List direct Semantic Path children |
| `cd PATH` | Change shell Semantic Path |
| `mv SOURCE DEST` | Move a Path and its descendants transactionally |
| `view [ADDRESS]` | Display metadata and send payload to `PAGER` on a TTY |
| `cat [ADDRESS]` | Write only the canonical payload to stdout |
| `edit [ADDRESS]` | Edit through `EDITOR` and write back changed content |
| `get ADDRESS [LOCAL]` | Download one payload |
| `put LOCAL ADDRESS` | Upload a Page Revision or update an Asset HEAD |
| `mget REMOTE_PATTERN [DIR]` | Download matching Semantic Paths |
| `mput LOCAL_GLOB REMOTE_DIR` | Upload matching local files |
| `revision [ADDRESS]` | List a Page's immutable Revision history |
| `diff REV REV` | Produce a unified Document diff |
| `restore REV` | Create a new Revision from an old Revision |
| `link ADDRESS PATH` | Add a Semantic Path projection |
| `unlink PATH [ADDRESS]` | Remove a Path projection only |
| `pwd` | Show the shell Semantic Path |
`put` and `mput` infer a Revision Content-Type from the local filename, with
the destination Semantic Path as a fallback. Markdown input is converted to
canonical HTML and loses only its `.md` / `.markdown` Semantic Path suffix.
Other extensions remain part of the Path. Binary payloads are stored as
immutable Documents and retain their original filename and Content-Type.
| `help` | Show usage |
| `exit`, `quit` | Leave the shell |
`get ADDRESS -` writes to stdout and `put - ADDRESS` reads from stdin.
Unlike `view`, `cat` never invokes a pager and is suitable for pipelines.
## Supported addresses
| Address | Meaning |
| --- | --- |
| `path:/...` | Mutable Semantic Path projection |
| `uuid:<uuid>` | Stable Page/Object identity |
| `rev:<uuid>` | Immutable Revision identity |
| `asset:<type>:<name>` | Mutable Asset HEAD |
A single address without a command retains the earlier inspection behavior.
## Write semantics
### Page `put`
1. Resolve `path:` or `uuid:`.
2. Create a Page UUID and Path binding if a Path does not exist.
3. Hash the input and atomically persist its immutable Document.
4. Compare with the newest Revision in `PAGESWIKI_LANG`.
5. Skip identical content or create a new Revision UUID.
Page payloads must be UTF-8. Binary data belongs in an `asset:` address.
### Asset `put`
1. Hash and atomically persist arbitrary bytes.
2. Create the Asset identity if absent.
3. Skip an identical HEAD or update `head_document_hash`.
### `edit`
1. Resolve the current canonical payload.
2. Write it to a private temporary file.
3. Invoke the command in `EDITOR`.
4. Apply Page or Asset `put` semantics when the editor exits successfully.
5. Skip unchanged content and remove the temporary file.
Immutable `rev:` addresses cannot be edited.
### `mv`
- Semantic root cannot be moved.
- A destination cannot be inside its source.
- Every destination is collision-checked before mutation.
- A unique temporary namespace avoids primary-key collisions.
- All Path updates commit in one SQLite transaction.
### `restore`
Restore never rewrites history. It reads the selected immutable Revision and
creates a new Revision on the same Page with the same language.
### `link` and `unlink`
These commands mutate reachability only. They never delete the Page,
Revisions, or Documents. An unlinked Page may become a zombie and remains
recoverable by UUID until garbage collection.
## Consistency and safety
- SQLite writes use explicit transactions and a busy timeout.
- The CLI initializes the additive `assets` table when absent; core schema
migrations remain owned by the server.
- Documents use temporary files, `fsync`, and atomic replacement.
- Payload hashes are verified on every read.
- Revision addresses reject writes.
- rsync-like transfers skip identical hashes.
- Missing canonical Documents may use the transitional SQLite body only when
its SHA-256 matches.
This CLI is a trusted local administrative interface and intentionally bypasses
future HTTP Capability evaluation. Filesystem and database permissions define
who may use it.
## Environment
| Variable | Meaning |
| --- | --- |
| `PAGESWIKI_DATA_DIR` | Directory containing `pages.db` and `documents/` |
| `PAGESWIKI_DB` | Explicit SQLite path |
| `PAGESWIKI_DOCUMENTS_DIR` | Explicit Document root |
| `PAGESWIKI_LANG` | Language used for selection and Page writes |
| `PAGER` | Pager invoked by `view` on an interactive terminal |
| `EDITOR` | Editor invoked by `edit` |
| `PAGESWIKI_HISTORY` | Interactive command-history file |
## Asset metadata
The normative Asset lifecycle and authorization model are defined in
[`asset-model.md`](asset-model.md). The CLI is a trusted local implementation
of HEAD reads and updates, not the public Asset API.
| Column | Meaning |
| --- | --- |
| `domain_uuid` | Domain containing the mutable Asset name |
| `address` | Canonical Domain-local `asset:` address |
| `object_uuid` | Stable owning Object UUID |
| `head_document_hash` | Current immutable Document snapshot |
| `created_at_ms` | Creation timestamp |
| `updated_at_ms` | HEAD update timestamp |
Administrator HTTP Asset mutation is available with Domain/Campus-admin checks
and ETag preconditions. The CLI remains the trusted local bulk interface.
+278
View File
@@ -0,0 +1,278 @@
# 日記・ブログ:ビルトインシステム拡張
初期版 0.1/URL・入口拡張(2026-09-17)。MIME連携アダプタではなく、通常のWiki Pageに
付随する日付・記事情報とナビゲーションを提供する。任意の外部コードをロードする
プラグイン機構ではない。
## 実装した範囲
| | 日記 | ブログ |
| --- | --- | --- |
| ID | `content.diary` | `content.blog` |
| UI | `/v/diary` | `/v/blog` |
| 設定 | `/-/sys/config?panel=diary` | `/-/sys/config?panel=blog` |
| 一覧 | 月別/日付別 | 日付降順、任意の月・カテゴリで絞り込み |
| ナビゲーション | 月カレンダー、最近8件、自分の記事 | 月カレンダー、最近8件、カテゴリ、自分の記事 |
| 登録制約 | 同日複数件可 | 同日複数件可 |
両方ともDomainごとに**既定無効**。Domain管理者が有効化する。無効化しても
登録情報、Wikiページ、Revisionは削除しない。
「新規記事」ダイアログは日付・keyword・タイトル・カテゴリ・Markdown本文を
まとめて保存する。既存Pageを使う場合は従来のPath指定による登録も可能。
本文の継続編集は既存エディタ、記事情報の変更・登録解除は一覧から行う。
## 同一インスタンスのDomain射影(第一段階)
- ドメイン一覧の歯車内で日記・ブログそれぞれの参照元DomainとPath prefixを指定。
空の参照元は従来の自Domain記事を表示する。入口形式・有効状態は上の設定で指定する。
- `periodical_projections(domain_uuid,kind,source_uuid,prefix)`で参照のみ保持し、
Page、Revision、Path、記事情報は複製しない。既存の表示先記事も削除しない。
- Campus管理者による設定に限定。同じCampus(同じコントローラの管理単位)の
有効なlocal配置Domain同士のみ許可。managed Campus上のローカル設定変更は禁止。
自己参照・連鎖・循環を拒否し、閲覧時にも有効状態・配置・所属・連鎖を再検査する。
- 初期射影範囲は認証済みの一覧・月/日カレンダー・カテゴリ・最近の記事・own。
参照元・表示先の双方の読取ポリシーとprefixで絞り、件数/カテゴリ/ページ送りを
その後に計算する。ソース権限は一括読込し、記事ごとのSQL認可問い合わせはしない。
Source側がlegacyモードでも、射影では明示的な読取権限を要求する。
- 匿名の場合も、設定済みのlocal射影先の `/` と `/v/{kind}/YYYY/MM[/DD]` を
閲覧できる。参照元の現在有効な公開承認済みPath/Revisionだけから一覧・件数・
最近の記事を構成する。日付は承認済み `{prefix}/YYYY/MM/DD[/keyword]` Path、
**登録タイトルはPageの公開状態を継承する公開表示情報**とする(2026-09-19変更)。
現在有効な公開承認のあるPageのタイトルだけをJOINで取得し、タイトル変更は
次の要求で反映する。カテゴリ・著者・登録日付は公開しない。タイトル未登録なら
最新20記事に限り承認済みHTMLのh1を64KiB以内で読み、なければ日付/Path末尾。
不正/期限切れの認証情報は匿名へ降格しない。公開変更/失効は次の要求で反映する。
記事本文の公開権限を増やさず、未承認の最新Revisionを公開しない。
- 表示先の認証セッションでPrincipalを特定する。参照元の同じPrincipalに読取権限が
必要。クロスサブドメインCookie共有やセッションの暗黙継承は実装しない。
- 専用ドメイン(入口形式がblog/diary)では記事リンクを表示先の
`/YYYY/MM/DD[/keyword]` とし、承認済み本文を同じホストで表示する。
カレンダー・同月の他の記事(最大31件)・前後の記事を同じ公開集合から構成する。
サイト名称・テーマは表示先Domainのもの。記事本文は既存のinert HTML境界を使う。
本文中の既知の公開記事リンクも表示先へ変換し、それ以外は参照元URLを保持する。
画像のCSPは設定済み参照元originだけ追加許可し、画像自体の公開権限は拡張しない。
「参照元で閲覧・編集」は従来通り元FQDNへ移動する。作成・登録・更新・解除の
APIを射影先で使うと403。Store transaction側でも射影設定を再検査して拒否する。
- entries APIは`source_domain`・`source_origin`・`projection`を付加する。
Path/Page UUIDは参照元のもの。日付ナビゲーションは表示先の `/v/{kind}/...`。
日付既定値は参照元Timezone、テーマ・名称は表示先Domainの設定。
- 一覧は毎回DBから読み、`private, no-store`。ACL失効・元記事変更は次の要求で反映。
参照先が無効/remote配置/別Campusへ変わった場合は503で閉じ、自Domain記事への
黙った切り替えはしない。同期や既存公開承認をこの参照設定で拡張しない。
- この参照設定は現時点では**ローカル専用でmanifest v2には含めない**。
別インスタンスへの参照設定配信・認証付き取得・失効・
タイトル以外の匿名メタデータ公開・SSOは後続。従って「同じコントローラなら任意の配置先で射影」
までを実装済みとはしない。
- 専用ドメインの `/rss.xml` はRSS 2.0、最新20記事のタイトル・正規リンク・GUID・
Revision日時と、`content:encoded` の安全化済み本文HTML(CDATA)を配信する。
本文取得前に正確なRevision公開承認を再確認する。記事リンク・ページ内リンクは
表示先の絶対URL、画像やその他の参照元リンクは元ドメインの絶対URLに変換する。
上限は1記事1MiB入力、本文合計8MiB、生成処理5秒。上限超過は503とし、黙って
本文を切り捨てない。HEADは本文を生成しない。非公開登録情報は含めない。絶対URLは登録FQDNから
作り、Hostヘッダー由来のfeed汚染を防ぐ。HTMLにRSS autodiscoveryとRSSリンクを設置。
毎回公開承認を確認し、HTTP共有キャッシュは使わない。ただし購読者が既に取得した
RSSや本文のコピーは、公開撤回しても遠隔削除できない。
## 原子的な新規投稿のAPI
- `/v/blog/-/new`、`/v/diary/-/new` は認証必須の独立した新規記事画面。
Domain timezoneの日付・keyword・カテゴリ・Markdown本文を既存posts APIへ送る。
作成/Pathの権限はAPI内のtransactionで検証し、保存だけでは公開承認しない。
未有効のプラグインは404、射影先は参照元の同ルートへ303で案内する。
GETで記事やPathを作成せず、POSTはこの画面URLでは受けない。
- 投稿完了画面にXのWeb Intentリンクを表示し、ユーザのクリックでのみXへ移動する。
サーバーからの自動投稿・外部SDK・OAuth資格情報の保持は行わない。
posts応答の`share_targets`は有効な同Campus/local射影先と入口設定から生成する。
ブログ専用ドメインの `/YYYY/MM/DD[/keyword]` を優先し、複数なら選択式。
射影設定があるのに利用可能な共有先がなければ、元ドメインへ黙って切り替えない。
`published`は公開承認の検査結果。未公開なら警告とクリック時の確認を出す。
共有操作で公開承認は変更しない。公開記事ページにはタイトル・canonical URLの
OGP/Xカードメタデータを出す(カードの表示や更新時期は外部サービス側に依存)。
- `POST /_/api/v1/extension/content.{diary|blog}/posts`:`request_id`(UUID)、
`title`、`date`、`keyword`、`category`、`body`(Markdown)。未知フィールド拒否。
要求JSON上限320 KiB、変換後HTML上限256 KiB。ブラウザ本文入力上限131072文字。
- prefixとTimezoneはサーバー設定を使用する。Path・Page UUID・著者の指定は不可。
通常のMarkdownレンダラでHTML化し、通常のRevision挿入関数とasset固定処理を共有。
- SQLite write予約後にactive membership、ローカル配置、有効状態、create/writeの
両権限、上位Locality制限、現在のprefix、Pathの不存在を検査する。
Page・provenance・Path・Revision・asset参照・記事情報・journal・再送receiptを
同一transactionで確定する。既存Pathは409で拒否し、上書きしない。
- `periodical_posts`のDomain/kind/actor/request_idと要求digestで再送を識別する。
同じ要求は既存Page UUIDを返し、変更された要求は409。再送にも現在の権限と
Path binding/記事登録の存続を要求し、削除後の再送で復元しない。
- Document Storeのimmutable blobはcommit前に書き込む。DB rollback時には
未参照blobが残り得るが、Page・Revision・Path・記事索引は可視化されない。
未参照blob回収とreceipt保持/GCの統合は後続。プロセス内revision書込ロックを共有。
- 本文保存は公開承認ではない。匿名の本文・記事情報・日付・件数を追加公開しない。
- 日付付き一覧へ遷移して保存結果を表示する。ダイアログ内の入力は送信失敗時に
保持し、同じrequest_idで再試行する。ブラウザ永続ストレージには下書きを保存しない。
- Search/公開失効など既存DB triggerは共通Revision/Path挿入で適用される。
City自動生成・通知への共通outbox配送は依然として後続(journalだけで配送済みとしない)。
## 記事URLとドメイン入口(2026-09-17)
- 設定画面の日記/ブログパネルにDomain・kind単位のprefixを追加。既定値は
`/diary`・`/blog`。`/`も指定可能。設定変更は既存Pathを移動しない。
- 「新規ページ」は日付・任意keywordを指定するダイアログを開く。
`{prefix}/yyyy/mm/dd` または `{prefix}/yyyy/mm/dd/{keyword}` をサーバーで
検証し、書き込み権限とPathの不存在を確認してエディタへのリンクを返す。
日本語keyword可。keywordは単一要素とし、traversal・制御文字・URL区切りは拒否。
- エディタを別タブで開き、本文保存後に元のダイアログから記事登録へ進む。
この任意の従来フローは二段階操作。新規記事ダイアログでは一括保存できる。
計画APIはPageを作成・予約しない。確認後に別の操作が同じ
Pathを作成した場合、既存エディタはそのPageを開くため、排他的な新規作成保証は
専用の原子的投稿APIが提供する。計画APIは自動上書き・自動登録を行わない。
- keywordなしのPageとkeyword付きの子Pageは同日に共存可能。
`idx_diary_owner_date`はschema初期化で削除し、既存データは保持する。
- Domain設定のトップページ形式に日記/ブログを追加。`home_layout=diary|blog`
は認証後の `/` で直接一覧・カレンダーを描画する。日付選択は以下のviewer Pathへ
進む。従来の `/v/diary`・`/v/blog` は残す。既存home_pathを削除しない。
- 配置先の表示名を「配置先」に短縮。物理的なInstance FQDN・fingerprintと
home_layoutは別設定のままとし、別Domainの記事や権限を暗黙に共有しない。
- 匿名公開処理は従来の承認済みPage/ディレクトリ表示を維持する。
home_layout設定で記事メタデータを公開しない。設定済み射影の匿名カレンダーは
上記の公開Path/Revision由来データのみで構成する。
- Domain一覧の歯車「ドメインの入口設定」からトップページ・日記/ブログの有効状態・
prefixを一括指定できる。配置先そのものは既存のDomain更新フォームで指定する。
初回ダイアログは現在の設定値を引き継ぐ。保存後はこのパネルを設定の正本とし、
homeパネル/個別プラグインの設定APIからの競合する更新を拒否する。
### コントローラからの配信
- `domain_entry_settings(domain_uuid,settings_json)`が明示的な入口設定を保持。
Campus管理者のみ更新可能。外部キャンパス管理下ではローカル更新不可。
設定検証・管理権限確認・保存・ローカル配置への反映は同一transaction。
- 署名付きCampus manifestに`domain_entries`を追加。明示設定がある場合だけ
schema_version=2を使用し、設定のない従来manifestはversion=1のままとする。
入口設定も署名・内容hash・revision更新の対象。対象CampusのDomainのみ含める。
- 受信時はDomainの所属・Layout列挙・prefix・選択したプラグインの有効状態を検証。
Domain snapshot、入口設定、local配置のhome_layout/periodical設定、受信checkpointを
同じtransactionで更新する。remote配置のDomainには実行設定を有効化しない。
- 記事本文・記事登録・公開承認・資格情報・任意configは配信しない。
prefix変更も記事移動を発生させない。同一revisionの再送は従来どおり冪等。
- v2非対応ノードは同期を拒否するため、**配置先を先に更新してから**入口設定を保存する。
enrollmentで`manifest_version=2`と専用の`capability_signature`を送信し、
インスタンス鍵で署名した`normalized FQDN + LF + token + LF + manifest=2`を検証する。
従来のenrollment署名も併せて必須。古いクライアントの申告なしはversion=1として扱う。
- `federation_instance_capabilities(instance_uuid,manifest_version)`で鍵identity単位に
能力を記録。FQDNが同一でも鍵変更は別の申告・承認が必要。
Campus manifestは全承認済みノードに渡るため、一台でもv2未確認なら入口設定保存を
409で拒否し、ダイアログにも案内を表示する。受信側は従来どおり署名・versionを検証し、
設定を削ったv1へ黙ってフォールバックしない。
- capabilityは対応機能の申告であり、稼働状態・実際の反映完了の証明ではない。
enrollment更新時に申告がなくなればversion=1へ戻す。署名は既存enrollment同様
token期間中のproofでありnonce付きの鮮度証明ではない。反映ACK/鮮度管理は後続。
- manifestで設定が省略された場合は設定解除を意味しない。既存設定を保持する。
現段階では管理を解除する操作はなく、Wiki形式への変更・各プラグインの無効化も
入口設定パネルから明示的に行う。削除Domainの設定保持/GCは後続。
### 日付ナビゲーションの正規URL
- 月別:`GET /v/{diary|blog}/yyyy/mm`。
- 日別:`GET /v/{diary|blog}/yyyy/mm/dd`。
- 自分の記事:存在フラグ`?own`。カテゴリは`?category=...`、ページ送りは
`offset`を維持する。組み合わせは`?category=Work&own`。
- `/v/blog`は全期間、`/v/diary`はDomain timezoneの当月。
- 旧`month`/`date`/`mine=1`は認証確認後、302で正規URLへ転送する。
date単独でも対象月を導出する。矛盾する日付、存在しない日付、重複した日付query、
桁数不正は400。未知のHTTPメソッドを許可せず、現段階はGETのみ。
- カテゴリ/ownは月移動・日付選択・ページ送りで保持する。フィルタフォームの
再適用は日付選択とoffsetをリセットし、指定月(ブログは全期間も可)から表示する。
- 専用ドメインの `/` は入口。`/yyyy/mm/dd`は通常Pageの本文を置けるため、
日付別一覧をそこへマウントしない。月・日への移動は`/v/{kind}/...`を使用する。
- 新規投稿計画APIの`date`は記事作成パラメータであり、閲覧URLの日付queryとは別。
将来のQUERY対応はTODO。現段階ではルータや認証middlewareに追加しない。
## データモデル
- `periodical_settings(domain_uuid, kind, enabled)`:Domainとプラグイン単位の有効状態。
- `periodical_prefixes(domain_uuid, kind, prefix)`:投稿先prefix。未設定時はkind別既定値。
- `periodical_entries(domain_uuid, kind, page_uuid, owner_uuid, local_date,
title, category, timezone, version)`:正本はPage UUIDで参照する。Pathは保存せず
閲覧時に現在のbindingから解決する。renameで日付・カテゴリが失われない。
- `periodical_events(id, domain_uuid, kind, page_uuid, actor_uuid, operation,
version, created_at_ms)`:設定/登録/解除と同一transactionで記録する変更journal。
本文・タイトル・資格情報は複写しない。これは共通outbox配送や通知機能そのものではない。
本文はDocument Storeと既存Revisionにのみ保存する。登録解除はメタデータだけを
外し、Pageを削除しない。最後のPathがなくなったPageは一覧から非表示になる。
メタデータの孤立行のGC・完全消去・履歴保持期間は後続の保持ポリシーで扱う。
日付は明示したcivil date(YYYY-MM-DD)であり、UTC時刻への暗黙変換をしない。
作成時のDomain timezoneをサーバーで記録する。設定timezone変更後も過去の記事を
別日に移動しない。未来の日付を指定しても公開予約や時刻による自動公開は行わない。
著者は最初の登録時の認証済みPrincipalで固定し、クライアントのowner指定は信用しない。
## 権限と公開範囲
- 登録情報を利用するUI/APIは**認証必須**。legacyモードでも匿名メタデータ一覧を
許さない。設定済み射影の公開Path/Revision由来一覧は上記の限定例外。
- 設定変更はactiveなDomain管理者グループのowner/manager。
- 記事登録・情報更新・解除は対象Pathのwrite権限と上位Locality制限を再検査する。
`upload_only`や`read_write_once`だけでは、既存Pageの記事情報を書き換えられない。
既存エディタと同じく、本人の `/u/{username}` Wiki配下は本人の編集権限で扱う。
この場合もactive membership・実際のidentity・上位Locality制限を検査する。
- 一覧・カテゴリ・カレンダー件数・最近の記事・ページ送りは、現在のPage閲覧権限で
絞った後に構築する。非公開の別名で先にdedupeして可視Pageを失わない。
- 登録操作は匿名公開承認ではない。Page本文の匿名公開は従来の正確なRevision承認に
従う。登録タイトルのみPageの公開状態を継承し、それ以外は個別の承認契約が必要。
- 下書き/公開/予約公開を、この登録テーブルの有無や日付と混同しない。
公開一覧・RSS/Atomには記事メタデータの承認されたスナップショットと失効検査が必要。
変更transactionは最初にSQLiteのwrite予約を取得し、active membership、設定、Pathの
Page UUID、write grant、group/locality caps、期待versionを同一snapshotで検査する。
既存記事の変更にはPage UUIDも要求し、renameやPathの再bindを検出する。
削除・再登録後もjournalからversionを引き継ぎ、古い編集要求の再利用を拒否する。
古いversionによる登録更新は409で明示し、上書きしない。同日複数記事は許可する。
## APIと負荷上限
`/_/api/v1/extension/content.{diary|blog}/{settings|entries}`:
- `GET settings`:有効状態。`POST settings`:`{enabled: bool}`。
GETはprefixも返す。POSTは任意の`prefix`を受け、有効状態・prefix・journalを
同一transactionで保存する。prefix省略時は既存設定を維持する。
- `GET plan?date=YYYY-MM-DD&keyword=...`:認証・有効状態・対象Pathのwrite権限を
確認し、未使用Pathとeditor_hrefを返す。既存Pathは409。POSTは不可。
- `GET entries`:現在読める記事のメタデータのみ。
- `POST entries`:`path,title,date,category,version`(新規は0)。更新/解除は
`page_uuid`と現在のversionを付け、解除には`remove:true`を指定する。
- 未定義メソッド・未知フィールド・複数JSON値は拒否。本文上限8 KiB。
- UIは20件単位、最近の記事は8件。索引取得は本文を読まない2秒のmetadata query、
最大5,000 binding行/4 MiB。超過時は黙って結果を欠落させず503。
設定・更新transactionは3秒。UIの編集権限評価は表示中の最大20件だけ。
- 初期版はpermission-filtered keyset pagingではない。大規模サイト向けには上限を
単純に広げず、バッチ権限評価・索引分割・不透明cursorを先に実装する。
- ブラウザ側に記事や権限を永続キャッシュしない。BFCache復帰時に再読込。
設定パネルは表示された時だけ非同期で読む。fetchは15秒で打ち切る。
## 次の実装/TODO
- [x] Page+初回Revision+記事情報を原子的に作成するMarkdown投稿フォーム。
既存の本文保存・asset固定とjournal、冪等再送を実装。
- [ ] 投稿journalから共通outboxへの配送、City自動生成・通知、receipt保持期間とGC。
- [x] 同一Campus・同一インスタンスの参照型一覧/カレンダー射影と双方の読取権限検査。
- [x] 承認済みPath/Revision由来の匿名射影一覧・カレンダー。
- [x] 専用ドメイン内の記事本文、登録タイトル、同月/前後ナビゲーション、RSS 2.0。
- [ ] タイトル以外の登録メタデータの匿名射影承認、別インスタンスの
参照設定配信と認証付き取得・失効、クロスDomain SSO。
- [x] Domain入口設定ダイアログと署名付きmanifest v2による設定配信・原子的反映。
- [x] 署名付きcapability申告・全承認済みノードのv2事前確認・UI案内。
- [ ] Domain新規作成フォームとの一体化、設定の管理解除・GC、配信反映ACK/鮮度管理。
- [ ] 残りの記事メタデータの公開承認・Atom。承認ポリシー/Inboxと接続し、
Pageや上位階層の公開範囲を広げない。タイトル変更・公開撤回で即時失効。
- [ ] 本文抜粋、表紙、複数カテゴリ/タグ。
- [ ] 個人/共有コレクション、著者選択、所有者移管/引き継ぎ。
- [ ] コメント基盤と最近のコメント:閲覧制限、モデレーション、削除・通知を共通化。
- [ ] tDiary取り込み:提供サンプルと公式仕様を確認して対応版/形式を固定する。
Ruby/eval/Marshalの任意実行は禁止。既存import session/journalを使い、事前確認、
日付・timezone・旧URL・セクション・添付・コメントの対応、再開・重複防止を実装。
移行著者は出典であり、ローカル認証ユーザーと偽って紐付けない。
- [ ] 共通outbox・保持/export/削除ポリシー・metadata履歴、孤立メタデータのGC。
- [ ] 索引のkeyset pagingと負荷試験、外部拡張の署名・分離runtimeは別途。
検証:登録/更新/解除、同日複数記事、うるう日、write拒否、Domain越境、グループ失効、
rename追従、非公開カテゴリ/件数の漏えい、journal失敗時のrollback、version再利用、
無効化時のデータ保持、厳格JSON、匿名拒否、PC/モバイルでの設定と登録操作。
+244
View File
@@ -0,0 +1,244 @@
# Domain Namespace and Federation
## Decisions
### Paths are Domain-local
A semantic path is identified by the pair:
```text
(domain_uuid, normalized_path)
```
Therefore `/handbook` in the Sales Domain and `/handbook` in the Engineering
Domain are different bindings and do not conflict.
The `paths` table uses a composite primary key:
```sql
CREATE TABLE paths_v2 (
domain_uuid TEXT NOT NULL,
path TEXT NOT NULL,
object_uuid TEXT NOT NULL,
created_at INTEGER NOT NULL,
is_primary INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (domain_uuid, path)
);
```
Existing rows are assigned to the current default Domain during migration.
All path reads and mutations must receive the request Domain UUID explicitly;
silently using a process-wide default is prohibited in request handlers.
### Objects are instance-global
Page Objects are not owned by a Domain. Their UUID is globally unique within
an instance, while a Domain owns bindings from its paths to Object UUIDs.
This preserves the MagicPage property:
```text
sales.example.test/guide ─┐
engineering.example.test/guide ├─> Page UUID 7a…
support.example.test/runbook ─┘
```
Revisions, immutable Documents, tags, and assets remain attached to Object
identity. Authorization may still restrict which Domains may bind or expose
an Object.
If stronger tenant isolation is later required, it belongs in policy and
storage-placement layers rather than by weakening UUID identity.
## Link addressing
Links have two forms:
- Same Domain: a root-relative semantic path such as `/guide`.
- Another Domain or another instance: an absolute URL containing the
destination FQDN, such as `https://engineering.example.test/guide`.
A cross-Domain link must not be stored as a bare `/guide`, because its meaning
would change with the request Host. Internally, a locally managed destination
may additionally retain `domain_uuid` for rename tracking, but its rendered
and exported form is still an absolute URL.
Object addresses (`uuid:`, `rev:`, and `asset:`) are local-instance addresses.
To address another instance they require an origin:
```text
https://engineering.example.test/o/uuid:…
https://engineering.example.test/o/rev:…
```
No request should transparently resolve a remote UUID against the local
SQLite database.
## Local and remote Domains
A configured FQDN does not by itself mean that a Domain is remote. One
PagesWiki instance may serve several local FQDNs. Domain placement therefore
must be explicit:
| placement | Meaning |
| --- | --- |
| `local` | This instance serves the Domain and owns its path namespace. |
| `remote` | The Domain is a catalog entry served by another instance. |
A remote Domain has no writable local path namespace. Its origin is derived
from its owning Campus FQDN and Domain host. Proxying remote content is not
implicit.
Implemented additive Domain fields:
```sql
placement TEXT NOT NULL DEFAULT 'local',
```
The URL scheme is transport policy and is not part of Campus or Domain
identity.
## Host resolution
Campus stores an FQDN without `http://` or `https://`. A Domain host resolves
as follows:
- empty host: the Campus FQDN;
- label without a dot: `label.<campus-fqdn>`;
- host containing a dot: the host is already an FQDN and is used unchanged.
Resolution is exact. An unknown request Host does not fall back to an
arbitrary default Domain. The Domain with an empty host is selected only when
the request Host equals its Campus FQDN.
## Campus authority modes
Campus configuration has an explicit authority mode:
| mode | Responsibility |
| --- | --- |
| `standalone` | All Campus and local Domain configuration is maintained on this instance. |
| `controller` | This instance is authoritative for a Campus topology and publishes desired configuration for managed instances. |
| `managed` | This instance accepts topology configuration from one external Campus controller. Local topology edits are disabled. |
Implemented additive Campus fields:
```sql
authority_mode TEXT NOT NULL DEFAULT 'standalone',
fqdn TEXT NOT NULL
```
The FQDN identifies the Campus in every authority mode. A controller Campus
may contain both local and remote Domain records. A managed instance stores
the controller Campus UUID, FQDN, and Domain UUIDs unchanged.
A Domain owned by a controller Campus also records `instance_fqdn`. This is
the concrete PagesWiki instance selected to serve that Domain. Controller
Domains require this value; standalone Domains default it to the Campus FQDN.
Managed instances treat Domain topology as read-only because those records
are reconciled from the controller.
Controller instances issue random enrollment tokens. Only a SHA-256 token
hash is stored by the controller, and the plaintext token is displayed once.
A joining instance presents the token, its FQDN, and the SHA-256 fingerprint
of its persistent Ed25519 public key.
The enrollment request includes that public key and a signature over the
normalized instance FQDN and enrollment token, so the controller verifies
proof-of-possession before recording the fingerprint.
Registration creates a pending instance. An administrator verifies its
fingerprint and approves it before it becomes available in the Domain
placement selector. The placement stores both the instance FQDN and
fingerprint. Re-registering the same FQDN with a different fingerprint
creates another pending identity and never inherits the old approval.
Pending instances receive Campus metadata but no Domain assignments. Revoking
an instance disables Domains assigned to that exact FQDN and fingerprint.
Enrollment tokens are individually revocable.
An external Campus is initialized with:
```text
pageswiki --add-external-campus CAMPUS_FQDN TOKEN
```
Development environments may explicitly select unencrypted transport with
`--add-external-campus-http CAMPUS_FQDN TOKEN`. HTTP is never selected
implicitly; the standard command always uses HTTPS.
Either enrollment form may be combined with `--no-default-campus` for a
dedicated managed instance. Its bootstrap standalone Campus and Domain remain
available while enrollment or approval is pending, and are removed
transactionally after the controller assigns the first enabled Domain.
`PAGESWIKI_INSTANCE_FQDN` supplies the joining instance FQDN. Enrollment
fetches and applies the controller Campus name, title, default language,
FQDN, and Domain snapshot. These fields are read-only locally. The token is
stored under the data directory with mode `0600`; the instance Ed25519 private
key is also stored with mode `0600`.
The Managed Campus Save action performs a Controller health probe without
making inherited fields editable. The management view displays a yellow-green
lamp after a successful probe and a red lamp after failure. The transport
(`https` or explicitly selected `http`) is retained from enrollment. Delete
detaches the Managed Campus, its Domain projections, Domain settings, and
Path bindings; Page Objects and immutable Revisions remain intact.
## Domain-scoped settings
Campus and Domain topology are instance-level administration. All other
settings use `(domain_uuid, key)` identity. This includes locale, timezone,
theme, home path, editor behavior, navigation, and
maintenance preferences. Legacy settings are assigned to the default Domain;
new Domains start with product defaults.
The displayed Wiki name is not an independent setting: it is always the
resolved Domain title. Recent Objects are selected by Domain Path bindings.
The administration view may inspect another Domain through an explicit
Campus and Domain selector.
## Configuration distribution
Federation is desired-state synchronization, not shared SQLite access.
The preferred flow is:
1. A controller publishes a versioned, signed Campus manifest.
2. A managed instance authenticates with its enrollment token and pulls the
manifest.
3. The instance validates schema version, Campus UUID, allowed hostnames, and
monotonic manifest revision.
4. It applies the manifest transactionally.
5. It reports the applied revision and validation errors.
Pulling avoids requiring inbound administrative access to every managed
instance. A push notification may trigger a pull, but does not contain
authoritative configuration itself.
The manifest controls topology and presentation defaults only. It must not
carry SQLite files, revisions, Documents, local secrets, OIDC client secrets,
Macaroon root keys, or instance-private signing keys.
## Security requirements
- Transport security policy is evaluated separately from the stored FQDN.
- Controller manifests require signature verification and replay protection.
- A managed instance is pinned to one Campus UUID and controller identity.
- Hostname changes require explicit allow-list validation.
- Local emergency override and last-known-good rollback are retained.
- Controller unavailability does not remove the last applied configuration.
- Remote links are rendered as external navigation, never server-side fetched
merely because a page contains the link.
## Migration order
1. [x] Add placement and authority metadata. Remote Domains have no local
writable Path namespace, and managed Campus topology is read-only.
2. [x] Rebuild `paths` with `(domain_uuid, path)` identity and assign legacy
rows to the default Domain.
3. [x] Thread Domain UUID through Store APIs, server handlers, recursive
operations, browser/tree queries, and the data CLI.
4. [x] Add cross-Domain absolute-link helpers and tests.
5. [x] Add signed controller manifests and managed-instance reconciliation.
Steps 2 and 3 shipped together. Adding `domain_uuid` while leaving unscoped
queries would have created cross-Domain data disclosure.
+99
View File
@@ -0,0 +1,99 @@
# Event Scheduler
## Purpose
CPU-, memory-, and decoder-heavy work must not execute in Viewer requests.
PagesWiki uses a persistent SQLite event queue for deferred work.
`event_jobs` records:
- event UUID and type
- idempotency key
- JSON payload
- `queued`, `running`, `succeeded`, or `failed` status
- scheduled time and attempt count
- latest error
- creation and update timestamps
The current implementation is a single scheduler loop that may run up to the
configured concurrency in goroutines. It checks foreground request activity
between claims and keeps one reserved background slot so a long-lived editor or
stream cannot starve all jobs. Cross-instance leases are still future work. The
transactional Domain outbox now records Page/Revision creation and Path/Object
mutations. Domain-local consumer watermarks are durable and monotonic;
notification delivery remains future work.
External preview tools run through bounded
commands with timeouts.
The in-process clock, loop, admission control and retry policy now live in
`internal/dregarnuhr`. Application handlers are registered in
`internal/server/event_handlers.go`; SQL ownership remains in `store` through
the server queue adapter. The existing API, JSON, IDs and schema are unchanged.
Admission is serialized within one runner, not across processes. Atomic claims
are not expiring leases, and startup recovery must not run alongside another
active writer. See [Dregarnuhr contracts and pending HA work](ja/dregarnuhr.md).
## Event log retention and trends
`event_logs` remains the recent, drill-down log. Each record is associated with
its event type when the job is known. `event_log_policies` provides per-event
retention, aggregation window, bucket size, and maximum raw-row settings; `*`
is the safe default. `event_log_buckets` stores compact counts and first/last
timestamps by event type, level, and time bucket, so trend charts do not scan
the raw log table. The API exposes `/event/logs/trends` and `/event/policies`.
Retention removes old rows in place and keeps bounded aggregate buckets. A
future partition/rotation implementation may replace the physical table, but
must use an atomic rename-and-recreate procedure; an online `DROP TABLE` is
never used while workers can append logs.
## Preview Events
`preview.generate` uses the immutable Document hash as its idempotency key.
Viewer never generates a thumbnail. A children projection:
1. checks the immutable preview cache;
2. emits the cached thumbnail URL when present;
3. otherwise enqueues the event and immediately renders a format icon.
The browser polls only the lightweight cache URL for visible entries. Once the
job succeeds, it replaces the icon with the cached image without reloading the
Page.
Current generators:
| Input | Generator | Cached form |
| --- | --- | --- |
| PNG/JPEG/GIF | bounded Go image resampler | JPEG |
| PDF | `pdftoppm`, first page | JPEG |
| Video | `ffmpeg`, one frame | JPEG |
| Excalidraw | bounded server SVG renderer | sandboxed SVG |
Individual PDF pages use the same queue. A cache miss returns `202` with
`Retry-After`; the browser retries until the scheduler writes the immutable
`<document-hash>-p<page>.jpg` file. PDF page counts and non-ZIP archive listings
are persisted output of `projection.metadata.generate`, not Viewer work. See
[deferred projection metadata](projection-metadata-cache.md) for cache quotas,
pending/retry responses, authorization, and decoder limits.
Generated files live beside the Document Store in `previews/` and are addressed
by immutable Document hash. Cache responses are immutable for one year.
## Scheduling
- Missing previews are enqueued when an icon projection encounters them.
- A periodic sweep runs every 15 minutes.
- The first sweep runs shortly after startup.
- Configuration → Execution monitor combines resource charts with running jobs
only. Job history & logs contains paginated completed jobs and per-job logs.
Event definitions exposes the manual “Generate missing previews” trigger.
See [Configuration panels](configuration-panels.md) for polling, indexed queue
listing, lazy log retrieval and legacy resource-panel URL compatibility.
Failures remain visible in scheduler statistics and logs. Retry/backoff and
cancellation are persisted in the queue; a later sweep or manual trigger may
enqueue a missing cache again. Preview requests never execute the generator in
the normal Viewer request path. Non-ZIP entry extraction remains a bounded
foreground stream pending its separate cached-entry design; metadata listing
and PDF page counts have migrated to deferred projections.
@@ -0,0 +1,78 @@
# Signed Campus manifest integrity
Implemented bounded single-writer-SQLite correction, 2026-09-14. This is not
controller HA, key rotation, multi-node metadata fencing or a new federation
trust protocol.
## Controller sequence allocation
`BuildSignedCampusManifest` canonicalizes the supplied Domain list by UUID,
rejects duplicate/missing identities and hashes the Campus plus filtered Domains.
A single SQLite `INSERT ... ON CONFLICT DO UPDATE ... RETURNING revision`
allocates the sequence against the current persisted hash:
- First content gets revision 1.
- Identical content retains its revision and timestamp.
- Changed content atomically increments the previous revision.
- Database errors and integer overflow fail, never resetting to revision 1.
The returned revision is signed with the existing Ed25519 identity. Concurrent
calls cannot sign different content with the same revision. Ordering-only changes
to the supplied Domain list do not churn revisions. This orders supplied
snapshots; it does **not** make the caller's earlier Campus/Domain reads a
transactional configuration snapshot. A controller-wide configuration generation
is still needed before claiming DR/HA or preventing an old caller snapshot from
being issued later with a newer revision.
## Managed-instance application
Verify signature, schema, Campus identity and Domain identity/hostname shape
before acquiring a database write reservation. Then, in one transaction:
1. Reserve SQLite's writer with a no-op update to the manifest-state table.
This works even for a first enrollment with no row; competing first pins
serialize instead of both reading an absent checkpoint.
2. Read the pinned fingerprint, applied revision and exact payload. Only
`sql.ErrNoRows` means an unpinned Campus. Other database errors fail closed.
3. Reject changed signing identity, lower revisions and a reused revision with
different payload. An exact duplicate succeeds without reapplying the snapshot.
4. Check authority ownership and apply the Campus/Domain snapshot using the
transaction-owned helper. Update placement/default locale as before and keep
`projectCampusAdminMembershipsTx` in this same transaction.
5. Persist the verified public key/fingerprint, revision and payload checkpoint;
commit all snapshot, inherited-permission and checkpoint changes together.
A checkpoint/projection/domain-write failure rolls everything back, including a
new trust pin. Retrying the same manifest after a rolled-back failure is safe.
No failed/replayed manifest silently retries with relaxed verification.
## Authority and legacy compatibility
A valid signature establishes a signing identity, not authority over every
local UUID. A managed snapshot cannot replace a standalone/controller Campus
or reparent a Domain UUID owned by another Campus. A new managed Campus and
updates to its own Domains remain supported, including initial pending enrollment
followed by an approved manifest that deploys Domains to the matching instance.
The legacy unsigned snapshot entry point still permits enrollment/resynchronizing
an **unpinned** managed Campus. It acquires the same writer reservation before
checking the pin. Once a signed checkpoint exists, unsigned fallback is rejected;
omitting a manifest can no longer bypass signing-identity/replay checks. Explicit
trust reset or key rotation needs a separate administrator-controlled workflow.
Initial enrollment remains trust-on-first-use through the configured controller
and enrollment transport; signing does not make an HTTP bootstrap resistant to
an active network attacker.
## Verification and remaining work
Tests cover 20 concurrent same/changed-content sequence allocations, canonical
ordering and overflow, 24 concurrent duplicate/out-of-order deliveries, competing
first keys/payloads, injected initial/updated-checkpoint and inherited-projection
failures, missing state tables, a one-connection pool, cross-authority collisions,
legacy-to-signed upgrade, approved placement/locale inheritance and unsigned
downgrade rejection. Existing managed enrollment/reconciliation tests remain in
the suite.
There are no new tables, shared keys or schema migrations. Key rotation, explicit
global authority transfer, independently versioned instance projections, bounded
configuration capture and DR/HA are follow-ups rather than implied guarantees.
+536
View File
@@ -0,0 +1,536 @@
# September 2026 execution ledger
User direction: continue the remaining roadmap in priority order; defer extremely
complex/high-risk changes with explicit prerequisites. Preserve existing identity
and deployments. This ledger separates completed slices from unresolved gates.
## Active batch
### Japanese design consolidation and Wiki publication (2026-09-19)
- Added a 22-chapter Japanese handbook plus index, separating current behavior,
proposals, unresolved choices and historical claims. Original source documents
remain unchanged in identity/location; a source inventory maps them to chapters.
- Expanded Organizer/tasks/tickets, calendar/contacts/DAV, RSS/FreshRSS/newsboat,
IMAP capture and incremental HTML editing proposals with identities, authority,
conflict/retention behavior, delivery slices and acceptance fixtures. No new
external integration or connector is enabled by this design work.
- Reviewed primary RFC and FreshRSS/newsboat documentation; runtime/client
compatibility remains a prerequisite, not a claimed implementation.
- `tools/design-export` renders with the application Markdown renderer, validates
internal chapter links and emits source/payload hashes. Tests cover tables,
chapter count, bad links and staging overwrite refusal.
- Uploaded 23 new `ja` Page revisions through the trusted local data CLI to
`https://pages.anya.jp/lab/pageswiki-dev/`; destination was empty beforehand.
Read-back comparison and manifest/DB hash checks passed for all 23 payloads.
Receipt: `publication-ja-2026-09-19.json`. Existing English sources are retained.
- Anonymous HTTPS access still redirects to login. No publication/authorization
settings, external credentials, application runtime or deployment were changed.
`go test ./...` and diff checks passed.
### Scoped automatic publication and compact blog index (2026-09-19)
- Added explicitly confirmed Street/Spot automatic publication, live revision and
publisher-authority checks, inherited manual-approval boundaries, ACL/alias
exclusions, and versioned impact previews. Existing settings stay unchanged;
no production locality is opted in by this deployment. See
[automatic publication](automatic-publication.md) for limits and remaining work.
- Blog/diary and projected public indexes use compact date/title tables. The month
selector sits above the sidebar calendar; authenticated row actions remain
accessible through labelled icons. Existing post editing and public article
navigation are preserved.
- Full Go tests, vet and diff checks passed. Opt-in Chromium coverage passed for
automatic publication, City governance, periodicals and public projection.
- Follow-up: month input, apply and reset form a single compact row; SVG buttons
have localized tooltips/accessibility labels. Desktop/mobile browser tests
enforce single-line controls and placement above the calendar.
- Pre-migration online backups of both databases passed quick checks under
`/home/anya/.cache/pageswiki-backups/pre-auto-publication-kv0VzP/`.
- Deployed app2 then app with image
`sha256:8ca7973bb12126ba2291dc9e6e1c9a4c5faad77ff92b56cf4f79ee69e9620523`.
Both instances responded successfully with zero restarts; app is healthy.
Public HTTPS serves the compact index and three RSS embedded article bodies;
both databases still contain zero automatic-publication rules. Production mobile
rendering confirms the one-line month controls. Anonymous source editing still
redirects to login.
- Future tasks/tickets, organizer/DAV, mail ingestion and HTML editing ideas are
recorded in [design backlog](organizer-and-editing-ideas.md), not implemented in
this batch. Full WYSIWYG remains an explicit undecided design choice.
### Full-content RSS, standalone composition and X sharing (2026-09-19)
- RSS `content:encoded` uses the inert HTML pipeline, namespace-aware XML/CDATA,
absolute public article/image/anchor URLs and exact live Revision rechecks.
Limits: 20 articles, 1MiB input/article, 8MiB total HTML, 5s generation; HEAD skips
body work. RSS-reader copies cannot be remotely recalled after unpublishing.
- Added authenticated GET-only `/v/blog/-/new` and `/v/diary/-/new`, themed responsive
forms backed by the existing atomic posts API. Projection targets redirect to the
configured source composer. No GET creates content; disabled plugins return 404.
- Post completion offers X Web Intent with title and registered projection-domain
URL; multiple eligible Domains are selectable. Invalid/disabled/cross-Campus
projections are excluded, and pending publication is explicitly warned/confirmed.
No external posting occurs without user action; no OAuth secrets or new grants.
- Added namespace/CDATA, absolute link, unsafe HTML, composition route and share-
domain/prefix regressions; Chromium covers standalone posting and sharing controls
at desktop/mobile widths without navigating to X or creating production content.
- Full Go tests, vet, JS syntax, diff check and opt-in Chromium suites passed.
Deployed app2 then app with image
`sha256:f51dadce7481b480bdd924ad65d050e6d9570a37081c91a7cd5562394d9be4bc`.
HTTPS RSS returned 200 with three embedded public HTML bodies and the content
namespace; article returned 200 with target-domain OGP URL; both new-entry routes
redirected anonymous requests to login (303). app healthy, both containers running
with zero restarts. No production article creation or X posting was performed.
### Dedicated public blog articles and RSS (2026-09-19)
- Dedicated diary/blog Domains now project approved source article bodies at
`/YYYY/MM/DD[/keyword]`. Header/browser title use target Domain branding; article
pages keep month calendar, other entries that month and older/newer navigation.
Known public article links in sanitized bodies are remapped to the target host.
- Registered titles explicitly inherit the Page's live publication status. This
changes the earlier h1-only index contract at the user's request. Private category,
author and registration date are still omitted; dates derive from approved Paths.
No registration table writes, grant changes or migration are needed.
- `/rss.xml` emits RSS 2.0 with up to 20 public titles, target-domain links/GUIDs and
Revision timestamps. HTML advertises the feed. No private metadata or body copies
in RSS; no shared HTTP caching. Subscriber-retained copies cannot be revoked.
- Full Go tests, vet, diff check and Chromium tests at 1440/390 px passed. Tests cover
keyword routes, domain title, escaped registration titles, sanitized body, previous/
next links, feed XML, private entries, edited-revision revocation and both auth modes.
- Remaining: Atom, excerpts/covers, publication approval of other metadata, cross-
instance projection and SSO. Non-public embedded assets do not gain public access.
- Deployed app2 then app with
`sha256:2a9743aadf687dedb7465b6e9c22f313818aa59fe341a94c5c6ec82c139386cf`.
Anonymous HTTPS index, `/2026/09/17` and `/rss.xml` all returned 200 (one published
entry); parent editor still returned login 303. Confirmed registered Japanese title,
sanitized body, blog Domain header and calendar in
`/tmp/pageswiki-blog-article-review-0919.png`. app healthy; publication count remains
one and enforcing config unchanged. No production content/config writes were made
by this task; current Object count is 1456 (live instance continues background work).
### Public diary/blog projection index (2026-09-17)
- Root cause: configured projection views always required a session, even with
approved publications in the source prefix. Added anonymous root/month/day views
in enforcing and legacy target modes, using only the source's live publication
manifest. Source authorization mode must remain enforcing.
- Dates derive from approved Paths; headings from approved HTML, bounded to 64KiB
for at most 20 displayed and 8 recent entries. No private article metadata,
categories, authors, editor dialogs or unrelated navigation is exposed.
- No grants, auth modes, schema or content changed. Body links remain on the source
host. Source disablement, invalid credentials and publication revocation fail
closed. Registration metadata approval and same-host body projection remain TODO.
- Validation: full `go test ./...`, `go vet ./...`, JS syntax, diff check and existing
Chromium periodical flow passed. New tests cover both target modes, private
registration metadata, unapproved Paths, invalid credentials and edited revisions.
- app2 then app deployed with image
`sha256:36f5fba8a86db9ed6ea94872400bc86745c6499e3c09b49c11896303a953d86f`.
Anonymous HTTPS root/day views on blog.pages.anya.jp returned 200 with exactly one
approved article; source article returned 200, parent editor and invalid credentials
returned login 303. Browser screenshot checked at
`/tmp/pageswiki-public-blog-review.png`. app healthy, both running without restarts.
Object/publication counts remained 1455/1; authorization configuration unchanged.
### Domain entry dialog CSS correction (2026-09-17)
- Root cause: the modal was nested in a dense administration table cell. Broad
descendant selectors made forms inline-flex, help paragraphs inline, and controls
inherit compact table sizing. Previously the browser test checked `open` but not
actual geometry (a hidden panel ancestor could still conceal the dialog).
- Moved modals outside tables and panels into the body. Added isolated responsive
fields, block help text, themed controls, toggle labels, sticky header/close button,
background scroll lock, accessible dialog title and CSS cache-version update.
- New Chromium regression checks at 1440/390 px with Light/Nanoha Dark: visible
viewport bounds, no horizontal overflow or overlapping controls, form/help display,
sticky close button and close/reopen. Screenshots inspected under
`/tmp/pageswiki-domain-entry-review/`. Existing periodical browser flow, full Go
tests, vet and diff check passed.
- UI-only release; no schema/content/permission changes or database restore.
app2 then app updated to
`sha256:d2cb4a02e780790294245e60a394e6046be1f3160bc9e4186c0daf7f9a8a4369`.
Both health endpoints responded, zero restarts; served config.css matches source.
### Local diary/blog projection deployment (2026-09-17)
- First slice: same-Campus, local-placement metadata projection, source-prefix
filtering and source+destination read policies; no copied content. Article/body/edit
links open the source Domain. No anonymous, cross-instance or host-preserving body
projection yet. Local projection settings are not in federation manifests.
- Backup directory `/home/anya/.cache/pageswiki-backups/pre-periodical-projection-aw54pH/`:
app.sqlite and app2.sqlite, private permissions, both quick_check OK.
- app2 then app deployed with image
`sha256:e44fbb1a55f7eef4a4c4fec08a87efa8bc7e592f222c99090930c95590d435cf`.
Main DB initialization briefly left the listener unavailable during restart;
final app health=healthy, app2 health endpoint OK, zero restarts, public HTTPS=200.
Running template contains the domain-projection settings forms.
- Full tests, vet, diff check, focused race and desktop/mobile Chromium passed.
Fixture tests cover no-copy behavior, source ACL denial/revocation, prefix filter,
source links, target write rejection, self/cycle/cross-Campus rejection and remote
source fail-closed behavior.
- Production Objects=1454, publication entries=4, matching pre-update counts.
No production projection bindings were created automatically; user chooses source.
### Diary/blog and Domain entry deployment (2026-09-17)
- User explicitly authorized backup and deployment. Online SQLite backups:
`/home/anya/.cache/pageswiki-backups/pre-domain-entry-6a73yw/app.sqlite`
and `app2.sqlite`, directory private / files 0600; both `PRAGMA quick_check=ok`.
- Deployed accumulated date-path navigation, atomic article composer, Domain
entry-settings gear/dialog and signed manifest v2 to app2 first, then app.
- Continued implementation: signed manifest capability proof in enrollment;
unknown/legacy approved peers block entry-setting updates with 409 and UI guidance.
The check covers all approved Campus peers because they share a Campus manifest.
- Final running image on both instances:
`sha256:0dbcb4aacbc924855f0b686a859bd10c7932eea9d1736fc9f4be9a6337b694ae`.
app healthy, app2 running, zero restarts; Host-routed health checks both OK.
`https://pages.anya.jp/` returned 200. Served periodicals.js hash matches source.
Running template includes `data-domain-entry-open`; authenticated gear/modal and
posting interactions were exercised in isolated Chromium fixtures, not by creating
production administrator sessions.
- Actual approved ext-node enrollment reports manifest_version=2. Old pending
fingerprint remains pending; no approval or identity changes were made.
- Main DB before/after: 1454 Objects and 4 publication entries. No entry settings,
article content, user roles or publication grants were changed during deployment.
- Tests: full `go test ./...`, vet, diff check, browser fixtures at desktop/mobile,
capability proof validation, legacy downgrade detection, incompatible-peer
rejection, signed manifest integrity/rollback and focused race tests passed.
- Remaining: anonymous metadata/calendar publication approval, Domain-create form
integration, entry-control release/GC, apply ACK/freshness and journal/outbox delivery.
- [x] Diary/blog initial built-ins — 2026-09-16.
`content.diary` and `content.blog` provide Domain-scoped, default-disabled
settings, Page UUID metadata, calendar/recent entries, category/month/mine
filters, and register/edit/unregister dialogs. Canonical content and revisions
stay in ordinary Pages. Metadata writes reserve SQLite's writer before live
membership, Path binding, grant/Locality and version checks; own-user Wiki
authority matches the semantic editor. Metadata journal commits atomically;
unregister/re-register generations prevent ABA replay. Private metadata is
not exposed anonymously; Page publication behavior is unchanged. Atomic
new-post composition, public metadata approval/feeds, comments and tDiary
import remain [explicit follow-ups](diary-blog-plugins.md).
Full Go suite PASS (Server 12.417 s), focused race PASS (Server 5.745 s,
Store 1.790 s), vet/JS syntax/diff checks PASS; combined config, public
navigation, governance and diary/blog Chromium PASS (9.332 s). PC/mobile
screenshots inspected under `/tmp/pageswiki-periodicals-review/`.
DB-only backups at
`/home/anya/.cache/pageswiki-backups/pre-periodicals-drFK6Ljf/` are mode 0600;
both quick_check results are `ok`. Backups use low-priority I/O.
Rebuilt image:
`sha256:9948aca99fa03eb43dda06a03ae59a680cad3d5b5284fd6c2217e09d69f75c22`.
Deployed to both nodes: local Host-routed health checks return `ok`, zero
restarts, served JS matches source. Both have all three additive tables and
zero enabled instances. Main anonymous root remains 200; diary/blog require
login (303) and unauthenticated metadata APIs return 401. Managed legacy-mode
diary also requires login. Existing main objects (1,448), publication
manifest (one) and approved entries (four) remain unchanged; no plugin was
enabled and no article was registered in production during verification.
- [x] Public children view controls now match the authenticated icon order and
sizing (`▦`, `☷`, `≡`), with localized tooltips/accessible labels and retained
active state. Desktop/mobile Chromium PASS (1.948 s), focused navigation tests
PASS (0.454 s), JS syntax and diff checks PASS. Rebuilt image:
`sha256:35b1fd0e0c291e42769b81dd50a53d237b94071431387c41b289d692a578e2b7`.
- [x] Anonymous navigation usability — 2026-09-16 23:19 JST.
Added public root/virtual-parent directory landings and read-only
`/v/browser/{path}`; separate Page/child-folder links, compact responsive
header and direct City exits. Public tree now supports all/per-folder
expand/collapse, name/Path filtering and arrow/Home/End navigation. Component
sorting keeps `/a/x` before `/a-b` for correct branch operations. Children
requests are capped at three concurrent fetches, cancellable and timed out,
with generation-safe retries and view-only session preference. Initial
directory/pagination links and macro fallback links work without JavaScript.
Full Go suite PASS (Server 12.387 s), vet/JS syntax/diff checks PASS;
focused race suite PASS (Server 7.287 s); broad Chromium suite PASS (19.837 s),
including anonymous navigation at 1440 px and 390 px. Screenshots inspected:
`/tmp/pageswiki-public-navigation-review/public-navigation-{1440,390}.png`.
Tests cover failure/retry, view switching, hierarchy prefix collisions,
pagination/exclusions, private-parent body suppression, bad credentials,
mutation denial and warm-cache revocation. See [contract](public-navigation.md).
Both nodes rebuilt/deployed with
`sha256:cf75c402c04cb185e1c7965480c35b352e2182c857af9367e7b3dfa672a68597`.
Both local Host-routed health endpoints return `ok`, zero restarts; managed
home returns 200. Main anonymous root/browser/tree/children/City return 200
(approximately 1–21 ms in local checks); protected autoplan remains 401.
Public directory headers retain private/no-store and inert CSP. Served JS/CSS
hashes match source on both nodes. No new schema migration; existing main
publication remains one manifest/four entries and its governance, topology,
binding, hash and head checks pass. No approvals or production content were
changed; no public TLS ingress change or verification is claimed.
- [x] Permission-aware public navigation — 2026-09-16 22:44 JST.
Ordinary approved Pages now offer asynchronous children, structural breadcrumbs,
read-only tree and filtered City Map; remove the public download control.
Navigation-free presentation remains a future share-URL concern. Cached data
uses SQLite mutation generations; authenticated membership/ACL decisions are
evaluated per request. City reload/retry discards previous authorized labels
before revalidation. See [navigation contract](public-navigation.md).
Full Go suite PASS (Server 9.335 s), vet, JS syntax and diff checks PASS;
broad Chromium suite PASS (16.670 s); focused navigation/ACL/publication race
tests PASS (Server 7.529 s, Store 14.706 s). Coverage includes warm-cache
publication withdrawal and group removal without data-cache invalidation.
DB-only SQLite backups in
`/home/anya/.cache/pageswiki-backups/pre-public-navigation-FMqnj5pu/` are 0600;
both quick_check results are `ok`.
Both nodes rebuilt/deployed with
`sha256:aade0a87dbdd0966c92908836dda57f5d5a3440028cb2181194104c8ede9f9ae`.
Host-routed local health checks return `ok`, both running with zero restarts;
managed home returns 200. Main anonymous tree/children/City HTML/map API return
200 (approximately 1–18 ms in the local smoke check), while autoplan remains
401. Tree response is private/no-store with inert CSP; main served JS hashes
match source. Additive navigation schema has 45 invalidation triggers on each
node. Existing main publication records remain one manifest/four entries;
managed records remain zero. No publication was activated or revoked by this
deployment. Public TLS ingress was not changed or certified by local checks.
- [x] Document hierarchical approval requirements and approver Inbox delivery.
[Approval design](publication-approval-inbox.md) defines inherited required
floors, immutable review subjects, live reviewer authorization, transactional
outbox/retry and recipient-private notifications. AP-1 through AP-6 remain
unchecked implementation work; this design does not enable a new workflow.
- [x] Named user/group locality read or read/write permissions, stable subject
identity resolution, superior-cap/authority-rank restriction preservation,
next-request membership checks, Object alias caps and SQLite namespace fences.
Explicit bounded anonymous current-revision approval/withdrawal now serves a
separate inert semantic Viewer and streaming raw-media path with safe OGP.
Per-scope publication generations prevent replaying pre-revocation consent.
See [scope, operational limitations and follow-ups](locality-access-publication.md).
Whole Go suite passes (Server 10.176 s, Store 4.560 s), focused race suite
passes (Server 6.206 s, Store 1.013 s), and approval/governance Chromium
checks pass (1.994 s); the broader browser suite passed (27.547 s).
Source-Domain restrictions also clamp exceptional foreign Object grants;
a regression verifies that these exceptions cannot bypass the new ACLs.
Vet, JS syntax and diff checks pass. SQLite backups (DB only) are under
`/home/anya/.cache/pageswiki-backups/pre-locality-access-lpnkGqwV/`, mode 0600;
both pass quick_check. Both existing nodes deployed as
`sha256:4a797ee34ef0c0fe0d48cea891a943393f0f722f07c192879f61b5dc3456ba9d`.
Host-routed local health checks return `ok`, both running with zero restarts.
Served governance JS hashes match the workspace on both listeners. All five
new access/publication tables exist and remain empty on both DBs: no production
restriction or public approval was created. Main anonymous home remains 303,
managed legacy home 200; anonymous access/publication administration requests
are denied (main 401, managed 403). Managed public TLS/vhost remains the
previously recorded infrastructure caveat; these are local listener checks.
- [x] Quartier/Spot co-owner appointments, Domain/local publication-cap drafts,
epoch/topology concurrency checks and in-transaction membership/authority
revalidation; local before/after audit and Japanese/English modal settings.
Includes bounded appointment-authorized member lookup. Full impact reports,
structural mutation fences and anonymous publication remain pending; existing
production visibility is unchanged. See [governance contract](locality-publication-ownership.md).
Verification: full Go suite (Server 8.584 s, Store 4.015 s), focused race
tests (Server 2.698 s, Store 1.015 s), combined City/governance/search/HTML
Chromium regressions (16.275 s), vet, JS syntax and diff checks pass.
Pre-migration SQLite backups (DB only, not a complete DR package) are under
`/home/anya/.cache/pageswiki-backups/pre-governance-aNwngqHe/`, mode 0600.
Both backups pass `PRAGMA quick_check`. Both existing app nodes deployed as
`sha256:dc6960ba1b01800b08351f660d20848dcca0e0f7f382fe18e362d93f7171d147`.
Local Host-routed health checks return `ok`, zero restarts; all four additive
governance tables exist on both DBs, with no owners/policies automatically
inserted. Served governance JS hashes match the workspace on both nodes.
Anonymous governance requests are denied (main 401, managed 403); main home
still redirects to login (303). No production owner appointment or public
policy change was made as a smoke test. Existing managed public TLS caveat
remains; these are local listener/Host checks, not a TLS infrastructure fix.
- [x] P0 inert HTML display boundary across semantic/home/profile/object/history/
preview/print and HTML/Markdown Editor previews; canonical content unchanged.
Application nonce CSP, forged macro-hook stripping, escaped failure fallback,
bounded/cancellable City macro reads, and ACL-checked sandboxed diagram cache.
See [precise policy and compatibility](html-projection-trust.md). Owner/public
APIs and new anonymous access remain disabled/pending. Final full Go suite
passes (Server 7.866 s), focused race tests pass (Server 4.247 s, Store
1.057 s), and final HTML/PDF/archive/editor/bookmark Chromium tests pass
(14.006 s). Broader City/search/login Chromium checks also passed (27.590 s).
`go vet`, editor JS syntax and diff checks pass. Read-only compatibility
sampling was limited to 500 HTML revisions; it was not a complete Path audit.
Deployed both existing nodes as
`sha256:41cd97c7705b8be6b1bf763aee7678c5d510878d5ec6d78aa3b3792a7eb32517`.
Both local Host-routed `/health` endpoints return `ok`, zero restarts; managed
home returns 200 with nonce CSP, nosniff, private/no-store and
`X-PagesWiki-HTML-Policy: inert-v1`. Main anonymous home still redirects to
login (303). Both served Editor scripts match workspace SHA-256. No schema
migration, production document rewrite, visibility change or TLS change.
Managed public TLS/vhost caveat from the earlier social-card check remains
outside this deployment; these checks do not claim public HTTPS there works.
- [x] OGP/X cards for existing public semantic Page/home/City HTML, bounded
canonical text extraction, configured-host canonical/image URLs, original
image-document cards and an application-owned cached PNG fallback. No crawler
auth bypass, external image fetch, schema migration or visibility change.
Private/enforcing HTML suppresses metadata and adds noindex. See
[social card policy](social-link-cards.md). Full Go suite (Server 6.818 s),
social race tests (2.434 s), City/search Chromium checks (3.846 s), vet and
diff checks pass. No claim of actual X post/card display; no post was sent.
Deployed both nodes as
`sha256:a16fa6cbe8a3a6b5141d7d29de0e6faef67989285a9983b2ffa306d4e016b75c`;
both health endpoints return `ok`, zero restarts. Managed-host anonymous home
includes HTTPS OGP/card URLs, image HEAD returns 200/image/png (7,327 bytes),
main `pages.anya.jp` still redirects anonymous crawlers (303). The common
image is also reachable through public HTTPS; its 1200×630 output was visually
inspected. No private production Page was made public for this test.
External caveat: `https://wiki.extpages.anya.jp/` and its image fail TLS with
`unrecognized name`; managed-host checks above used its local app listener
with the configured Host. Public HTTPS image success was on `pages.anya.jp`.
Fix the managed FQDN's TLS/vhost before expecting external cards there.
- [x] City named-reference follow-up: qualified Street URLs, shared links,
explicit UUID locality search, ancestor-aware lightweight details and literal
Page links in both the map and inspector. No schema/data migration required.
Whole-map loading is also deferred out of simple City breadcrumb macros.
Verification: full Go suite, City/search browser regressions (including
repeated Street names and three initial-focus levels), `go vet` and JS syntax
checks pass. City/search race tests also pass (Store 35.000 s, Server
7.391 s); the combined Chromium suite passes in 13.166 s. The HTML trust gate
and broader CITY-A–D work remain open.
Deployment: both existing app nodes rebuilt/recreated as
`sha256:8e53fec5c485ed5763bf44705cfdf4071d04f2b8932dca038148eedda26f4c42`.
Both health endpoints return `ok`, zero restarts; main unauthenticated City
request redirects (303), managed-host search returns 200 under its existing
policy. Served City map/estate scripts match workspace SHA-256 hashes.
- [x] OIDC issuer JWKS signatures, claim validation, bounded discovery/key cache;
one-use browser state and collision-safe external account linking.
- [x] Bounded SMTP delivery, safe headers/mailboxes, live account checks.
- [x] Campus-admin inheritance revocation, provenance and local-right preservation.
- [x] Recursive Path mutations: transactional authorized snapshots and literal prefixes.
- [x] Browser mutation CSRF protection and explicit credential precedence/logout;
browser/API purpose prefixes and atomic active-session checks.
- [x] Object authority scope and atomic creation provenance; safe all-alias purge.
- [x] Signed manifest allocation/application/checkpoint atomicity and authority guards.
- [x] City map/detail/Page list read filtering and private response caching policy.
- [x] Batch/request-scoped read authorization to reduce listing query amplification.
- [x] PDF/archive metadata scheduler projections with persistent bounded cache.
- [x] City generator Worker and directional keyboard navigation.
- [x] Search results: compact overview, full Page Path, matching-text excerpts and
safe keyword highlighting; retain list/detail/thumbnail integration and read ACLs.
- [x] Object/Revision metadata pagination and localized history navigation.
- [x] Literal URL encoding in children list/detail/icon projections.
- [x] City one-connection-safe read snapshot and literal boundary lookup/health.
- [x] Atomic bounded zombie GC and Domain-provenance-scoped HTTP administration.
- [x] Integrated Go/browser tests, rebuild, and both instance health checks.
Update this list only after the corresponding implementation and verification.
The broad phase checkboxes stay open when only a bounded slice has landed.
Focused implementations, constraints and regressions are linked from
[Implementation status](implementation-status.md#confirmed-corrections-2026-09-14).
## Verification and deployment
Completed 2026-09-14 01:43 JST. All source changes were frozen before rebuilding;
later edits only recorded the final outcomes in design documents.
- Final `GOMAXPROCS=2 GOCACHE=/tmp/pageswiki-go-cache go test ./... -count=1`:
PASS for all packages (server 5.947 s, store 3.105 s).
- Targeted server/store `-race` covering authentication, inheritance,
recursive/Object integrity, federation, GC, City and metadata projections:
PASS (server 21.774 s, store 38.202 s). Independent focused race runs also
covered search, pagination and the final deterministic GC regression fixture.
- Opt-in Chromium: 13 suites PASS (48.544 s), covering real HttpOnly-cookie
login/logout, City normal/popup/mobile and Worker/navigation, search in all
three modes, PDF/archive/comic metadata, settings, execution monitor,
bookmark/Vim settings UI and editor/home. This does not validate the paused
vim.wasm runtime. Artifacts: `/tmp/pageswiki-hardening-final`.
- `go vet ./...` and `git diff --check`: PASS.
- Before schema migration, online SQLite backups of both nodes completed with
`PRAGMA quick_check` = `ok`. Directory:
`/home/anya/.cache/pageswiki-backups/pre-hardening-20260914-1wH8jcKm/`.
`app.sqlite` and `app2.sqlite` are mode `0600`; their containing directory is
private. These are DB snapshots, not complete Document-store/key DR backups.
- `docker compose build app` and `docker compose up -d --no-deps app app2`:
PASS. Both containers run image
`sha256:dc0ac3305c6a6abb42d84e56c8291cabc9c582a421f4ce91ca4276b123735ad2`,
with no restart loop. Both local `/health` endpoints return `ok`.
- Main-instance search CSS and City Worker client hashes match the working
files; managed-instance search CSS matches too when using its configured Host.
`https://pages.anya.jp/` and the protected main search route return the
expected unauthenticated `303`; the managed Domain search route returns
`200` under its existing read configuration. A foreign-Origin unsafe request
is rejected with `403`. No production account credentials were used.
Browser behavior tests used isolated fixture databases, not production content.
Live external OIDC/SMTP provider credentials and production multi-user flows
were not exercised; protocol, denial and concurrency coverage used test fixtures.
### Search timeout follow-up — 2026-09-14 03:47 JST
The user reported a search timeout after the initial deployment. Read-only
production-data investigation found a Path-first repeated FTS scan and
unbounded snippet input work, neither represented by the original small fixtures.
Corrected the join order, rowid excerpt lookup, per-document/total snippet input
budgets and optional 400 ms enrichment deadline. Results survive snippet-only
timeouts; genuine search failures now have localized, non-leaking retry UI.
See [evidence and remaining search limits](search-results.md#timeout-regression-and-correction-2026-09-14).
- Full Go suite PASS (server 6.869 s, store 3.511 s); search `-race` PASS
(server 2.967 s, store 8.063 s); Chromium search PASS (0.740 s);
`go vet ./...` and `git diff --check` PASS.
- Added query-plan assertions, a 600-Page/1,200-Revision regression, Unicode
byte-budget/alias tests, deadline/error tests and large-body UI coverage.
- Read-only diagnostics against the existing main DB passed twice, with
representative search processing times around 72–506 ms. No production
account credentials or Page content were output or changed.
- Rebuilt/deployed both app nodes with image
`sha256:0f10da866a03167da7811ddc88f66d6a65bd27aae17557c824a9f7b5071b88a8`.
Both `/health` endpoints return `ok`, no restart loop, search CSS matches
source on both configured hosts. No schema/index migration was needed.
### City-first search and filters — 2026-09-14 04:05 JST
Added permission-filtered, metadata-only Quartier/Street/Spot results above
Pages. Added a compact options disclosure for MIME/suffix categories, a single
extension, and creation/update calendar-date ranges. Options persist in the
GET URL and retry link. File-specific filters exclude City metadata; date
filters apply to both. City/Page queries share the existing deadline and do
not load the full City graph. See [search contracts](search-results.md#city-first-results-and-search-options).
- Full Go suite PASS (server 6.746 s, store 3.470 s); combined search/City
`-race` PASS (server 4.288 s, store 10.257 s). Added filtered-query plan
coverage also passed in the final Store search race run (18.373 s).
- Chromium desktop/mobile (1440/390 px) PASS, including real City results,
safe links, match highlights, preserved filters, all Page view modes and
separation from file multi-selection. Final run: 1.173 s. Screenshots:
`/tmp/pageswiki-search-city-final/search-results-desktop.png` and
`search-results-mobile.png`. `go vet ./...` and `git diff --check` PASS.
- Read-only main-DB verification: representative keyword search 79–515 ms,
filter-only cases 1–6 ms, City metadata 2 ms. These are Store timings, not
authenticated production HTTP measurements. No schema migration is needed.
- Both app nodes rebuilt/deployed with image
`sha256:1f6d64c354e940c0941db9914a836f47e7285ac962a9c68f9b6cebcddd2e0dc3`.
Both `/health` endpoints return `ok`, zero restarts, source CSS hashes match
each configured host. Protected public search returns the expected `303`
without credentials; managed-node filter-only search returns `200` under
its existing read configuration.
## Deliberate high-risk deferrals
- Executable authored-HTML compatibility is not supported. Normal HTML now uses
the inert projection boundary; any future executable mode requires separate
origin/capability approval and isolation tests. Public locality ACLs and owner
governance still require transaction/cache revocation gates. See
[policy and acceptance tests](html-projection-trust.md).
- HA/LB writable metadata, controller DR promotion and distributed job leases:
need a selected replication strategy, fencing, key custody and fault drills.
- Untrusted third-party code/plugin execution and native application launch:
need isolated runtime, signature/capability lifecycle and a separate helper
trust model; existing hosted editor integrations remain supported.
- Production S3/SSHFS/WebDAV/MWDB/Karton/SNMP/LDAP integrations: implement only
after endpoint/credential policy and test services are available. No fabricated
live integration claims or unrequested credential/infra provisioning.
- Broad notifications/inbox, comments, diary/tDiary import, rankings and external
search: retain the existing detailed contracts; gate on transactional activity
provenance, privacy/retention and authorization. Small prerequisites may land
first, without claiming those product features complete.
- vim.wasm runtime remains explicitly on hold from the earlier user instruction.
These are not removed from the roadmap. Lower-risk outstanding items that do not
fit this batch remain unchecked, with the next sequence recorded at handoff.
## Next implementation sequence
1. Complete locality Page/alias/dependency impact reports and structural mutation
fencing on the implemented owner/draft-policy foundation. Require fresh
activation consent and endpoint/cache revocation coverage before anonymous
access; never silently activate saved drafts.
2. Complete mutation authority/version fencing and local/external account
ownership lifecycle, then remaining endpoint/adapter denial audits.
3. Add shallow, paginated City projections and transactional mutation outbox /
reconciliation provenance; retain manual curation when deriving from URLs.
4. Replace remaining full directory aggregates and historical/raw-HTML FTS work
with current-only bounded projections; add cancellation-aware streaming or
immutable-hash clone operations for large binary changes.
5. Resume activity/notification and trusted system-extension product slices
after those authority and event-provenance prerequisites. Keep HA/DR and
external-service provisioning behind their separately documented decisions.
+146
View File
@@ -0,0 +1,146 @@
# Home navigation and hosted Vim follow-up
## Application assets and editor routes
Templates load CSS and JavaScript from `/static/`. The legacy
`/o/path:/system/static/` namespace remains a read-only compatibility alias to
the confined static file server. It is not a UUID/Object address and must not
be sent through document authorization. Other `/o/` addresses remain protected.
`/e/docs` and `/e/-/docs` both edit `/docs`; `/e/`, `/e/-`, and `/e/-/`
edit `/`. The authorization target and editor use the same path decoder.
Database resolution errors must not be mistaken for a missing page.
## Bookmark sections
The Domain-scoped `home_bookmarks` JSON array stays backward compatible:
```json
[
{"kind":"section","label":"Reference","href":"","children":false},
{"kind":"url","label":"Example","href":"https://example.com/"},
{"kind":"section","label":"Documents","href":"/docs","children":true}
]
```
A section starts a card and owns subsequent links until the next section.
Legacy links before any section form a default card. `children=true` replaces
that section's displayed links with the immediate children of its parent Path;
manually configured links remain stored for when the toggle is turned off.
Virtual folders are included, but `/u`, `/_`, and `/-` are excluded at the root.
Management is available through a small settings icon on every home layout to
Domain/Campus admins. A native modal dialog provides focus containment, Escape
and close-button dismissal, while retaining unsaved edits if closed. The same
section-card editor is used at `/-/sys/config?panel=bookmarks`. Section moves
and deletion operate on the entire group; links can be moved to another
section using a selector. Page content is never deleted by bookmark deletion.
Saves use the existing server validation and request JSON acknowledgement;
errors remain in the form instead of navigating away. On success the home
reloads its cards. A navigation warning protects unsaved edits. Classic and directory layouts show cards; the
search-only layout retains its search focus while allowing management.
Limits: 64 rows total, including at most 16 section rows. Labels and URLs are
validated server-side; invalid submissions leave previous settings untouched.
Path URLs escape literal percent signs, fragments, spaces and Unicode.
`GET /v/bookmarks?section=<array-index>&after=<last-child-name>` loads children
as JSON. It resolves the section within the active Domain, checks the parent
and each child against current read permissions, and returns at most 80 scanned
children per request with a continuation cursor. No revision bodies, metadata
stats, thumbnails or full-Domain listing are loaded. Queries use the Path
prefix index and request cancellation. Browser loading is lazy and limited to
two concurrent requests. Responses are `no-store`, avoiding stale permission
results. Empty filtered pages may still contain a continuation cursor.
Follow-up: batch authorization and materialized immediate-child indexes for
extremely large subtrees; optimistic concurrency for multiple administrators
editing the same configuration (currently last successful save wins).
## vim.wasm configuration and plugins
The pinned runtime writes injected files using `wx+`: an existing bundled
`/.vim/vimrc` is skipped, despite the package README suggesting overwrite.
Use a new `/pageswiki.vimrc` MEMFS file and explicit `-u /pageswiki.vimrc`.
This is verified inside the real Wasm binary, not merely by HTTP success.
The Gateway image bundles the existing licensed Hack Nerd Font locally.
`document.fonts.load('16px PagesWikiHack')` completes before startup so canvas
cell widths use the final font. `guifont=PagesWikiHack:h16` controls actual Vim
rendering; CSS alone does not. Font, Worker and Wasm retain same-origin isolation.
`assets/vimwasm/app/vimrc` sets line numbers, cursor line, status line, search,
indentation and dark colors. `pageswiki.vim` is a bundled pure-Vimscript plugin
in `/.vim/plugin`, with an explicit runtimepath. Available operations:
- `:Files` / Space+f: `:find` with Tab completion beneath `/work`.
- Space+b: buffer listing and selection.
- `[b` / `]b`: previous/next buffer; `:next` still traverses arguments.
- Space+w: write; Space+/: clear search highlight.
The existing BufWritePost export and scoped Gateway write-back remain in use.
The base settings are image-managed. Domain administrators can customize them
at `/-/sys/config?panel=adapter-vim.wasm`, separately from connection settings.
### Domain customization contract
`POST /-/sys/config/vim` stores one `vim_wasm_settings` JSON config value per
Domain: `enabled`, `font_size` (10–32), `vimrc`, `colorscheme`, `color_name`.
The settings panel uploads UTF-8 files into editable text fields, supports
download, custom-colorscheme removal, and reset to defaults. Each script is
limited to 64 KiB; NUL bytes, invalid UTF-8 and path-bearing filenames are
rejected server-side. One custom colorscheme is managed initially; uploading
another replaces that slot. Nothing is written into the host filesystem.
Only active builtin Domain/Campus administrators can save; readonly accounts
cannot mutate executable configuration even if they retain group membership. A supplied Origin must
match the request Host. Vimscript is executable trusted configuration and may
invoke browser integration features; it is not a safe format for untrusted
uploads. This is explicitly disclosed in the UI. No server shell, terminal,
package manager or arbitrary process execution is exposed.
An authorized `vim.wasm` workspace response includes the session Domain's
`vim_settings`. The Gateway retains that snapshot in its server-side session
and supplies it with `GET /vim/api/session` (`no-store`). Other adapter types
do not receive these settings. Changes apply on a newly opened session.
MEMFS initialization order: bundled defaults, configured font size, source
`/pageswiki-user.vimrc`, then a fixed-path custom colorscheme under
`/pageswiki-config/colors/pageswiki_custom.vim`. A conventional uploaded name
matching `[A-Za-z0-9_-]+.vim` is also registered as a local alias before the
user vimrc is sourced, so `colorscheme uploaded` works for `uploaded.vim`.
Other uploaded filenames remain display metadata; no uploaded name is inserted
into a command. Disabled customization
retains source text but does not execute it. `?safe=1` on the Vim frontend URL
skips all Domain customizations, including the font size, so a broken startup
script cannot prevent recovery through the defaults and settings panel.
Follow-up: versioned per-user settings, multiple named scheme slots, and an
isolated test-startup window. A restricted terminal is not needed for the
current configuration workflow. Any future terminal bridge needs a separate
capability, runtime sandbox, resource limits and explicit authority design.
Third-party plugin policy: vendor pinned, licensed pure-Vimscript files into
MEMFS and test against this old pinned Vim feature set. Candidate classes are
surround/text-object/comment helpers without native or network dependencies;
no unreviewed third-party plugin is added by this change. Runtime vim-plug
installation is unsupported. Vendoring `fzf.vim` alone is insufficient because
fzf requires a native executable; shell/system()-dependent plugins belong in
code-server or need a separately designed Wasm/JavaScript bridge.
## Verification
`go test ./...` includes asset routing, matching editor/authorization paths,
section persistence, invalid input, admin restrictions and child access tests.
Opt-in browser tests:
```sh
PAGESWIKI_BROWSER_TEST=1 go test ./internal/server -run 'TestEditorHomeChromiumSmoke|TestConfigChromiumSmoke|TestVimWasmChromiumSmoke|TestBookmarkAndVimSettingsChromium' -count=1
```
The Vim test uses CDP real-time waiting (virtual time can finish before Worker
compilation), two isolated files and a fake save endpoint. It asserts the actual
guifont, loaded `:Files` command, argument count and absence of startup errors.
It also edits a buffer with `:write` and verifies that BufWritePost exports the
changed bytes to the fake Gateway save endpoint. Both custom settings and safe
mode are exercised. UI tests open and close the settings dialog, save a
section with automatic children, and upload/save vimrc and colorscheme files.
+317
View File
@@ -0,0 +1,317 @@
# HTML projection trust boundary
Audited and remediated: 2026-09-14; controlled external-resource extension:
2026-09-23. Status: **normal HTML projections use `controlled-v2`**. Authored
active HTML remains denied by default. A Domain administrator may separately
approve an exact external resource URL; approved JavaScript is intentionally
executable and is therefore a high-trust Domain configuration action.
## Implemented contract — controlled-v2
`internal/server/html_projection.go` applies a parser-based allowlist using
[bluemonday](https://github.com/microcosm-cc/bluemonday), pinned to v1.0.27. The
sequence is bounded token preflight → authored allowlist → tree normalization
and authorized macro expansion → diagram references → final allowlist. This
changes display projections only, not saved Documents, hashes or Revisions.
- All known authored-HTML surfaces share this boundary: semantic Viewer,
configured home, profile Wiki, object UUID, historical Revision, Preview,
print, and HTML/Markdown editor preview. Object viewing checks adapter MIME
kind; non-HTML is escaped text or a download link, never blind HTML.
- Formatting, tables, code, internal/external links and local passive images
are supported. Author scripts, event attributes, forms, iframes, SVG/MathML,
styles, arbitrary classes and `data-*` hooks are removed. This is intentional
compatibility tightening, not a source migration or an executable trust flag.
- By default images must resolve to local `/v/raw/`, `/_/obj/`, `/static/`, or
`/o/path:/system/static/` resources after path normalization. External images,
relative semantic URLs, data images and arbitrary authenticated GET targets
are not loaded by authored markup. Use immutable asset/raw references.
- The Domain setting “Approved external resources” is the only exception.
External images require an exact normalized HTTPS URL match. External CSS
and JavaScript require both an exact HTTPS URL and pinned SHA-256 SRI. The
server discards authored integrity/crossorigin attributes and emits canonical
`integrity` and `crossorigin=anonymous` attributes from configuration.
Approval entries are converted to unpredictable text placeholders before the
authored and generated allowlists, then restored as server-authored tags;
macro output therefore cannot forge an approval. CSP is derived from the same
list: image/style origins and JavaScript content hashes are added while
`connect-src` remains same-origin. Revocation takes effect on the next render.
Wildcards, HTTP, credentials, fragments, inline script/style and event
attributes remain unsupported. Because approved JavaScript can invoke
same-origin APIs as the viewer, only immutable audited code should be added.
- Authored IDs and fragment targets are SHA-256-namespaced to avoid application
DOM collisions. Only code-language classes survive the authored pass.
- Only text-node macros expand, not macros in attributes, links or code.
Application-owned children hooks cannot be forged by authored `data-*`.
Each macro is dispatched once; lookup failures remain escaped authored text,
including entity-decoded tags and nested-macro-looking strings.
Existing read filtering is retained; City reads inherit request cancellation
and use bounded Store projections. Macro output is not cached across readers.
- Input is at most 1 MiB; token/tree budgets are 16,384 with depth 128; output
is at most 4 MiB before final sanitization; at most 32 macro invocations.
Preflight is deliberately conservative for deeply/unclosed malformed HTML.
Cooperative request/DB work has a two-second deadline. These are bounds, not
hard CPU isolation of the in-process parser. Failure shows a localized warning
and at most 64 KiB of escaped source, never original active HTML.
- `/_/api/v1/render/markdown` accepts optional `html` only with `projection:true`.
Display results are private/no-store. Canonical Markdown-to-HTML conversion
remains separate and preserves authoring semantics; it must not be displayed
without projection. Editor preview requests debounce/abort obsolete work and
fail to escaped text. Saving does not save the sanitized preview.
- Normal display/editor templates use per-response CSP nonces for application
scripts and prohibit authored inline event handlers. Nonces are inserted into
trusted template source **before** request data, never rendered body strings.
Same-origin application frames/resources remain available; document styles
are removed while application/theme styles remain enabled. Config and other
non-document screens are not claimed to have this CSP yet.
- Diagram SVG responses, including cache hits, have sandbox CSP and nosniff.
GET cache use requires current Domain/Path read authority and a live bounded
source reference (128 entries, 32 KiB/source, 30-minute lifetime). These
references are keyed by policy version, Domain, Path and source; responses
are private/no-store so browser caches do not bypass fresh checks. Reopening
the Viewer registers references after restart/eviction. Renderer requests
inherit cancellation; source/cache/output reads are size-bounded. Durable
diagram scheduling, disk quotas, atomic artifact publication and hard decoder
isolation remain follow-up work; foreground diagram GETs are not eliminated.
- There is no grandfathering of old active HTML and no document-level trusted
executable mode. The explicit Domain approval list above is the only active
external-resource exception.
Registered application/plugin frames are separate trust contracts. Anonymous
locality ACLs, ownership appointments, alias/move fences and public cache
revocation must land before enabling new public access.
Verification: `html_projection_test.go`, `html_projection_browser_test.go`, and
`diagram_projection_security_test.go` cover inert attack corpora, resource limits,
cancelled requests, unchanged stored content, cache authority/revocation and CSP.
Real Chromium loads nine display/editor surfaces as an administrator after a
path-scoped writer saves a hostile test document through the real save API.
Authored execution and privileged sentinel requests must stay absent; parser-
inserted non-nonced scripts/events are also blocked by CSP. Existing PDF/archive/
comic, metadata, bookmark and editor browser regressions remain required.
Read-only compatibility sampling of the main DB's latest 500 HTML **revisions**
(not a complete current-Path inventory) found no script/event/iframe/SVG markers
in their first 128 KiB, one style-bearing revision, and no body over 1 MiB.
This limited marker count is not a sanitizer/security proof or a guarantee that
no existing document needs layout changes. No production document was rewritten.
The following sections preserve the original diagnostic and design rationale;
their unsafe behavior describes the **pre-remediation** implementation.
## Finding and release gate
Before remediation, saved HTML executed JavaScript in the Pages Wiki application origin
when a reader opens its semantic Viewer. Document write permission therefore
implicitly grants the ability to execute code as a later reader, including a
reader with stronger permissions. Authentication, path ACLs, and CSRF checks do
not make authored HTML safe after it is inserted into the authenticated page.
HttpOnly cookies cannot be read directly, but same-origin authored code can
issue authenticated requests and read accessible responses or the page DOM.
**Before enabling or broadening shared links, comments, anonymous/public
viewing, or untrusted-author publication, this boundary must be closed and its
browser regression suite must pass.** This is also an existing risk in private
Domains with authors and readers of different privilege levels; it is not
limited to future public features. Do not describe the pre-remediation Viewer as an
isolated HTML viewer, or treat an HTML MIME classification as a trust decision.
## Observed entry paths (before remediation)
The central semantic path is:
`Revision.Body → expandMacrosForRequest → renderDiagramsInHTML → template.HTML → parent page DOM`
The macro and diagram helpers replace recognized constructs; they are not HTML
sanitizers and preserve unrelated scripts and event attributes. Go's
`template.HTML` deliberately bypasses normal template escaping.
| Entry / projection | Current implementation and insertion |
| --- | --- |
| `/{semantic-path}` | `internal/server/semantic_handlers.go`, `handleSemanticRoute`: HTML-kind `Revision.Body` is expanded and cast to `template.HTML`; `templates/pages/page.gohtml` inserts `.RenderedHTML` directly into `.page-html`. |
| `/` with a configured HTML home Path | `internal/server/handlers.go`, `handleHome`: the saved home body is macro-expanded and inserted through `page.gohtml`. |
| `/o/uuid:{object-uuid}` | `internal/server/object_uri_handlers.go`, `handleObjectView`: casts `latest.Body` directly to `template.HTML`, without even the MIME/adapter-kind guard; `object_view.gohtml` inserts it into the parent document. |
| `/r/view/{revision-uuid}` | `internal/server/revision_routes.go`, `handleRevisionView`: HTML-kind historical body is cast directly and inserted by `revision_view.gohtml`. |
| `/v/preview/{path}` | `internal/server/preview_handler.go`, `handlePreview` / `renderPreviewDocument`: HTML-kind body is expanded and inserted by `preview.gohtml`. |
| `/_/view/print/{path:…\|uuid:…\|rev:…}` | `internal/server/print_handler.go`, `handlePrint`: HTML body is expanded and inserted by `print.gohtml`. |
| `/u/{username}` | `internal/server/profile_handler.go`, `handleUserProfile`: the associated user Wiki HTML is expanded and inserted by `profile.gohtml`; child Wiki pages use the semantic handler. |
| `/e/{path}` live Preview | `static/js/editor.js`: HTML source and Markdown projection responses are assigned to `preview.innerHTML`. This is another active-markup sink to cover; script-tag behavior under `innerHTML` differs from initial document parsing, but event attributes and other active content must not be considered safe. |
The first row was verified in a real browser. The remaining rows were traced in
source and require endpoint-specific browser coverage during remediation. This
is an inventory of known sinks, not proof that no additional projection exists.
### Protections that do not cover these paths
Raw HTML download/attachment handling in `internal/server/raw_view.go`, the CSP
sandbox on raw SVG, and the sandboxed EML HTML frame are separate response
paths. They do not constrain saved HTML inserted into the normal Viewer.
The audited semantic response had no `Content-Security-Policy` header and its
authored content was not in a sandboxed iframe. `nosniff` or a correct
`text/html` Content-Type would not remove executable content.
## Local verification
A scratch Go test, injected with `go test -overlay` rather than added to the
repository, created a temporary migrated DB and temporary Document root. It
saved a test-only HTML Revision, bound `/audit/sentinel`, and served the real
semantic handler and templates through a localhost `httptest.Server`. A test
owner session supplied request authorization; no external instance was used.
Headless Chromium loaded that local page.
The payload contained only a paragraph, an inline script setting
`document.documentElement.dataset.htmlScriptAudit`, and an invalid `data:` image
whose `onerror` set `dataset.htmlEventAudit`. Both sentinel values became
`executed` in the parent DOM. The HTTP response still contained the original
inline script and its CSP header was empty. There was no data extraction,
credential access, or application mutation from the payload.
This diagnostic confirmed the current unsafe behavior; it is not a permanent
test that expects the vulnerability to remain. Remediation must invert those
expectations into denial tests.
## Mismatch with existing design
- [Architecture](arch.md) distinguishes immutable Document content from Revision
identity. Canonical HTML support does not itself authorize executable
JavaScript, and the reviewed architecture does not define all authors as
trusted application-script authors.
- [Projection adapters](projection-adapters.md), “Security and Failure Rules,”
says stored HTML and SVG retain their existing isolation rules. In practice
the raw/SVG isolation is not shared by semantic HTML projections. That wording
was insufficient to establish the missing boundary. That document now
records the implemented common projection boundary and its limits.
- [Platform extension architecture](platform-extension-architecture.md)
explicitly says document content may request a widget provider/resource but
may not supply executable JavaScript; providers use a sandboxed frame or a
sanitized server projection. Direct authored HTML execution bypasses that
intended separation.
- The Preview adapter-kind check prevents treating every binary/text file as
HTML, but it does not make a legitimately classified HTML body trusted.
## Policy rationale: immutable source, controlled projection
### Normal HTML is inert
Keep canonical Document bytes, hashes, Revision IDs, history, and authoring
round trips unchanged. Enforce safety when producing a display projection,
using a parser-based allowlist rather than regular-expression replacements.
The normal mode must not allow authored scripts, inline event handlers,
executable URLs, active embedded documents, or attributes that create a second
execution context. Define URL schemes, relative URL resolution, allowed markup,
CSS/layout limits, and external resource/privacy behavior explicitly. SVG,
MathML, `srcdoc`, forms, CSS URLs, and DOM-clobbering attributes need explicit
decisions, not implicit passthrough.
Introduce one projection boundary shared by Viewer, home, profile, historical
Revision, preview, print, and client-side editor preview. Object-UUID viewing
must also use the MIME/adapter registry rather than blindly casting a body.
If projection parsing fails or exceeds resource bounds, fail safely to escaped
source/download; never fall back to unsanitized HTML.
Distinguish authored nodes from server-generated macro/widget output. Parse
recognized macro requests into structured inputs, authorize their data access,
and produce escaped, typed output through trusted renderer contracts. A string
that resembles generated markup is not proof of trust. Avoid sanitizing only
before an expansion that can reintroduce authored unsafe attributes, or only
after expansion without accounting for required application-owned controls.
No general template call site should be able to turn arbitrary `Revision.Body`
into trusted markup by convention alone.
Editor authoring mode continues to preserve source, while its visual preview
uses the same safe projection policy. Do not conflate a safe display result
with a replacement for the canonical body when saving.
### Explicit trusted HTML is isolated
If executable HTML compatibility is genuinely required, make it a separate,
administrator-granted capability and an explicit Domain policy, not an implied
consequence of page write permission. Bind approval to the actual approved
content/revision or to a clearly defined trusted publishing workflow: granting
trust to a mutable Path must not let any later lower-privilege editor inherit
script execution. Define revocation, audit records, and visible mode indicators.
Executable content must not run with application-origin authority. Consider a
dedicated uncredentialed origin or an opaque-origin sandbox, with the minimal
permissions needed for that use case. Never combine same-origin document
content with permissive `allow-scripts allow-same-origin` and call it isolation.
Do not forward application cookies, bearer tokens, unrestricted save callbacks,
or privileged data into that context. Any necessary `postMessage` integration
must be narrowly typed, sender-checked, capability-scoped, and bounded; an
opaque-origin sender requires validating the known frame/window and a dedicated
channel rather than trusting the string `null` as an origin.
Existing application/plugin scripts remain a separate administrator-controlled
surface. Registered widget/plugin frames must not become an escape hatch for
arbitrary page-authored code or inherited reader capabilities. Shared/public
projections and comments must always use the safe policy unless a separately
reviewed, isolated publication contract explicitly permits more.
### Defense in depth and scaling
Adopt an application CSP compatible with approved application scripts, using
nonces/hashes or external script assets rather than unrestricted inline script
execution. CSP is defense in depth, not a substitute for safe markup and
isolation. Frame responses need their own policy; tightening only the parent
page does not fully define each child document's privileges.
Bound input size, parse depth/nodes, output size, CPU time, and any external
resource work. Cache content-only safe projections by immutable Document hash,
renderer version, and safety-policy version. Keep ACL-dependent macro output
separate or key/revalidate it with Domain, principal/grants, and request context;
never reuse a privileged projection as an anonymous/shared response. Cache and
preview artifacts generated under an older unsafe policy need invalidation.
## Compatibility impact and staged implementation
Existing pages may rely on inline scripts, handlers, style blocks, custom
attributes, embedded frames, forms, or SVG. Blanket filtering can break those
pages, diagrams, child-list controls, and editor previews. A restrictive CSP
also affects inline application scripts. The implementation above deliberately
separates these template-owned scripts from authored HTML; original source stays
available for editing/download if custom layout needs adjustment.
1. Approve the default allowlist, external-resource policy, and whether a
trusted executable mode is needed at all; record explicit ownership.
2. Inventory active constructs with a read-only diagnostic and report affected
Paths only to authorized administrators. Do not rewrite stored content.
3. Build the common projection boundary and server/client regression fixtures;
preserve supported macro, diagram, print, and editor behavior deliberately.
4. Integrate every known route, safe error fallbacks, policy-versioned caches,
CSP, and any separately reviewed isolated mode. Do not grandfather all old
HTML as trusted or infer trust from MIME, filename, creation time, or author
display name.
5. Verify upgrades and rollback without changing canonical bytes. Only then
remove the P0 gate for public sharing, comments, and anonymous expansion.
## Required tests and acceptance criteria
- Browser denial fixtures for parser-inserted scripts, event attributes,
executable URL variants, SVG/MathML active content, iframe/srcdoc, malformed
markup, DOM clobbering, and CSS/resource-policy violations. Use harmless DOM
sentinels and local test endpoints, not credential-exfiltration payloads.
- Apply the same corpus to semantic Path, configured home, object UUID,
historical Revision, user Wiki, preview, print, and both HTML/Markdown editor
previews. Test direct navigation, client-side replacement, and restored
revisions, not just the first render of new content.
- Cross-account fixtures: a lower-privilege author cannot execute in a reader's
page or issue reader-authorized mutations. Anonymous/shared viewers and
comments cannot activate markup or receive private macro results.
- Canonical Document bytes, SHA-256, Revision IDs, downloads, and authoring
round trips remain unchanged. Non-HTML bodies are never interpreted as HTML
by the object-UUID route. Unknown formats and parser failures stay inert.
- Supported formatting, links, images, children macros, diagrams, code blocks,
accessibility, themes, and print layouts still work. Template-owned controls
are not confused with forged authored markup.
- If trusted mode is implemented, verify origin isolation, cookie/token
exclusion, navigation/form/download permissions, message validation,
approval-on-edit behavior, revocation, and public-projection restrictions.
- Test CSP headers and violation behavior, resource bounds, cancellation,
oversized/deep input, cache policy-version changes, ACL revocation, and no
privileged-to-public cache reuse. A single endpoint-only sanitizer test is
insufficient to close this gate.
Acceptance requires inactive authored markup in every normal projection, an
explicitly reviewed policy for any executable mode, preserved canonical
content, and passing endpoint/browser integration tests. The normal projection
slice meets this boundary with no executable mode. Additional public/shared
surfaces must inherit it and pass their own ACL/revocation tests before release.
+395
View File
@@ -0,0 +1,395 @@
# Language Projection Draft
## Goal
Language should NOT affect semantic hierarchy.
Language is treated as:
```text
projection
representation
view
```
rather than containment.
* * *
# Core Philosophy
## URL hierarchy is language-neutral
Paths represent:
```text
semantic location
knowledge locality
human navigation
```
NOT language ownership.
* * *
# Example
Preferred:
```text
/dev/api/auth
```
Avoid:
```text
/ja/dev/api/auth
/en/dev/api/auth
```
because language hierarchy pollutes semantic hierarchy.
* * *
# Important Separation
Layer Meaning Path semantic hierarchy Page UUID identity Revision representation Language projection
* * *
# Internal Model
## Page
Language-neutral semantic object.
```text
Page UUID
```
Example:
```text
Page
├─ Japanese revision
├─ English revision
└─ Chinese revision
```
* * *
# Revision
Language-specific content snapshot.
## `revisions` table
| Field | Meaning |
| --- | --- |
| `uuid` | Immutable Revision UUID |
| `object_uuid` | Parent Page UUID; retained name during the Object-to-Page migration |
| `document_hash` | SHA-256 of the immutable canonical payload |
| `lang` | Canonical BCP 47 language tag or `und` |
| `message` | Commit message |
| `created_at_ms` | Unix epoch milliseconds |
The payload is addressed by `document_hash`; it is not identified by the
Revision UUID.
* * *
# Language Resolution
## Request
```http
GET /dev/api/auth
Accept-Language: ja-JP,ja;q=0.9,en;q=0.8
```
* * *
# Resolution Flow
```text
Page UUID
↓
find latest revision(lang=ja-JP)
↓ not found
find latest revision(lang=ja)
↓ not found
find latest revision(lang=en)
↓ not found
find latest revision(lang=campus default)
↓ not found
find latest revision(lang=system default)
↓ not found
find latest revision(lang=und)
```
* * *
# Fallback Philosophy
Language fallback is preferred over URL duplication.
* * *
# Recommended Fallback Chain
Example:
```text
ja-JP
↓
ja
↓
campus default
↓
system default
↓
und
```
Duplicate candidates are removed while preserving order. Invalid language
tags are ignored. `und` is the compatibility projection for revisions created
before language-aware delivery.
* * *
# Zone Language Metadata
Zones may optionally define language affinity.
## `zones`
| Field | Meaning |
| --- | --- |
| `default_lang` | Preferred fallback language |
| `allowed_langs` | Allowed languages |
* * *
# Example
## JP-oriented campus
```text
default_lang = ja
allowed_langs = ja,en
```
* * *
# Global campus
```text
default_lang = en
allowed_langs = *
```
* * *
# Spot Language Metadata
Spot may optionally specify:
| Field | Meaning |
| --- | --- |
| `preferred_lang` | Preferred rendering language |
This influences rendering preference only.
It does NOT change identity.
* * *
# Canonical URL Policy
Canonical URL remains language-neutral.
Example:
```text
/dev/api/auth
```
Language selection is handled at rendering time.
* * *
# Optional Explicit Language URL
Explicit language paths may exist for:
- debugging
- preview
- direct linking
- translators
Example:
```text
/_/lang/ja/dev/api/auth
```
However:
```text
this is NOT canonical
```
* * *
# Why This Design
## Prevents hierarchy pollution
Avoids:
```text
language hierarchy
```
overwriting:
```text
semantic hierarchy
```
* * *
# Benefits
## 1. Stable semantic URLs
The same knowledge object keeps the same path.
* * *
## 2. Shared semantic identity
Multiple languages refer to the same:
```text
Page UUID
```
* * *
## 3. Better multitree behavior
Aliases and bindings remain language-neutral.
* * *
## 4. Better graph compatibility
Semantic relations do not duplicate per-language.
* * *
# Relation to Existing Architecture
This design aligns with:
```text
path != object
```
and extends it into:
```text
representation != object
```
* * *
# Conceptual Mapping
Concept Meaning Path semantic locator UUID identity Revision representation Language projection
* * *
# NDN-like Interpretation
## Name
```text
/dev/api/auth
```
semantic and language-neutral.
* * *
## Content Negotiation
Language selection behaves similarly to:
```text
content negotiation
```
using:
```text
Accept-Language
```
* * *
# UI Philosophy
Language switching should feel lightweight.
Example UI:
```text
[Japanese] [English] [中文]
```
without changing semantic location.
* * *
# Important Principle
Language should not fork the knowledge universe.
Avoid:
```text
ja-wiki
en-wiki
```
style silo fragmentation unless intentionally desired.
* * *
# Current Recommended Direction
Layer Model UI semantic hierarchy URL language-neutral Internal revision projection Identity UUID Rendering Accept-Language + fallback
* * *
# Summary
Language is treated as:
```text
projection
representation
view
```
NOT:
```text
ownership
containment
hierarchy
```
This preserves semantic structure while supporting multilingual knowledge naturally.
+753
View File
@@ -0,0 +1,753 @@
# Architecture Implementation Roadmap
## Current backlog reconciliation (2026-11-08)
Use the [Japanese implementation backlog](ja/implementation-backlog.md) for the current
execution order, stable task IDs, acceptance criteria and explicit holds. Baseline: `e65ca56`.
Broad historical checkboxes below may contain implemented slices; they are not a count of
unimplemented features. Source completion, physical-client acceptance and production rollout
remain separate gates. The 2026-10-31 UX priorities below are historical context.
## Daily email history and controlled retries (2026-11-08)
- [x] Private, paginated delivery history and explicit same-day retry from personal settings.
- [x] Durable request receipts, attempt-generation fencing, opt-out/version/revocation rechecks and duplicate-delivery acknowledgment.
- [x] Bounded 90-day detail cleanup; preserve daily reservations and retry receipts permanently.
- [x] Additive schema v4 with old-state preservation, interrupted-migration rollback and integrity tests.
- [ ] Production rollout; backup restoration must reconcile external SMTP effects before resuming delivery.
- [ ] Retention controls, safe receipt compaction, queued-request cancellation and Inbox outcomes.
See [delivery contracts and recovery](ja/task-digest-delivery.md).
## Work history and project reporting (2026-11-08)
- [x] Owner-scoped paginated work history with inline correction/cancellation and lazy revision display.
- [x] Atomic immutable audit snapshots, optimistic concurrency, current authorization and cross-Domain overlap checks.
- [x] Exclude cancelled time from reports; aggregate by current project without per-project queries.
- [x] Task/dashboard navigation, Japanese/English UI and desktop/mobile browser coverage.
- [ ] Production rollout with an aligned DB backup; old binaries must not read the upgraded DB.
- [ ] Revoked-Domain recording recovery, cancellation restoration and audit retention policy.
See [work history contracts and migration](ja/task-productivity-contacts.md).
## Organizer dashboard (2026-11-02)
- [x] Common navigation for tasks, calendars, contacts and dashboard; home/navbar contacts entry.
- [x] Owner-only personal settings at `/-/sys/config?panel=organizer`, without expanding admin permissions.
- [x] Task tabs, today/tomorrow tasks, daily completion, calendar and activity widgets; reorder/resize and versioned layout persistence.
- [ ] Shared dashboards, multiple widget instances and sandboxed third-party widgets.
See [dashboard contracts](ja/organizer-dashboard.md). Production rollout is a separate gate.
## Task productivity and private contacts (2026-11-01)
- [x] Add idempotent task capture API and neomutt configuration reference.
- [x] Extend pages-cli with task planning, start/stop records, reports and private contact lookup.
- [x] Add per-user defaults, accomplishment heatmap and opt-in daily SMTP task digest.
- [x] Add home search commands with project/calendar suggestions and explicit creation confirmation.
- [x] Add private contacts CRUD, trash/restore, vCard import/export and unknown-property preservation.
- [x] Add private CardDAV serving with isolated credentials, conditional writes, bounded reports and sync tokens; revision preview/restore and whole-book export (2026-11-02).
- [ ] Shared address books, external CardDAV connectors and physical-device compatibility.
See [CardDAV operation and limits](../operations/carddav-contacts.md). The private profile is not full CardDAV compliance; production rollout is tracked separately.
See [contracts and remaining scope](ja/task-productivity-contacts.md). These checks describe
the implemented slice; production rollout and test evidence are recorded separately.
## Current highest priority (2026-10-31)
The user-prioritized [daily UX / browser / editor / CLI backlog](ja/priority-ux-20261031.md)
takes precedence over the feature backlog below. It covers type-to-search, virtual
Path navigation and browser commands, editor/plain-text and code styling, theme
switching, session timeouts, contacts, City labels and a session-aware terminal client.
Preserve authorization and storage safeguards. Track implementation, tests and
deployment separately; a source checkpoint includes the pre-existing dirty worktree.
## Principles
### Named architecture components (2026-11-01)
- [x] Adopt stable responsibility-based names for eight components; preserve UI terminology,
public APIs, persisted IDs, event consumer checkpoints and deployment boundaries.
- [x] Extract Grammaratur: ordinary and protected authoring conversions, compatibility
facades and application connections. Verification is recorded separately from deployment.
- [x] Extract Dregarnuhr clock/loop/admission/retry contracts and the queue adapter;
register the 14 existing business handlers outside the core. Preserve IDs/schema and atomic claims.
- [ ] Add Dregarnuhr expiring leases, fencing, multi-process scheduling and application shutdown integration;
these are not provided by the initial extraction. See [contract and remaining work](ja/dregarnuhr.md).
- [x] Extract Geduldh local payload storage and bounded materialization; preserve the Document Store
contract through aliases, connect application consumers, and test layout/Range/backfill/DR compatibility.
External backends and placement changes remain separate work; see [contract and remaining work](ja/geduldh.md).
- [x] Extract Mestionora SQLite search/index/City lookup with explicit read-policy callbacks,
legacy aliases, candidate/snippet budgets, plan and authorization tests; see [search contract](ja/mestionora.md).
- [ ] Preload lazy user-space read policies before Page search; retain snapshot consistency while
removing nested connection requirements. Keep current-only/Domain-partitioned and language-aware indexing as later migrations.
- [x] Extract Ordoschnellic bounded Web Push dispatch, retry policy and encrypted transport;
preserve transactional Inbox enqueue, authorization, event IDs and subscription checkpoints.
See [delivery contract and remaining work](ja/ordoschnellic.md).
- [ ] Add distinct Push claim generations and subscription-version fencing, Retry-After and
delivery metrics. Initial extraction retains existing retry/lease limitations, not exactly-once delivery.
- [x] Extract Schutzaria pure Path/locality/group/publication/user-space evaluation and approval state transitions;
retain authenticated fact loading, alias checks and transaction ownership in the application.
See [policy contract and remaining work](ja/schutzaria.md).
- [ ] Complete consistent authorization snapshots, lazy user-space preloading, reviewer inheritance/quorum,
and publication revalidation. State-machine extraction alone does not enable the reviewer workflow.
- [x] Extract Kunstzeal built-in adapter selection, bounded Kroki rendering and diagram hash/cache;
retain source authorization/CSP and add redirect refusal plus atomic cache replacement.
See [rendering contract and remaining work](ja/kunstzeal.md).
- [ ] Add provider-versioned cache keys, render concurrency/coalescing, unified disable/metrics and
additional diagram/slide providers. Initial extraction does not implement nwdiag or slides.
- [x] Extract Seheweit resource collection, typed snapshots and history window/retention policy;
retain APIs/schema/event IDs, connect foreground and background paths, bound proc reads,
guard storage arithmetic, propagate write cancellation and reject read-only sample writes.
See [observation contract and remaining work](ja/seheweit.md).
- [ ] Add source availability, bounded/coalesced collection, rollups/retention budgets,
component/provider metrics and instance-scoped HA/SNMP collectors. These are not part of the initial extraction.
See [component names, boundaries and acceptance criteria](ja/component-names.md).
All eight initial component extractions are implemented; unchecked items describe remaining capabilities.
Released and deployed as [v0.2.0](../operations/deployments/2026-11-01-v0.2.0.md) on 2026-11-01.
- [x] Eliminate AddEventLog deferred read-to-write transaction contention with writer-first
acquisition, atomic raw-log/bucket writes and independent concurrent-writer tests.
Completion-log failure does not replay successful business work; see [Dregarnuhr](ja/dregarnuhr.md).
- [ ] Recover missing logs, improve event-only fallback/retention and define completion/log
atomicity. Writer-first acquisition does not guarantee all logs survive prolonged contention.
### Storage portability and operations (2026-09-27)
Implemented and fixture-tested; included in the [v0.2.0 production release](../operations/deployments/2026-11-01-v0.2.0.md).
This does not mark the unchecked backend/HA capabilities below as implemented.
- [x] Immutable local Document Store contract, bounded materialization and streaming archive access.
- [x] Resumable document backfill with store identity, UUID cursor and batch checkpoints.
- [x] SQLite migration ledger with version/checksum, atomic adoption and current-schema assertions.
- [x] Managed projection, resource, broadcast, navigation, adapter-session, import and event-queue schemas.
- [x] Read-only `pageswiki schema status|check` with JSON output, pending steps and explicit legacy exclusions.
- [ ] Remaining legacy schema/repository contracts and context propagation; configurable materialization limits/metrics.
- [ ] PostgreSQL/MySQL, S3/SFTP and mounted-store contracts, optional Redis cache.
- [ ] Coordinated backup/restore, portable export/import, Web operations and initial setup wizard.
See the [Japanese design and acceptance criteria](storage-portability-and-operations.md) and
[preflight / deployment procedure](../operations/storage-schema-rollout.md).
Passing managed-schema checks does not replace an isolated full-startup rehearsal.
### Task management first slice (2026-09-24)
2026-09-25 task CalDAV (`0.8.0`, implemented and fixture-tested, **deployed to app/app2 on 2026-09-26 at 00:24 JST**; [deployment record](../operations/deployments/2026-09-26-task-caldav.md)):
private fixed VTODO collection, revocable read/read-write app passwords with UI,
conditional writes and archive deletion. Real vdirsyncer 0.21.0 / Todoman 4.7.0
discovery and bidirectional sync tests. Date-only deadlines; not full CalDAV
conformance. See [task CalDAV contract and CLI setup](ja/task-caldav.md).
2026-09-25 task follow-up (`0.7.0`, implemented and deployed to app/app2 at 07:07 JST):
daily/tomorrow planning, explicit carryover rescheduling, hierarchy-aware pagination
and parent context, and atomic Wiki-note creation with UUID-bound view/edit actions.
See [daily task / Wiki note contract](ja/task-daily-notes.md). Capacity, bulk family
planning, manual sorting and shared tasks remain open.
2026-09-25 follow-up (implemented, fixture-tested and **deployed to app/app2 at 05:59 JST**; [deployment record](../operations/deployments/2026-09-25-active-notifications-calendar-mobile.md)):
- [x] Inline title/status/date editing, independently dated two-level subtasks,
bounded monthly day schedule, home shortcut and Today navigation modal (`0.6.0`).
- [x] Inbox presentation preferences per account/browser, focused expiring toasts,
opt-in return modal, sound presets/volume, notification-only Service Worker.
- [x] Opt-in Web Push: transactional Inbox outbox, revocable session-bound subscriptions,
VAPID, vendor allowlist/DNS pinning, bounded scheduler delivery/retry/expiry and focus acknowledgements.
- [ ] Live OS/browser delivery acceptance with an explicitly opted-in test device
(Chrome/Firefox/Safari), iOS home-screen installation and OS power-saving behavior.
See [active notification contract](ja/active-notifications.md) and
[inline task/day schedule contract](ja/task-inline-schedule.md).
- [x] `organizer.tasks` private per-principal/Domain items and projects, CAS payloads,
atomic head/revision/change journal, conditional writes and idempotent creation.
- [x] Today, Inbox, Backlog, Someday, Review, Completed and Archive; project/task detail,
quick capture, due/planned/review dates, priority, estimate and readable Wiki UUID links.
- [x] Bounded lists/history, EN/JA, theme-aware desktop/mobile dialogs, extension kill switch.
- [x] Typed independent outbox subscriptions; atomic task/project events, bounded journal
migration through the scheduler, no City jobs for task-only changes.
- [ ] Shared collections/assignees, export/recovery and bulk review.
- [x] Opt-in due/planned-date task reminders, atomic reservations/cancellation,
bounded background Inbox delivery, live metadata authorization and badge refresh.
- [x] Private local calendar time-block links, durable retry intents, current event time/ETag lookup, unlink tombstones; imported/shared/public calendars excluded.
- [x] Persistent Pomodoro: start/pause/resume/finish/cancel, one active session per principal, conditional transitions, bounded scheduler completion and Inbox notification.
- [x] Today bounded estimates/current local time blocks/recorded focus intervals, overlap and incomplete-total indicators. Calendar event summaries include an explicitly approved task-title snapshot, never notes.
- [x] Daily carry-over, work-period/current-project reports, personal notification defaults and
opt-in daily task digest with delivery history and manual retry (source through 2026-11-08).
- [ ] Personal capacity, historical focus-span backfill, general calendar reminders, quiet hours,
Pomodoro personal defaults/cycles and routines; see TASK-01 / ORG-01 in the current backlog.
See [time-block and Pomodoro contracts](ja/task-planning-focus.md) for API, privacy,
two-store recovery, polling limits and remaining work. Canonical calendars and task
DB metadata both require backup. Production app/app2 deployment completed on
2026-09-25 JST; see the [deployment record](../operations/deployments/2026-09-25-task-today.md)
for backup, verification and the observed transient scheduler lock contention.
- [x] Limited private VTODO / CalDAV with vdirsyncer + Todoman file synchronization.
- [ ] Time-of-day deadlines, recurrence/alarms, shared task collections, incremental
sync-token and additional CalDAV clients; dependency graph, tickets and offline UI.
See [Japanese task implementation contract](ja/tasks-implementation.md).
The existing `/v/plan` calendar default is unchanged; tasks start at `/v/plan/today`.
### Calendar subproject (2026-09-22)
- [x] Small-screen/touch calendar navigation, readable week columns and day selector,
scroll-first gestures, compact month counts and VisualViewport-aware editor.
[Mobile contract](ja/calendar-mobile.md): fixture-tested and deployed 2026-09-25; iPhone Safari acceptance pending.
- [x] Initial file-canonical calendar/month snapshots, rebuildable SQLite index,
conditional event CRUD and bounded CalDAV time-range REPORT.
- [x] `/v/plan` year/month/week/day/events/timeline, event dialogs and per-calendar collaborators.
- [x] Per-user default calendar overlays and explicit comma-separated `pretty_name` selection.
- [x] Week/day time grids, configured-timezone day boundaries, zoom through 400% with 5-minute precision,
ETag-protected move/start/end resizing without refetch, equal overlap lanes, drag-range creation,
horizontal rescheduling and `week/list`.
- [x] Read-only ICS polling with interval/conditional GET and bounded external CalDAV REPORT/multiget;
background event queue, encrypted Basic credentials, SSRF guards and failure-safe snapshots.
- [x] Outlook published-ICS projection for METHOD:PUBLISH, VTIMEZONE/TZID, Windows timezone names,
bounded RRULE expansion, EXDATE/RDATE, moved/cancelled RECURRENCE-ID instances and orphaned
exception projections, plus multi-megabyte synthetic fixtures.
- [ ] Recurrence PERIOD/THISANDFUTURE, alarms/durations, provider validation, discovery/OAuth,
configurable synchronization windows and conflict-aware bidirectional synchronization.
- [ ] Full iCalendar temporal semantics (recurrences, timezones, exceptions), CalDAV discovery/client compatibility,
application credentials, synchronization tokens and maintenance/metrics jobs.
- [ ] Group sharing, audited administration, ownership recovery and calendar scheduling/notifications.
See [Japanese implementation contract and limitations](ja/calendar-implementation.md).
The initial supported subset is not a claim of full CalDAV conformance.
System-service plugins are distinct from document adapters. See
[System extensions](system-extensions.md) for implementation boundaries and the next slices.
- [x] Responsive wider bookmark-card grid (viewport-dependent columns).
- [x] Built-in resource.sample event, SQLite history, retention and bounded graphs without foreground collection.
- [x] Separate system-extension catalog and settings area for trusted built-ins.
- [x] Unified execution/resource monitor (running jobs only), paginated job history with lazy logs.
- [x] Protect group owners from ordinary member downgrades and atomically retain an owner during concurrent removal.
- [x] Transactional Domain-sequenced activity outbox and independent typed consumer checkpoints;
task/project source mutations append atomically. See [delivery contract](ja/outbox-delivery.md).
- [ ] General Domain/principal recent-edit projections, consumer operations and safe outbox retention.
- [x] Diary/blog initial built-ins: Page UUID metadata, calendar/recent entries,
categories, read-filtered indices, per-Domain enable/disable and registration UI.
See [initial scope](diary-blog-plugins.md); this is authenticated, existing-Page registration only.
- [x] Diary/blog atomic Markdown new-post composition, transactional metadata/Revision,
embedded-asset freezing, idempotent retry and create/write enforcement.
- [x] Diary/blog approved anonymous metadata and full-body RSS feeds; see [current scope](ja/blog-diary.md).
- [ ] Recent comments and safe resumable tDiary import; declarative time/OS/sunset theme rules.
- [x] Diary/blog per-Domain prefixes, optional keyword URL planner, multiple articles
per day, and authenticated Domain-root diary/blog entry view (2026-09-17).
Legacy Page registration remains available.
- [x] Domain entry-settings dialog and signed manifest v2 provisioning of layout,
diary/blog enablement and prefixes; atomic destination apply and managed edit guards.
- [x] Signed enrollment capability proof and Campus-wide approved-peer v2 preflight
before entry-setting changes; unknown/legacy peers block with a visible explanation.
- [ ] Entry provisioning inside Domain creation, controlled-settings release/retention,
applied-setting acknowledgments and freshness policy.
- [x] Diary/blog dated viewer routes `/v/{kind}/yyyy/mm[/dd]`, `own` presence
filter, legacy-query redirects, and canonical calendar/filter/pagination navigation.
- [x] Local same-Campus diary/blog index/calendar projection without copying articles;
source and destination read-policy filtering, source body/edit links, target write guard.
- [x] Same-instance destination-host article bodies, approved anonymous projection and RSS.
- [ ] Cross-instance source transport/revocation and source-setting federation.
- [ ] Rankings, reactions and moderated Page/inline comment services.
- [ ] Linkding-compatible bookmark-source injection with scoped credentials.
- [ ] Explicit external-search fallback and PagesWiki/GROWI/PukiWiki connectors.
- [ ] S3-compatible Document Store; read-only WebDAV/SSH external collections.
- [ ] Optional SNMP resource collector with target/OID allowlists and SNMPv3 secrets.
- [ ] Third-party system-plugin signature validation, isolated runtime and lifecycle operations.
vim.wasm runtime fixes are on hold by user request; its existing configuration UI is retained.
### Prioritized execution order (2026-09-14)
Feature checkboxes above are not an instruction to bypass the hardening gates.
The execution order is maintained in [Implementation status](implementation-status.md#ordered-next-work).
1. Completed this slice: unified monitor/history UI, bounded/indexed job/log retrieval;
first P0 correction: owner downgrade and concurrent last-owner removal protection.
2. Completed bounded hardening: OIDC/JWKS, inherited-admin revocation,
browser mutation/purpose checks, Domain/Object scope and read filtering;
see the [execution ledger](hardening-execution-2026-09.md).
3. Completed recursive authorized snapshots, literal prefix transactions and
signed manifest/checkpoint atomicity. Remaining write-policy fencing and
content-author trust are gates before public/shared collaboration.
4. Completed PDF/archive metadata jobs, bounded read-policy snapshots,
Object/Revision pagination and improved search; continue lazy City/directory
projection, decoder isolation and durable entry extraction.
5. Transactional activity outbox, actor attribution, idempotent URL reconciliation
and recent-edit projections.
6. Diary/widgets/import and themes, then comments/rankings, external search/bookmarks,
storage connectors and SNMP. HA leases/fencing gate multi-node deployment;
external plugin installation remains disabled until isolation/security gates pass.
- Migrations must preserve existing UUID and path reachability.
- New architecture is introduced in vertical slices with regression tests.
- Reserved authorization features must not be simulated with insecure
allow-all placeholders.
- New collaboration and extension features must use the stable Page UUID,
immutable Revision, Domain-scoped policy, projection-adapter, event, and
scheduler contracts described in
[Platform Extension Architecture](platform-extension-architecture.md).
- SQLite remains a supported single-node metadata backend. HA work must add a
replicated metadata strategy or explicit single-writer fencing; sharing a
live SQLite file over a network filesystem is not an HA design.
## Phase 1 — Identity and projection foundations
- [x] Separate Object creation from Semantic Path binding.
- [x] Rename the implemented District model to Domain.
- [x] Separate Revision UUID from Document SHA-256.
- [x] Add Revision language metadata and fallback selection.
- [x] Move immutable payloads to `documents/sha256/` with a transitional
SQLite compatibility cache.
- [x] Add rsync-like and interactive `path:` / `uuid:` / `rev:` / `asset:`
data administration.
## Phase 2 — Knowledge City topology
Current review: [City reconciliation and navigation](city-review-and-reconciliation.md).
The completed items below are initial implementations. The next City slices are:
- [x] Audit City/autoplan/gen-city and design URL-driven event reconciliation.
- [x] CITY-A named links: parent-qualified Street URLs, ambiguity-safe lookup,
shared search/map/macro links, ancestor authorization, lightweight details,
literal Page URLs and map focus/locality-search regression coverage.
- [ ] CITY-A: per-result authorization, literal Paths, scoped links/names,
shallow/portal inference, manual ownership and idempotent application.
- [ ] CITY-B: transactional mutation outbox → debounced, durable City queue
consumer; scheduled/manual repair through the same pipeline.
- [ ] CITY-C: stable fabric, Worker/cache, connected roads, bounded lazy
projection and capacity-safe picking targets.
- [ ] CITY-D: icon/modal creation/settings, arrow/WASD/HJKL navigation,
focus/history handling and i18n browser regressions.
- [x] City label follow-up: visible resting labels, horizontal Street names,
retained hover/focus highlighting and working label toggle; normal/popup tests.
- [x] Manual City organization: display-title modal, Ctrl/Command selection,
existing/new Spot merge, Street relocation retaining Page URLs, transactional
validation and manual-inference overrides; see [contract](city-organization-ui.md).
- [x] Immersive City follow-up: focused Spot courtyards and surrounding Page
buildings, nonoverlapping bounded/paginated close-ups, themed architectural SVG,
modal utility menu and directory navigation; normal/popup regressions. See
[projection and limits](city-immersive-map.md); CITY-A–D gates remain open.
- [x] Filter City map/details/macros/badges by readable boundaries and Pages;
recompute visible portals/counts/relations and use private responses.
- [x] Local gen-city Worker with time/input/output/cache bounds, cancellation
and retry, plus spatial Arrow/WASD/HJKL navigation in normal/popup modes;
see [worker and navigation](city-worker-navigation.md).
- [x] One-connection-safe bounded City snapshots and literal Path
containment/overlap/context/health; true lazy Page pagination remains open.
- [x] Define Campus identity and defaults.
- [x] Define Domain ownership by Campus.
- [x] Scope Semantic Paths by `(domain_uuid, path)`.
- [x] Add local/remote Domain placement.
- [x] Add absolute cross-Domain link helpers.
- [x] Add standalone/controller/managed Campus authority modes.
- [x] Scope non-topology configuration by Domain.
- [x] Scope Recent Objects by selected Campus and Domain.
- [x] Record controller-selected Domain instance FQDN.
- [x] Add hashed enrollment tokens and fingerprinted instance registration.
- [x] Add pending/approved/revoked instance placement selection.
- [x] Add external Campus bootstrap enrollment.
- [x] Add Managed Campus health status and detach actions.
- [x] Add signed Campus manifest reconciliation.
- [x] Implement ZoneTree nodes for Quartier and Street.
- [x] Implement Spot identity and ZoneTree pointers.
- [x] Implement SpotGraph Page bindings and MagicPage traversal.
- [x] Define Quartier planning, Street routing, Spot locality, topology
invariants, projections, APIs, policy inheritance, and delivery slices.
- [x] Add the initial topology schema, monotonic Domain topology version,
topology events, and Quartier/Street/Spot creation stores.
- [x] Add the initial interactive `/v/city` SVG guide map and authorized map API.
- [x] Add SpotPage/MagicPage edges, Viewer locality badges, Spot detail Page
lists, and initial typed SpotGraph relation visualization.
- [x] Add Quartier/Street/Spot detail projections, transactional Street/Spot
moves, ordering, and reversible SpotPage/SpotRelation edge removal.
- [x] Add planning preview/apply, reversible before/after event payloads, and
city health analysis.
- [x] Add `/v/city` map and locality breadcrumbs.
- [x] Add locality-scoped search and topology macros.
## Phase 3 — Projection runtime
- [x] Formalize the initial Viewer projection adapter registry and contract.
- [x] Route Viewer and semantic editor selection through adapters.
- [x] Route preview through adapters, including bounded text and CSV projections.
- [x] Route bounded print layouts through adapters at `/_/view/print/<address>`.
- [x] Route the semantic Viewer through the initial adapter registry.
- [x] Add bounded text retrieval backed by seekable Document Store reads.
- [x] Defer PDF page rendering to idempotent scheduler jobs with retrying
Viewer delivery and bounded PDF page-count commands.
- [ ] Defer non-ZIP archive metadata and entry extraction to cached, bounded
scheduler projections.
- [x] Persist PDF page-count and non-ZIP metadata via shared scheduler jobs,
bounded immutable-hash cache and localized pending/retry views.
- [ ] Persist seekable non-ZIP extracted entries with quota/lease/Range and
archive-bomb controls; move ZIP central-directory work off the request path.
- [x] Add bounded structural JSON retrieval by immutable Revision and lazy
JSON-Pointer expansion without unmarshalling the complete payload.
- [x] Separate authoring and projection macro transpilation so HTML-to-Markdown
round trips preserve `children` while Markdown previews render it.
- [ ] Generalize the authoring/projection macro contract to typed extension
macros with versioned round-trip and projection tests.
- [x] Add ZIP/archive directory and Debian metadata adapters.
- [x] Add a bounded CSV table adapter.
- [x] Add the bounded `@excalidraw/excalidraw` hosted Viewer plugin with an
isolated immutable-Revision contract and safe built-in Canvas fallback.
- [x] Separate built-in projections from Domain-scoped hosted adapter plugins.
- [x] Restrict hosted adapter targets to HTTP(S) services such as vim.wasm or
a separately deployed editor gateway.
- [x] Define and implement one-use launch tickets and optimistic Revision
write-back with a reference hosted-editor gateway.
- [x] Connect an internal code-server instance to the hosted-editor Gateway,
with session workspace materialization and autosave Revision write-back.
- [x] Connect a cross-origin-isolated vim.wasm frontend to the hosted-editor
Gateway contract with hierarchical MEMFS loading and Revision write-back.
- [ ] Evaluate a local helper, temporary workspace, and executable allowlist
before enabling native applications such as Vim or VS Code.
- [x] Freeze embedded mutable Asset HEADs into immutable Document hashes on
Revision save and retain authorization references from the containing Page.
- [x] Scope mutable Asset addresses and management metadata by Domain.
- [x] Add typed Asset MIME/filename/size metadata and immutable HEAD history.
- [x] Add administrator HTTP create/update/delete with ETag preconditions.
- [x] Add the initial Asset management panel under Domain configuration.
- [x] Add Asset HEAD history/restore and Page/Revision dependency-detail UI.
- [ ] Add Asset retention controls, policy-matrix capabilities, and general
audit-outbox integration as specified in `asset-model.md`.
- [ ] Add a capability-checked mutable Asset authoring-preview endpoint and
typed `srcset`/CSS resource resolution without allowing live Asset URLs into
stored Viewer output.
- [x] Add authorized immutable object retrieval at `/_/obj/sha256:<hash>`,
limited to Documents referenced by a readable Path in the current Domain.
## Navigation and home layouts
See [Home layouts and sidebar design](navigation-home-sidebar.md).
- [x] Vendor Hack Nerd Font for Browser and Tree without a runtime CDN dependency.
- [x] Add Domain-scoped classic, centered-search, and search/bookmark home layouts.
- [x] Keep home search global to the current Domain even when rendering `/top`.
- [x] Scope `/v/tree/{path}` to the requested subtree with breadcrumbs and cached expansion.
- [x] Limit Browser revision lookups to displayed files and repair missing revision indexes.
- [x] Define a shared sidebar layout, state ownership, and bounded children API contract.
- [ ] Implement the shared sidebar and authorized, paginated lazy navigation API.
- [x] Group configuration navigation and split scheduler, queue, logs, resources,
authentication providers, and adapter-specific settings into separate panels.
- [x] Add accessible themed switches and a Domain bookmark manager for Paths/URLs.
- [x] Restore public application assets and align compact/legacy editor paths.
- [x] Manage bookmarks from home; section cards and permission-filtered async child links.
- [x] Refine bookmark management into an icon-launched modal and section-card editor with group operations.
- [x] Add admin-only Domain vimrc/colorscheme upload, download/reset, session-scoped delivery and safe-mode recovery.
- [x] Explicitly load hosted Vim configuration, bundle its font and add a pure-Vimscript navigation plugin; verify the real Wasm runtime.
See [home navigation and hosted Vim](home-bookmark-sections-vim.md).
- [x] Add bounded, visibility-aware live resource graphs without a CDN dependency.
- [x] Repair Asset GET authorization and align metadata/history/reference reporting.
- [ ] Add versioned adapter-specific settings schemas and secret-field handling;
see [Configuration panels](configuration-panels.md).
- [ ] Add a transactional directory index and batch authorization for large hierarchies.
## Phase 4 — Identity and authorization
- [x] Normalize password, TOTP, OIDC, and magic-link authentication into Principals.
- [x] Add the initial direct Membership, Role, and positive-grant schema.
- [x] Add the initial Domain-level Capability evaluation with default deny.
- [x] Replace capability inference from URL words such as `delete` and `rename`
with a centralized typed endpoint policy classifier; deny undefined mutations
in reserved namespaces and method-close exact API routes.
- [ ] Complete typed object/path target resolvers for derived views and remove
the remaining handler-specific authorization exceptions.
- [x] Resolve raw, windowed text, PDF-page, preview-cache, print, Object,
Revision, history, and diff reads through their source Object/Path grants.
- [x] Filter Tree, Browser, Search, Preview listings, and asynchronous children
results per visible Path without returning hidden child counts.
- [x] Scope the initial Object list and Revision list API responses to the
current Domain and readable Object/Path grants.
- [x] Declare JSON/multipart target resolution as handler-owned policy and
remove route-name bypasses from middleware; enforce every expanded source
and destination Path for recursive rename, copy, and delete operations.
- [x] Bind hosted adapter sessions to the issuing Principal and re-check
current read/write grants on every workspace/content operation.
- [ ] Move multipart/JSON source-and-destination mutation checks into typed
decoded-request objects shared by middleware and handlers.
- [x] Add cookie-session CSRF and credential-type separation (legacy unprefixed
sessions expire normally); POST-only logout and live membership checks.
- [x] Verify OIDC issuer JWKS signatures/claims/browser state and reject
ambiguous email matches or conflicting external-identity binding.
- [x] Add Domain-scoped LDAP/LDAPS authentication with service search bind,
escaped exact-one user lookup, user-DN password bind, encrypted secrets,
immutable provider identity, bounded time/concurrency, optional member-only
provisioning, saved-configuration test UI, and Japanese/English labels.
- [x] Revoke derived Campus-admin grants transactionally without deleting
independent local grants or regenerating removed bootstrap authority.
- [x] Scope foreign/unbound Object authority and record atomic API creation
provenance; check all local aliases for global Object purge.
- [x] Revalidate zombie eligibility within the purge write transaction; scope
HTTP/config cleanup by Domain provenance and reject orphan-producing writes.
- [x] Bound Object/Revision metadata APIs with cursor pagination and batched
read checks; compact search exposes full Path and safe matching excerpts.
- [x] Common inert HTML projection boundary, bounded macro expansion, application
CSP and cross-account browser denial tests; preserve canonical authoring bytes.
See [policy and compatibility](html-projection-trust.md). No executable mode.
- [x] Gate the first anonymous semantic/raw slice on explicit current-revision
manifests, inert HTML, mutation invalidation and denial/revocation tests.
- [ ] Extend those gates to anonymous discovery/dependencies and delegated
shares with full impact-aware mutation/alias handling.
- [x] Add Domain groups, group Role bindings, and `/g/` management surfaces.
Full authorization auditing and external group mapping remain separate gates.
- [ ] Add explicit OIDC/LDAP group mappings and policy-version invalidation;
LDAP authentication must remain usable without enabling group synchronization.
- [ ] Add auditable policy decisions.
- [ ] Add an authorized effective-permission inventory for Campus, Domain,
Quartier, Street, Spot, Page, share, group, Role, and Principal scopes.
- [ ] Add explicit anonymous-read policy grants without weakening default deny.
- [ ] Add signed PSK federation of selected Principal, group, Role, and grant
records between Domains, including origin, sequence, conflict, revocation,
and `campus_admin` inheritance policy.
## Cross-phase enabling work — Extension seams
These items must land before the later feature that depends on them. They are
listed separately because they replace concrete single-feature assumptions
without changing user-visible Page identity.
- [x] Introduce the seekable, content-addressed Document Store interface and Geduldh
local payload/materialization adapter. External backends, legacy context propagation
and placement contracts remain open; see [Geduldh](ja/geduldh.md).
- [x] Introduce a transactional Domain event outbox with typed subscriptions and checkpoints.
- [ ] Extend producer coverage and replay/retention contracts before new consumers depend on it;
see [remaining delivery work](ja/outbox-delivery.md).
- [ ] Add a typed extension manifest and provider registry before adding third-
party transpilers, widgets, scanners, storage backends, or themes.
- [ ] Inventory process-local sessions, caches, preview slots, scheduler state,
adapter workspaces, and signing material before declaring LB/HA support.
- [ ] Add typed annotation/reaction/assessment stores keyed by Page and Revision
UUID before exposing comments, likes, or quality evaluation in Viewer HTML.
- [ ] Add privacy, retention, moderation, audit, export, and deletion contracts
before collecting view history, engagement, comments, or notification data.
## Phase 5 — Delegation
- [ ] Add Macaroon root-key rotation and verification.
- [ ] Implement attenuation caveats.
- [x] Add server-rendered OGP/X link cards for already-public Page/home/City
views, absolute configured-host URLs and a cached PNG fallback; retain
authentication and private metadata suppression. See [social cards](social-link-cards.md).
- [ ] Add read-only share projections under `/s/`.
- [x] Add Quartier/Spot owner appointments and governance dialogs; subordinate
publication policies may only narrow superior ceilings. Keep permission to
publish separate from actual publication and preserve authenticated ACLs.
See [detailed ownership/publication design](locality-publication-ownership.md).
- [x] Add named user/group read and read/write settings with ancestor caps,
stable identities, next-request membership checks and recovery authority.
Initial namespace guards freeze protected moves/aliases and City structure;
replace them with impact-aware mutation workflows in the next slice.
- [x] Add bounded explicit current-revision approvals and a separate inert
anonymous semantic Viewer/raw-media path with safe OGP and revocation tests.
No automatic activation of drafts or future edits. See
[exact scope and follow-ups](locality-access-publication.md).
- [x] Public Page breadcrumbs, asynchronous children, read-only tree and filtered
City navigation; database-generation cache invalidation and fresh authenticated
read filters. Remove public download controls; reserve isolated presentation
for share URLs. See [navigation contract](public-navigation.md).
- [x] Add owner-managed `/u/{username}` audiences for profile metadata, avatar,
Wiki root and descendants, plus narrowing subtree overrides, a dedicated
anonymous reader/navigation snapshot, immediate superior-ceiling revocation,
alias/UUID/Revision fences and atomic rename tracking. Keep `/u/` out of the
global anonymous tree. See [user-space publication](user-space-publication.md).
- [x] Harden anonymous navigation: public root/virtual-folder landing, read-only
directory, Page/children round trips, tree expand/collapse/filter/keyboard
controls, mobile header and City exits, cancellable bounded children loading,
retries and progressive pagination; retain exact-publication boundaries.
- [x] Explicit Street/Spot automatic publication with inherited manual-approval
boundaries, confirmed impact previews, live publisher/ACL checks and cache
invalidation. Existing scopes remain opt-in; see
[implemented slice and limits](automatic-publication.md).
- [ ] Immutable review requests, scoped approvers, durable outbox delivery and
reviewer notification Inbox. Continue the remaining AP-1 through AP-6 work in
[approval workflow design](publication-approval-inbox.md); automatic publication
does not implement the human approval workflow or silently activate drafts.
- [ ] Complete dependency/alias-aware publication previews, large-site review
and per-capability audiences; current approvals exclude aliases, restricted
user/group scopes and dependencies rather than implicitly publishing them.
- [ ] Fence locality publication against alias/move/merge/archive/import bypass,
and share one anonymous read evaluator across safe Viewer/media, City/search,
children and OGP. Gate enablement on HTML isolation and revocation tests.
- [ ] Add share management, expiry, revocation, access audit, optional password,
download prohibition, language/path restrictions, and QR-friendly links.
- [ ] Ensure share credentials are separate from login, adapter, federation,
and API bearer credentials.
## Phase 6 — Activity, notification, and collaboration
- [x] Editor resilience E1–E5 initial delivery: local drafts, conditional language-head saves,
conservative merge/conflict review, revision restore, language forks, mobile input, macro completion,
and protected HTML authoring blocks. See [Japanese implementation contract](ja/editor-resilience.md).
- [ ] Editor E6: authorized cross-instance editing presence and feature-gated CRDT collaboration.
Follow-up: block-level HTML protection, hosted-editor/CLI migration, and extensible insertion palette.
- [ ] Formalize a Domain-sequenced event outbox for Page, Revision, Path,
topology, policy, share, comment, reaction, quality, and federation events.
- [ ] Add Page/Spot/Street/Quartier subscription and change-notification rules.
- [x] Per-Principal Inbox, unread state, cursor pagination, topbar/modal/detail UI,
task/broadcast producers and current-authorization checks for supported types.
- [ ] General notification grouping/filtering/retention and new-producer authorization coverage.
- [x] Web Inbox, opt-in Web Push and daily task email; their delivery contracts are distinct.
- [ ] General channel subscriptions, webhook delivery, quiet hours and retry/dead-letter operations.
- [ ] Add recently edited and recently viewed Page projections; keep private
activity private and provide retention/opt-out controls.
- [ ] Add privacy-aware Page view counters with bot filtering, idempotent event
aggregation, time windows, and “most viewed” projections.
- [ ] Add Page reactions/likes without modifying immutable Revision payloads.
- [ ] Add document-quality assessments with rubric/version, reviewer identity,
score history, and machine/human provenance.
- [ ] Add Revision-anchored inline comments with text-range anchors, context
hashes, orphan/rebase handling, resolution state, and permissions.
- [ ] Add Page-level threaded comments below the document with moderation,
mentions, subscriptions, edit history, and soft deletion.
## Phase 7 — Knowledge City services
- [ ] Complete Campus landing, Domain catalog, controller status, announcements,
shared navigation defaults, audit, export, and delegated Campus governance.
- [ ] Complete Quartier charter, entrances, localized signage, curators,
announcements, scoped activity/search, subscriptions, health, merge/split,
export/import, and policy-default surfaces.
- [ ] Complete Street route intent, sections, Inbox, landing, activity,
maintenance queue, subscriptions, merge/split/promotion, and policy override.
- [ ] Complete Spot generated portal, curator workflow, dynamic sections,
watchers/digests, backlinks, shortest routes, health states, split/merge,
archive/successor, and machine-readable context payload.
- [ ] Add Campus/Quartier/Street/Spot notification and permission inventory
integration without turning locality into Page identity.
## Phase 8 — Controller disaster recovery and high availability
First slice: [Japanese DR design and operator runbook](ja/disaster-recovery.md).
This is **instance-scoped offline recovery**, not automatic controller HA.
DR-D1 follow-up is **design complete, implementation pending** (2026-09-27):
[authority/rejoin](ja/dr-authority-and-rejoin.md),
[protocol/synchronization](ja/dr-protocol-and-sync.md),
[delivery plan and acceptance matrix](ja/dr-delivery-plan.md).
No DB/schema/code changes were made for this design-only follow-up.
- [x] Specify separate controller/data authority, shared-writer coupling,
authenticated rejoin/probe, Learning state, checkpoint verification,
fenced handover/preempt, API contracts, rollout gates and acceptance tests.
- [ ] DR01–DR03: implement validated models, pure state machine and
observation-only authenticated probe; discovery must not grant write access.
- [ ] DR04–DR05: connect restart write gates and authority providers only after
the separately managed storage interfaces and all writer paths are agreed.
- [ ] DR06–DR10: connect consistent checkpoints, bounded resumable transfer,
epoch-aware control manifests, verified fencing and manual group transitions.
- [ ] DR11–DR13: certify preempt policy, scoped operator UI and failure drills;
automatic lossless failover requires an independently verified durability boundary.
- [x] Node-local primary/standby configuration; read-only SQLite startup without
migrations, key generation, scheduler, GC or managed-campus reconciliation.
- [x] Offline snapshot/verify/restore CLI: committed WAL via SQLite Backup API,
canonical files and keys, payload/reference verification, atomic new generations.
- [x] Offline operator-attested promotion, old-primary identity/data-loss checks,
promotion audit and explicit restored-job resumption. External fencing itself
is not implemented or independently verified by these commands.
- [x] Reject standby writes and stale reads; separate read/write readiness;
coordinate server and data CLI with an instance maintenance lock.
- [ ] Rebuild-free read-only organizer/CalDAV paths, standby UI/role indicators,
DR settings panel and instance-operator authorization.
- [ ] Online cross-store snapshot barrier, GC pins, resumable bounded delta
transfer, encrypted/signed generations, retention and measured restore drills.
- [ ] Restore controller sequence safely across accepted newer manifests; local
audit epoch alone does not solve managed-instance replay protection.
- [ ] Add controller manifest journal replication to a designated DR instance,
including signing-key rotation/escrow, encrypted backup, sequence verification,
last-known-good snapshots, RPO/RTO reporting, and restore drills.
- [ ] Add controller leader election or manual promotion with fencing so two
controllers cannot publish the same Campus sequence concurrently.
- [ ] Add managed-instance controller failover using a pinned set of controller
identities and an auditable promotion epoch.
- [ ] Separate process-local cache/session/job state from canonical state and
define shared/replicated backends required by multiple application nodes.
- [ ] Add health, readiness, graceful drain, idempotency, forwarded-origin
validation, and node identity endpoints for load balancers.
- [ ] Add an LB reference deployment with TLS termination, WebSocket support,
upload limits, no mandatory sticky session, and failure tests.
- [ ] Add an HA metadata backend or a fenced single-writer/replica design;
never mount the writable SQLite database concurrently from multiple nodes.
- [ ] Add Document Store replication/read-repair and background-job leases so
only one node performs a claimed task.
- [ ] Exercise node loss, controller loss, network partition, stale replica,
key rotation, and rollback in automated fault tests.
## Phase 9 — Content composition and extension runtime
- [ ] Evaluate tasks/tickets, an organizer with calendar/contact interoperability,
RSS/Atom reading and FreshRSS/newsboat integration, IMAP mail-to-Wiki and
incremental HTML editor improvements. Full WYSIWYG remains
an open decision, not a commitment. See the design-only
[organizer and editing ideas](organizer-and-editing-ideas.md); these are not part
of the current implementation batch. The detailed design has since been expanded
in the [Japanese handbook](ja/index.md), chapters 12–17; implementation remains
unchecked and follows [the dependency-ordered plan](ja/roadmap.md).
- [ ] Add a versioned transpiler registry that converts PukiWiki, Redmine,
DokuWiki, MediaWiki, and future source syntaxes into canonical HTML while
preserving the original source as immutable metadata or a related Revision.
- [ ] Complete Markdown round-trip guarantees and record compiler ID/version,
warnings, source map, and canonicalization result on every conversion.
- [ ] Add Excalidraw and draw.io canvas projections with immutable asset
freezing and configured hosted-editor actions.
- [ ] Add Kroki and PlantUML diagram adapters with bounded input, pinned renderer
version, cached immutable output, timeout, and network policy.
Existing bounded Kroki execution is now in Kunstzeal; renderer-version keys and complete
provider network/resource policy remain pending rather than marking this whole item complete.
- [ ] Add nwdiag as a bounded network-diagram projection and editor adapter;
support static SVG fallback, pinned renderer versions, and explicit source
import/save without executing authored code.
- [ ] Add a widget/oEmbed registry for Spotify, YouTube, SoundCloud, and future
providers with Domain allowlists, consent placeholders, CSP sandboxing,
privacy mode, and no arbitrary document-authored script execution.
- [ ] Add administrator-packaged widgets with signed manifests, semantic
versioning, capability declarations, resource quotas, and disable/rollback.
- [ ] Add a generated top-page composer using authorized queries, static blocks,
recent work, recent views, popular Pages, featured City entrances, and feeds.
- [x] Add initial diary/blog template/query/navigation projections over ordinary
Pages, not a new Page identity or date-dependent storage hierarchy. Registration
metadata follows Page UUID; atomic new-post composition, local Domain projection,
dedicated public article routes/navigation and RSS 2.0 are implemented.
Remaining federation/metadata/Atom work is in the [follow-up plan](diary-blog-plugins.md).
- [ ] Add directory-scoped RSS/Atom feed widgets with scheduler fetching,
conditional requests, sanitization, per-host limits, caching, and provenance.
- [ ] Add a bounded CSV table macro and CSV editor projection. RFC 4180-compatible
CSV remains the interchange baseline; ODS is a separate multi-sheet adapter.
- [ ] Add Reveal-compatible slide projection adapters for HedgeDoc and Slidev,
plus a CSS-only scroll-snap/print editor fallback with explicit import/save.
- [ ] Add custom theme packages with inheritance, tokens, preview, validation,
CSP-safe assets, versioning, export/import, and one-click rollback.
## Phase 10 — External storage, inspection, and ecosystem
- [ ] Define a Document Store backend interface preserving SHA-256 identity,
seek/range reads, atomic put-if-absent, integrity verification, and streaming.
- [ ] Add S3-compatible immutable Document storage with multipart upload,
server-side encryption, credential isolation, lifecycle policy, replication,
local cache, and read-after-write verification.
- [ ] Keep metadata, mutable configuration, and secrets out of the Document
Store; external object keys must never replace Document SHA-256 identity.
- [ ] Add asynchronous file inspection states (`pending`, `clean`, `suspicious`,
`malicious`, `error`, `expired`) and quarantine-aware authorization.
- [ ] Add MWDB/Karton integration through a bounded inspection adapter and event
queue; Viewer/download policy must fail closed where a Domain requires audit.
- [ ] Record scanner engine/version, signatures, hashes, timestamps, verdict
provenance, rescan policy, administrator override, and audit events.
- [ ] Add an extension catalog/API for projection, editor, transpiler, widget,
notification, inspection, storage, search, and theme providers.
- [ ] Add typed IPAM, DNS zone/record, and inventory/asset extensions with
approval-gated provider writes, dry-run, etag checks, secrets, audit, and
bounded queue execution.
- [ ] Add extension signature verification, capability review, compatibility
ranges, install/upgrade/disable/rollback, health, logs, and resource quotas.
Each phase requires schema migration, rollback analysis, and tests before the
next phase begins.
+427
View File
@@ -0,0 +1,427 @@
# Implementation status and hardening backlog
2026-11-08 reconciliation: use the [current Japanese implementation backlog](ja/implementation-backlog.md)
for actionable IDs, priorities, dependencies and acceptance gates, based on `e65ca56`.
This is a source/document audit, not a production audit or a new deployment.
For the consolidated current reading order, use the [Japanese handbook](ja/index.md)
and [decision register](ja/decisions.md). Dated entries below preserve history;
older statements about missing anonymous navigation, blog publication or RSS are
superseded by the newer slices, not additional unfinished implementations.
This document is a working handoff for the current repository state. It is
based on the implementation-roadmap, architecture documents, and a static
review of the handlers and stores. Checked roadmap items describe a usable
first slice; they do not imply production-ready federation, HA, or extension
security.
## Current state
2026-11-08 digest follow-up: private delivery history and explicit same-day retry
are implemented in source. Manual requests are queued for the existing scheduler;
attempt IDs fence stale completions, and consent/version/authorization are checked
again before SMTP. Detailed terminal attempts expire after 90 days, while daily
reservations and request receipts remain. Additive schema v4 and recovery rules
preserve duplicate prevention; production rollout remains deferred. See
[delivery history and migration](ja/task-digest-delivery.md).
2026-11-08 work history: owner-only bounded history, inline correction/cancellation,
immutable audit snapshots and project-level reports are implemented in source.
Cancelled spans leave the report but not the audit. Task/dashboard navigation and
Japanese/English desktop/mobile UI are connected. Schema migration v3 requires an
aligned DB backup before production rollout; do not hot-swap older binaries onto
the upgraded DB. See [contracts and remaining scope](ja/task-productivity-contacts.md).
The dated task/inbox paragraphs below describe earlier slices. VTODO, time blocks,
Pomodoro, active notifications, daily task email, private CardDAV and the organizer
dashboard have since been implemented. Their remaining work is shared access,
external connectors, conflict resolution and retention, not reimplementation of
the existing private interfaces. The [Japanese roadmap](ja/roadmap.md) tracks the
current order.
2026-09-25 task reminders: explicit per-task due/planned-date switches and a local
notification time now schedule owner-only Inbox reminders. Reservations are updated
in the source transaction; the bounded scheduler rechecks current state/authority,
skips previous-day catch-up, and commits Inbox insertion plus acknowledgment together.
Titles are projected under current authorization rather than copied into notices.
Notification badges refresh on visible pages without reload. Calendar reminders,
quiet hours, digests and external delivery remain pending. See [contract](ja/task-reminders.md).
2026-09-24 outbox follow-up: City now subscribes only to relevant page/revision/path/object
events, with independent consumer checkpoints rather than shared processed flags.
Per-type high-water marks avoid repeated full-history discovery. Task/project mutations
append metadata-only events in their source transaction; a bounded scheduled bridge
replays the earlier task journal idempotently. Notification delivery and retention are
not included. See [delivery contract](ja/outbox-delivery.md).
2026-09-24 task first slice: private `organizer.tasks` items/projects, Today/Inbox/
Backlog/Review/Completed/Archive UI, conditional updates, idempotent create,
immutable CAS revision payloads and UUID-based Wiki references are implemented.
Ownership is distinct from Wiki ACLs; even Domain administrators cannot read
another principal's tasks through these endpoints. The local atomic change journal
is now connected to the common outbox, but notification delivery, time-block/calendar integration, Pomodoro,
sharing and VTODO synchronization remain follow-ups. SQLite metadata and the
Document Store must be backed up together. See [contract and remaining work](ja/tasks-implementation.md).
2026-09-19 outbox foundation: `activity_outbox` now has a persistent, bounded
envelope with Domain-local monotonic sequence numbers, actor/target references,
idempotency keys, retry state and pending-page retrieval. The append API is
transaction-bound and deliberately does not authorize or project payloads.
Page creation and Revision writes now append through this envelope in their
source transactions. Path rename/delete and Object purge are also recorded;
Domain-local consumer checkpoints are durable and monotonic. A debounced City
consumer now reconciles pending URL events through the scheduler and advances
its checkpoint only after a successful plan. Recipient-scoped notification Inbox
storage and an authenticated list/read API are now implemented with idempotent
deduplication and keyset pagination; approval request producers, reviewer UI and
delivery workers remain pending, so this is not yet a completed activity pipeline.
Street/Spot automatic publication is opt-in through the governance dialog. Current
and future eligible heads follow the pinned Path boundary with live audience and
enabling-actor authority checks. Domain/Quartier/Street/Spot manual-only floors narrow
it; anonymous navigation/blog projection/RSS/share readiness follow the same policy.
Existing scopes stay unchanged. Reviewer/Inbox and policy federation remain pending;
see [automatic-publication.md](automatic-publication.md).
Local diary/blog projection first slice: Campus admins select source Domain and prefix
inside the entry dialog. Same-Campus/local placement only, no copied content; both
Domains' read policies filter private metadata; editing stays on the source Domain.
Target article writes are rejected. Anonymous projection indexes/calendars now derive
only from approved source Paths/Revisions, never private registration metadata.
Dedicated-domain links now open `/YYYY/MM/DD[/keyword]` on that host with sanitized
published bodies, calendar/month/adjacent-article navigation, target Domain branding
and `/rss.xml` (20 items). Registered titles now explicitly inherit Page publication
visibility; category/owner/registration date stay private. No cross-instance transport,
source configuration federation or automatic cross-Domain session sharing yet.
RSS now includes sanitized, absolute-URL HTML in `content:encoded` with bounded
generation and exact publication rechecks. Authenticated `/v/{blog|diary}/-/new`
pages reuse atomic composition. Post completion offers X sharing with configured
projection-domain URLs, target selection and unpublished-link warnings; no automatic
social posting, publication grants or credentials are added.
Domain entry provisioning is implemented in source: Campus-admin dialog per Domain,
explicit allowlisted settings, signed manifest v2 when settings are present, and
transactional apply together with Domain snapshot/checkpoint. Legacy manifests without
entry settings remain v1. Destinations must be upgraded before activating v2 settings;
signed enrollment capability proof and Campus-wide preflight are implemented.
Create-form integration, apply acknowledgments/freshness and control-release/GC remain
pending. Managed nodes cannot override entry settings locally. No new public grants.
Atomic diary/blog composition is now implemented in source: Markdown composer,
create/write checks inside the writer transaction, Page/Path/Revision/article/journal/
receipt atomicity and idempotent retries. Shared Revision insertion and asset freezing
are reused. Anonymous article-index approval, City/notification outbox dispatch,
receipt retention and application acknowledgments remain pending. Existing
two-step editor/registration is optional, not the only new-post workflow.
2026-09-17 URL/entry slice (source implementation): per-Domain diary/blog prefixes,
date/keyword URL planner, same-day multiple diary entries, and home_layout diary/blog
rendering at `/`. Placement label shortened; placement identity remains separate.
The planner is read-only and does not reserve Paths. Editor save and registration
remain separate in the legacy flow; public calendar approval is still TODO.
Existing article Paths are not migrated.
Diary/blog initial slice: `content.diary` and `content.blog` are separate trusted
built-ins with Domain enable/disable settings, Page UUID-bound metadata,
calendar/recent entries/categories, registration/update/unregister dialogs and
fresh read filters. Disabled by default; indices require authentication and
registration does not publish a Page. Public index
approval/feeds, comments and tDiary import remain explicitly deferred; see
[detailed design](diary-blog-plugins.md).
2026-09-16 anonymous navigation hardening: public root and virtual parents now
offer safe directory landing pages, with read-only `/v/browser` and separate
Page/children links. Trees support all/per-folder expand/collapse, name/Path
filtering and keyboard navigation. Responsive public headers, direct City exits,
bounded/abortable asynchronous listings, retry and no-JavaScript pagination are
implemented. No additional publication or approval policy is enabled.
2026-09-16 navigation follow-up: approved public Pages now expose filtered
children/tree/City navigation and breadcrumbs, without a download button.
Database generations invalidate bounded metadata caches; authenticated filters
remain request-scoped, including group revocation on unchanged cached data.
Private parent bodies/titles are not exposed through structural folders. See
[navigation](public-navigation.md). Hierarchical approval requirements and the
reviewer workflow remain [designed and TODO](publication-approval-inbox.md),
while the reusable recipient Inbox API is available but not populated by approval
requests. Approval request subjects, decisions, generation fencing and bounded
expiry sweeps are persisted, but reviewer eligibility and decision UI are not
exposed; the current explicit approval workflow remains in force.
2026-09-14 user/group and anonymous follow-up: named local user/group read or
read/write settings now intersect ancestor caps, resolve stable subject IDs and
observe live membership. Explicit bounded current-revision approvals enable a
separate inert anonymous semantic Viewer and raw-media path with revocation and
OGP. Existing visibility is unchanged until configured. Important first-slice
limits: authenticated ACLs freeze protected namespace/City structural changes;
anonymous edits require reapproval, and discovery/dependencies remain private.
See [implementation contract and remaining tasks](locality-access-publication.md).
2026-09-14 locality governance: scoped Quartier/Spot co-owner appointments,
versioned publication-cap drafts, transactional authority rechecks, local audit
records and themed Japanese/English settings dialogs are implemented. Domain
ceiling is in the City Map utility menu. The subsequent bounded anonymous slice
and coarse namespace fences are described above; complete impact-aware
mutation/dependency handling remains a next gate. See
[exact first-slice scope](locality-publication-ownership.md).
2026-09-14 P0 HTML remediation, extended 2026-09-23: normal
Viewer/home/profile/object/history/print and both Editor previews use the
`controlled-v2` allowlist with application CSP. External resources remain
default-deny; Domain administrators may approve exact HTTPS image URLs or
SHA-256-pinned CSS/JavaScript URLs through the shared Projection/CSP policy.
Canonical bytes are preserved; authored scripts/styles/frames are not activated.
Diagram cache reads recheck Domain/Path authority and sandbox SVG. See
[contract, compatibility and tests](html-projection-trust.md). Ownership/public
activation remains separate work, not implicitly enabled by this security fix.
2026-09-14 publication governance design: explicit Quartier/Spot owners may
administer publication only within superior ceilings; appointment authority,
cap-vs-publication semantics, mutation/alias fencing and the release sequence
are documented in [locality ownership](locality-publication-ownership.md).
The owner API/UI and draft policies have subsequently been implemented as above;
production visibility has not been changed.
2026-09-14 social-card follow-up: already-public semantic Pages, home layouts
and City views now emit OGP/X metadata with bounded canonical-text summaries,
safe absolute URLs and a common PNG cover (original images for image documents).
Enforcing domains retain authentication and suppress content-card metadata;
no visibility policy was changed. See [contract and limits](social-link-cards.md).
2026-09-14 City reference follow-up: parent-qualified readable Street URLs now
replace ambiguous name-only links in map/search/macros. Details require ancestor
read permission and fetch only the requested node; map focus and locality search
work for Quartier, Street and Spot. Literal Page link encoding is shared by the
map and its inspector. See [read integrity](city-read-integrity.md).
2026-09-14 City/search options follow-up: metadata-only Quartier/Street/Spot
matches appear ahead of Pages, with visible ancestor context and safe links.
Search now supports file categories, one extension, and Domain-local calendar
date ranges over creation/update times. Filters run before candidate limits;
City and Page reads share the existing deadline. See
[search options and their exact semantics](search-results.md#city-first-results-and-search-options).
2026-09-14 search timeout follow-up: corrected a production Path-first repeated
FTS scan and costly snippet generation. Input/time-bounded optional excerpts
now preserve search results; query-plan and realistic-volume regressions plus
read-only live-data timing cover the original blind spot. See
[search correction](search-results.md#timeout-regression-and-correction-2026-09-14).
2026-09-14 hardening follow-up: authentication and inherited authorization,
recursive mutations, federation snapshots and garbage collection now have
bounded, denial/concurrency-focused regression coverage. City generation and
PDF/archive metadata work are off the request/rendering path; search and history
have bounded, clearer projections. See the [execution ledger](hardening-execution-2026-09.md)
for validation/deployment and explicit deferrals. The subsequently corrected
normal HTML boundary is recorded above; broader publication gates remain open.
2026-09-14 City UI follow-up: Spot clicks now explore their courtyards, with
surrounding Page links, bounded nonoverlapping close-ups and a modal utility
menu. Manual title/selection/merge/move controls remain available. See
[City projection and its remaining limits](city-immersive-map.md).
2026-09-13 follow-up: execution and resource panels are now one monitor with only
running jobs. Completed jobs and lazy, paginated logs share a history panel.
Active queue counts avoid scanning completed history; job queries use indexed
status filters and keyset pagination. Legacy `panel=resources` resolves to the monitor.
The first resumed hardening task protects group owners from ordinary update
downgrades and checks last-owner removal within the DELETE itself (including
concurrent removals and manager authority against the target's current role).
Current follow-up adds viewport-responsive bookmark columns, persistent
background resource monitoring, and a separate trusted system-extension
catalog. Other system services are designed and sequenced in
[System extensions](system-extensions.md); third-party plugin execution is
not yet enabled. vim.wasm runtime work is paused at the user's request.
Latest UI follow-up: application asset authorization and `/e/` target decoding
are corrected; home bookmarks support admin management, section cards and
bounded asynchronous child links. Hosted Vim now explicitly loads its vimrc,
local font and pure-Vimscript navigation plugin. See
[implementation and remaining limits](home-bookmark-sections-vim.md).
Viewer shortcut follow-up: `e/f/v/c/u/s/p` and the four Ctrl-arrow navigation
bindings are implemented in `static/js/viewer-shortcuts.js`. They suppress
themselves while typing or inside dialogs; sharing copies the current canonical
URL and opens a user-confirmed X Intent. The X action is cataloged as the trusted
builtin `sharing.x` extension; it does not grant publication or create a share
credential. Permission shortcut behavior depends on the City governance context.
The repository currently has working first slices for:
- Campus and Domain scoped semantic paths, local/remote placement, controller
enrollment, signed manifest reconciliation, and the initial Knowledge City
topology/map UI.
- UUID/Object, immutable Revision, SHA-256 Document storage, language fallback,
asset HEAD/history, bounded text and JSON projections, archive/CSV/EML/PDF
adapters, and hosted code-server/vim.wasm/Excalidraw adapters.
- Password/TOTP sessions, OIDC, LDAP/LDAPS, and SMTP magic-link flows, Domain
capabilities, built-in campus/domain administrator groups, group permission
profiles, and a centralized endpoint policy classifier.
- Persistent event jobs with priority, retry, cancellation, concurrency
settings, preview generation, import jobs, and manual scheduler controls.
The current deployment is a single writable SQLite application node plus a
managed external-domain node. This is a supported development topology, not
HA: canonical metadata, session revocation, scheduler claims, and adapter
state are still process/instance local.
## Confirmed corrections (2026-09-14)
The [execution ledger](hardening-execution-2026-09.md) records verification and
deployment separately. Completed, regression-tested slices include:
- OIDC issuer JWKS signatures, algorithm/claim checks, browser-bound state,
bounded key/discovery fetches and account-binding collision rejection;
[verification contract](oidc-verification.md).
- SMTP connection/protocol deadlines, bounded replies, STARTTLS, mailbox/header
validation, one-use challenges and live membership checks;
[delivery limits](smtp-magic-link-hardening.md).
- Campus-admin provenance and transactional derived-membership revocation while
retaining independent local grants; [inheritance contract](campus-admin-revocation.md).
- Same-origin browser mutation protection, POST logout, typed browser/API
credentials, atomic session-state checks and request-scoped batch Path reads;
[request boundaries](browser-request-security.md).
- Domain scope for Object reads/saves/binds, exact exceptional Object grants and
atomic API creation provenance; [Object scope](object-authority-scope.md).
- Transactional complete-binding snapshots for recursive Path changes and
all-alias Domain-local Object purges; [mutation integrity](path-mutation-integrity.md).
- Atomic controller manifest sequence allocation and managed snapshot,
inherited-permission and checkpoint application; authority collision and
unsigned-downgrade rejection; [manifest integrity](federation-manifest-integrity.md).
- Private, per-result City map/details/macros/badges, recomputed visible
portals/counts/relations, local generation Worker, cancellation/cache bounds,
directional keyboard navigation and immersive courtyards;
[navigation contract](city-worker-navigation.md).
- Persisted scheduler projections for PDF page counts and non-ZIP archive
metadata, localized pending/retry UI and immutable Revision entry links;
[projection cache](projection-metadata-cache.md).
- Bounded compact search with full Paths, matching-source badges and safe
excerpts/highlighting; [search contract](search-results.md).
- Bounded metadata-only Object/Revision APIs and paginated history, retaining
array response shapes and explicit continuation;
[Object pagination](object-list-pagination.md), [Revision pagination](revision-listing.md).
- Literal URL encoding in all three children views, including filenames with
question marks, fragments, percent signs and non-ASCII characters.
- One-connection-safe City read transactions with deadlines/ceilings and literal
boundary matching; [read integrity](city-read-integrity.md).
- Atomic bounded zombie selection/purge, current-Domain provenance for HTTP
administration, retained global maintenance grace and orphan-writer guards;
[GC integrity](zombie-gc-integrity.md).
These are not a claim that every endpoint, derived projection or write path has
finished authorization and concurrency review.
### 2026-09-23: user-space audience controls
- `/u/{username}/_/edit` now separates profile metadata, avatar, Wiki-root and
descendant audiences. Descendant exact/subtree overrides can only narrow the
default, are CAS-versioned/audited, and follow atomic Path rename.
- A dedicated anonymous live-head reader supplies user profile, Viewer/raw,
children, scoped Tree and Browser projections without adding `/u/` to global
anonymous navigation. It rechecks the Domain public ceiling per request and
rejects aliases and UUID/Revision bypasses.
- Authenticated listings and direct Object/Revision authorization also enforce
private user-space audiences. Owner and Domain-admin recovery are explicit.
Focused revocation, alias, navigation and recursive-rename tests pass. See
[user-space publication](user-space-publication.md).
## Remaining gates and risks
### P0: authority and content trust
1. Finish authorization-policy/version fencing inside every write transaction.
Recursive binding snapshots protect namespace races, not a concurrent
revocation of the grants themselves. Typed decoded mutation targets,
handler-method closure and idempotency remain wider cross-handler work.
2. Complete publication-impact review and mutation fencing on the implemented
owner/draft-policy and [inert HTML boundary](html-projection-trust.md). Keep public
enabling gated on structural mutation/alias fences and common anonymous-read
policy/revocation tests. See [ownership design](locality-publication-ownership.md).
Extend denial tests to each new projection; CSP is not an authorization grant.
3. Local-email ownership, explicit external-account linking, provider subject
migrations and administrator last-owner suspension/transfer need dedicated
lifecycle workflows. Rejecting ambiguous email matches is not email proof.
4. Domain-to-Domain PSK identity/group/grant inheritance needs its own origin,
replay, conflict and revocation contract. The completed Campus-admin local
projection is not that cross-instance synchronization feature.
5. Audit remaining list/projection, asset and adapter paths with
denial-focused tests before claiming complete Domain isolation. Legacy
unknown-origin unbound Objects require explicit grants or trusted recovery.
HTTP zombie GC is now provenance-scoped; trusted global maintenance remains
a distinct instance authority rather than a Domain administrator operation.
### P1: City correctness and responsiveness
See [City review and reconciliation](city-review-and-reconciliation.md).
CITY-A through CITY-D remain partially open. Remaining work includes shallow
Path/portal inference, deterministic generated Spot name collisions,
idempotent no-op application, explicit generated/manual ownership, exclusion
tombstones, event watermarks and a transactional mutation outbox.
Worker generation and bounded courtyard DOM do not solve connected/fair road
allocation or all main-thread topology/layout work. Lazy locality/Page
projections and a directory index remain important for much larger Wikis.
The Viewer sidebar / paginated navigation API is also separate from the
already-implemented home layouts and scoped Tree View.
### P1: projection and storage scaling
1. Non-ZIP entry extraction still runs a bounded foreground decoder. Durable
entry artifacts need quotas, leases, Range semantics and archive-bomb policy.
ZIP central-directory parsing and nested/compressed TAR expansion need their
own bounded worker/index contract.
2. Native decoders have input/output/thread/time bounds, not hard memory/cgroup
limits. A weighted scheduler resource budget and decoder isolation remain.
3. Search FTS still contains historical revisions; current-only indexing,
Japanese infix search and sanitized plain-text indexing are separate tasks.
SQL deadlines/candidate bounds limit work, not unlimited recall.
4. Browser/children/Tree and some link projections still aggregate large Path
sets. Request-scoped ACL removes repeated grant queries but does not replace
a directory index, bounded SQL candidates or fully opaque continuations.
5. Large-binary restore and several trusted full-list helpers still hydrate or
enumerate complete data. HTTP Object/Revision pagination does not silently
change explicit CLI full exports.
6. Remote Document Store backends, read-repair, retention proofs, cross-instance
job leases, dead-letter operations and writable metadata fencing remain.
### Operations and intentionally deferred high-risk work
- Existing deployment is two application instances with separate SQLite state,
not a shared-writer HA service. DR/LB require replication, fencing, key custody
and recovery drills; no live SQLite-over-network-filesystem shortcut.
- Enrollment tokens currently present in compose must be moved to a selected
secret mechanism and rotated deliberately. No credentials were automatically
replaced, printed or copied as part of these repairs.
- Untrusted plugin execution/native helpers remain disabled/deferred until the
runtime capability/signature/isolation model exists. vim.wasm runtime work
remains on the user's explicit hold.
- Live S3/SSHFS/WebDAV/MWDB/Karton/SNMP/LDAP integration requires selected endpoints,
credentials and test services. No external service was provisioned implicitly.
## Ordered next work
1. Add complete publication-impact previews and structural mutation/alias fences
to locality governance; keep anonymous access disabled until all publication
gates and fresh activation confirmations are satisfied.
2. Complete remaining mutation/policy transaction fences and account ownership
lifecycle; preserve denial tests and audited provenance.
3. Finish bounded/lazy City and directory projections; implement idempotent
URL inference with generated/manual ownership before automatic event-driven
application.
4. Add the transactional Domain activity outbox, verified actor attribution,
durable consumer watermarks and recent-edit projections.
5. Then add delegated shared links, notifications/inbox, moderated comments,
rankings/quality, diary calendar/recent entries/comments and resumable tDiary
import. Their detailed contracts remain in [System extensions](system-extensions.md).
6. External connectors, isolated third-party plugins and DR/LB/HA follow only
after their authority, retention and operations prerequisites are met.
## Verification expectations
Every hardening slice must run `go test ./...`, include a regression test for
the denied case, and update the relevant design document and roadmap checkbox.
Deployment validation must confirm both app nodes are healthy and that a
foreground request cannot expose another Domain's Object, Revision, Path, or
adapter workspace.
+65
View File
@@ -0,0 +1,65 @@
# アクティブ通知とWeb Push
更新:2026-09-25。実装・隔離fixture検証済み。同日05:59 JSTに本番2インスタンスへ反映済み。[反映記録](../../operations/deployments/2026-09-25-active-notifications-calendar-mobile.md)。実機Push受信確認は未実施。
2026-11-01追記:配送処理を [Ordoschnellic](ordoschnellic.md) へ初期分離。
以下のUI・API・DB・購読を維持するソース変更であり、この分離自体は未デプロイ。
claim世代と購読更新競合を含む配送保証・残件は同契約に記録する。
## UIと設定
通知ベルを開いたInboxの「⚙ 通知設定」に集約する。アカウント・ドメイン・ブラウザごとにlocalStorageへ保存し、他ユーザの設定を継承しない。通知本文はlocalStorageに保存しない。
| 状態 | 動作 |
| --- | --- |
| フォーカス中 | 右下に8秒で消えるトースト。最大3個。入力フォーカスを奪わない。既定ON |
| 別タブ・別ウィンドウ/タブ終了 | 明示的に許可した端末へWeb Push。OSの通知UIを使用 |
| 再訪 | 前回訪問以降の未読がある場合、設定ONならInboxを自動表示。既定OFF |
| ダイアログ・入力を操作中 | 自動Inboxは保留し、次の確認時まで待つ |
新着は30件ずつ取得し、Inboxは「さらに表示」でページングする。訪問watermarkはサーバー時刻を使う。Web Locksと通知UUIDの短い履歴(最大200個)で複数タブのトーストを抑制する。Web Locks非対応時はbest effort。
音は「なし/ソフト/チャイム/ベル」と音量・試聴。Web Audioはブラウザの自動再生制限に従い、利用者の操作後に有効になる。カスタム音はフォーカス中のページ通知用。背景/タブ終了時の音はOS設定で決まり、サイト側から任意の音に置き換えることは保証しない。
ブラウザの通知許可は設定操作時のみ要求する。HTTPSと対応ブラウザが必要。iOS等はホーム画面への追加など端末側の条件もある。サービス・省電力・通信・キュー待ちに依存するため、定刻到達を保証するアラーム用途ではない。
## 配信経路と安全性
`notification_inbox`挿入のDB triggerで`notification_push_queue`へ原子的に追加する。Inboxの全種類(タスク期限・計画日、Pomodoro、アナウンス等)で同じ経路を使う。購読開始前の通知を一斉再送しない。
`/_/api/v1/notifications/push`:
- GET: VAPID公開鍵のみ。秘密鍵はDB内部に保存。
- POST: PushSubscription、focused、seen(最大30通知ID)。購読更新/閲覧中のack。
- DELETE: 現在の利用者が所有する購読を削除。
購読はDomain・Principal・**元の認証セッションのハッシュ**に束縛する。Bearer tokenは保存しない。claim時にセッション期限/失効、Principal、membership、Domain配置を再確認する。この時点でログアウト済み・無効化済みなら送信しない。claim後の失効と送信の競合は残る。最大8購読/利用者・Domain、30日lease(閲覧時に更新)。
VAPID秘密鍵は`notification_push_keys`に保存するため、DBバックアップを秘密情報として扱う。鍵を失った復旧では再購読が必要。端末の購読URL・鍵・通知本文をログへ出さない。
送信先はChrome/Chromium FCM、Mozilla Push、Apple Web Pushに限定。HTTPS/443のみ、資格情報・fragmentを拒否し、公開IPをDNS解決後に固定する。proxy環境変数とredirectは使用しない。プライベートIPへの接続は禁止する。Push本文は暗号化された通知UUID/受取人UUID/Domain UUIDのみ。件名・本文をPushサービスへ送らない。OSに表示するのも一般的な「新着通知」のみ。クリック後に通常の認可されたInboxで詳細を表示する。
Service Workerは`/static/js/notifications-worker.js`、scopeは`/static/js/`。fetch handlerやページキャッシュを持たず、既存のWikiページ/エディタに古い内容を返さない。notificationclickは同一originの`/v/notification`だけを開く。
## スケジューラと上限
イベント`notification.push.deliver`、定義`builtin.notification.push`。既定30秒、priority -3。設定画面のイベント設定から停止・間隔変更・手動実行でき、再起動で既存設定を上書きしない。
- 最大20件/run、全体25秒、送信要求5秒。DB transaction内でネットワーク通信しない。
- 60秒のclaim lease。期限切れを再取得できる。クラッシュ時はat-least-onceだが、ブラウザ通知tagを固定し積み上げない。
- 429/5xx/通信失敗は指数backoff、最大5試行。404/410は購読を削除。
- キューの有効期間24時間、Push側TTL300秒。期限切れキュー削除は最大500件/run、期限切れ購読削除は100件/run。
- フォーカスlease90秒。閲覧中は送信を延期し、ブラウザが一覧を受け取った通知のackでキューを消す。表示できなかったままタブが閉じればlease終了後に再評価する。
- 前面pollは30秒。Pushが有効な背景タブは定期pollを停止し、Pushと復帰を契機に更新する。
- sent/failed集計をイベントログへ記録。再送でもInbox本体は複製しない。
フォーカスを失う瞬間やネットワーク遅延にはraceがあり、OS通知を完全に抑制する保証はしない。常に通知権限とOSの設定が優先する。
## 検証・残TODO
- Go: 原子rollback、claim重複防止、focus延期とack、他人のack拒否、失効後の配信遮断、backoff、購読API認証、公開鍵のみ、送信先とP-256鍵検証、イベント設定保持。
- Chromiumの合成fixture: 前面トースト、再訪Inbox、設定モーダルの横幅、既存Inbox既読/バッジ。実通知サービスへは送っていない。
- TODO: 実機Chrome/Firefox/Safari/iOSでの購読・閉タブ配信・復帰・logoutを確認。Pushサービス障害テストの拡充、端末一覧/VAPID rotation UI、quiet hours/digest、種類別設定、OS通知の詳細表示opt-in、端末間の個人設定同期。
- TODO: expired/revoked購読の清掃メトリクス、Push queue深さ/遅延/失敗コードの管理UI。旧worker購読の移行手順。
参考:[MDN Push API](https://developer.mozilla.org/en-US/docs/Web/API/Push_API)、[MDN Notifications API](https://developer.mozilla.org/en-US/docs/Web/API/Notifications_API)、[webpush-go](https://github.com/SherClockHolmes/webpush-go)。
+43
View File
@@ -0,0 +1,43 @@
# 2. アドレス・保存・アセット
状態:基本モデルとローカル保存は実装済み。外部保存・保持管理の一部はTODO。
## アドレス表
| 内部指定 | 意味 | HTTP上の主な入口 |
| --- | --- | --- |
| `path:/manual/start` | ドメイン内の可変な住所 | `/manual/start` |
| `uuid:…` | Page/Objectの安定ID | `/o/uuid:…` |
| `rev:…` | 特定の不変Revision | `/r/view/…`、`/v/raw/rev:…` |
| `sha256:…` | 不変Document | `/_/obj/sha256:…` |
| `asset:image:logo` | ドメイン内の編集用HEAD | 管理者向け `/v/raw/asset:image:logo` |
通常エディタの正規入口は `/e/-/{path}`。従来の `/e/{path}` は互換入口です。生データは `/v/raw/path:/…` 等を使い、通常Pathと紛らわしい `/raw/` を増設しません。別ドメインへのリンクは宛先FQDNを含む絶対URLです。
## 保存トランザクション
1. 言語・MIME・保存対象と現在の書き込み権限を検証する。
2. 内容のSHA-256を計算し、`documents/sha256/` に不変Documentを永続化する。
3. Page、Revision、必要なPathと依存参照を確定する。
4. 派生処理へ通知する。共通outboxの全書き込み接続は後続課題。
Documentを先に書くため、DBのrollbackで未参照blobが残る可能性はあります。逆に、DBだけ確定して正本バイト列が存在しない状態は許しません。SQLiteのbody列は移行互換用で、Document Storeと二つの正本を作りません。
## アセットの可変HEADと固定参照
Asset名は `(domain_uuid, asset address)` で一意です。HEAD変更は旧Documentを上書きしません。HTML保存時に `asset:` を解決し、表示用参照をSHA-256へ固定します。編集用の由来は別属性で残し、次回保存時にHEADを再解決します。
`revision_document_refs` が依存Documentを記録します。過去Revisionの閲覧で最新HEADへ追従しないことが再現性の条件です。CLIと管理画面ではHEADを扱えますが、公開ページの依存物を自動公開する根拠にはなりません。
## 認可・条件付き操作
ハッシュURLは現在のDomainで読めるPageや許可された管理操作との関係を検証します。ETagと `X-PagesWiki-Object-UUID`、`X-PagesWiki-Revision-UUID`、`X-PagesWiki-Document-SHA256` は識別情報であり資格情報ではありません。
Asset更新は条件付き更新を使います。ただし同じ内容でMIME等だけ変わる競合には、HEADハッシュとは別の状態versionを導入する設計が必要です。
## TODO
- 保持期限・削除プレビュー・export中の保護を統合したDocument GC。
- Assetの状態version、共通監査outbox、依存関係込みの公開レビュー。
- 大容量復元のストリーミング、外部Document backend、破損検出とread-repair。
- CLIの全書き込みとWebの正本保存契約の統一。CLIは現在も信頼されたローカル管理経路で、HTTP認可を迂回する点を明示する。
+63
View File
@@ -0,0 +1,63 @@
# 5. 認証・グループ・権限
状態:パスワード・TOTP・セッション、OIDC、LDAP、SMTPリンク、グループと主要な認可経路は実装済み。全経路の監査完了を意味しません。
2026-11-01、純粋な権限・公開範囲の評価を [Schutzaria](schutzaria.md) へ初期分離。
認証・権限情報の読込・更新transactionは既存側に残し、API・ACL設定・公開範囲は変更しません。
承認の継承・reviewer定足数を実装済みとするものではありません。
## 認証から認可へ
認証器はPrincipal identityを証明します。その後、Domain membership、グループ・Role、対象操作のcapability、上位Localityの制限を評価します。OIDCのclaimやメールアドレスだけで管理権限を付与しません。
Principalにはactive / readonly / suspended / quarantinedの状態があります。membershipや状態の失効は次の要求で反映します。
LDAPはドメイン単位で設定します。検索用アカウントで`{{username}}`を
RFC 4515エスケープして一意なエントリを取得し、そのエントリのユーザーDNで
再bindできた場合だけ認証成功です。初回だけローカルの一意なユーザー名または
メールアドレスへ連結でき、その後は`entryUUID`やActive Directoryの
`objectGUID`など、設定した不変属性とprovider UUIDの組で識別します。DN、表示名、
メールアドレスを継続ログインの識別子にはしません。自動作成は任意で、作成時は
ローカルパスワードを持たない通常のmemberに限定します。
LDAP/LDAPS、StartTLS、証明書検証、追加CA PEM、検索Base DNとfilter、属性mapping、
既存アカウントの照合項目、保存済み設定によるログインテストを管理画面から設定
できます。bindパスワードは暗号化し、利用者パスワードは保存・ログ出力しません。
LDAPグループをPagesWikiのグループや権限へ同期する処理は認証と分離し、明示的な
mappingだけを適用する後続タスクです。
## セッションの目的を分離する
ブラウザはHttpOnlyのホスト限定Cookie、APIは明示的なBearer credentialを扱います。現行はブラウザ用とAPI用のトークン種別を分け、相互に流用しません。「全画面でlocalStorageへBearerを置く」構成にはしていません。
adapter、federation、認証challenge、将来のshareはそれぞれ別の検証系です。明示された無効なAuthorizationをCookieで救済しません。Cookie経由の変更には同一origin検証、ログアウトにはPOSTを使います。
OIDCはissuer・署名・claim・state/nonce/PKCE・browser bindingを検査します。SMTPリンクは短命・一回限りのchallengeで、配送遅延をログイン要求に持ち込みません。メール一致とメール所有の証明、外部identityの明示的な連結は引き続き整理が必要です。
## 権限プロファイル
| 表示用プロファイル | 許可する内容 | 許可しない内容 |
| --- | --- | --- |
| read | 閲覧 | 全変更 |
| read/write once | 未存在の対象の新規作成・アップロード、閲覧 | 上書き、再編集、削除 |
| read/write | 閲覧・作成・更新・rename・link | deleteの暗黙付与 |
| upload only | 指定範囲の未存在対象へアップロード | 一覧、閲覧、上書き、削除 |
| group admin / user | グループ内の管理上の役割 | それだけを根拠にした本文権限 |
削除は独立したgrantです。新規作成・uploadの可否は正規化後の最終Pathをトランザクション内で検査します。Pathを先に調べ、後で無条件上書きする実装は禁止です。
## 管理グループと所有者
`campus_admin` はキャンパスとドメイン配置、`domain_admin` はDomain設定・アカウント・グループを管理するビルトイングループです。普通のグループ管理者がこれらを自分へ付与できてはいけません。
Quartier・Spotのco-ownerは公開設定等の委譲された管理者です。所有者であることと、上位が許可していない公開を広げることは別です。
## TODO
- 全変更経路で認可versionと更新対象を同一トランザクションに固定する。
- 最後の管理者の停止・移管、メール所有、外部identity連結・解除。
- OIDC/LDAP外部グループの明示mapping、由来を保持した同期・失効・監査。
- 説明可能な権限総覧:付与元、継承、上位制限、対象操作、失効理由を表示する。
- PSK継承、adapter、Asset等を含む拒否・競合・再送テストの拡充。
個人用 `/u/{username}`、グループ用 `/g/{groupname}` は本文領域と `/_/edit` 等の管理領域を区別します。プロフィール画像変更を本文の無制限履歴と混同しません。ユーザー領域の所有者はプロフィール情報・画像・Wiki root・配下のaudienceを個別に狭められます。`private` は通常のDomain read grantより優先し、所有者と復旧権限を持つDomain管理者だけが読めます。匿名公開はDomainのpublic上限を越えられません。
+35
View File
@@ -0,0 +1,35 @@
# 10. ブログ・日記・公開ドメイン射影
状態:投稿・登録・カレンダー・同一インスタンス射影・公開記事・RSSは実装済み。コメント、tDiary移行、遠隔射影はTODO。
## 記事の実体とURL
ブログと日記は別のシステム拡張ですが、本文は通常Page/Revisionです。Domainごとのprefixに対し `{prefix}/yyyy/mm/dd`、任意keywordがあれば `{prefix}/yyyy/mm/dd/{keyword}` へ作成します。同日複数記事が可能です。
日付・著者・カテゴリ等はPage UUIDに紐づく記事メタデータです。Path変更で失わず、登録解除でPage本文は消しません。日付はcivil dateで、timezone変更時に過去の記事を勝手に別日へ移しません。
## 表示と投稿
月別は `/v/blog/yyyy/mm`、日別は `/v/blog/yyyy/mm/dd`。日記も同様です。`?own` とカテゴリを日付移動で維持します。`/v/blog` は全期間、日記の入口は当月が基本です。日付の旧query入口は正規Pathへ案内します。
タイトル下は日付とタイトルのコンパクトな一覧、月入力・適用・リセットはカレンダー上の一行です。大量記事では読みやすさとページ送りを優先します。
`/v/blog/-/new` と `/v/diary/-/new` は認証された投稿画面です。Page、Path、Revision、記事情報、journal、再送receiptを原子的に作成します。既存Pathは409、同じrequest_idの再送は重複作成せず返します。入力変更後の同じID流用も409です。
## 専用ドメインはコピーではなく射影
例:元ドメインの `/blog` を `blog.pages.anya.jp/` に表示します。現行は同一Campus・同一インスタンスに限ります。編集は元ドメイン、表示の名称・テーマは表示先ドメイン、専用記事リンクは `/yyyy/mm/dd[/keyword]` です。
元が非公開なら、射影の設定だけで公開しません。匿名一覧・月件数・記事本文は承認済みまたは自動公開条件を満たす内容に限定します。登録タイトルはPage公開を継承しますが、他の登録メタデータまで無条件に公開しません。設定不整合で別の内容へ黙ってfallbackしません。
## RSSとX
専用ドメインの `/rss.xml` は最新20記事を配信し、`content:encoded` に安全化したHTMLを含めます。リンクを絶対URLへ直し、正確な公開状態を再検査します。1記事1 MiB入力、合計8 MiB、5秒の生成制限を持ちます。
投稿後のXシェアはWeb Intentをユーザーが押す方式です。設定された専用ドメインのURLを選び、未公開なら警告します。投稿完了、公開完了、X投稿完了を同じ状態にしません。既にRSS等へ複製された本文は撤回できません。
## TODO
遠隔記事射影の署名・取得・失効、metadata公開範囲、Atom、keyset paging、outbox接続、receipt/孤立データ保持、予約公開、コメント・最近のコメント、tDiary import。
tDiaryは対応版・入力形式をfixtureで固定し、Ruby/eval/Marshalを実行して取り込む方法は使いません。旧URL、セクション、添付、コメント著者の出典を保持し、重複・部分失敗から再開できる取り込みを設計します。
+70
View File
@@ -0,0 +1,70 @@
# 予定・タスク共通CalDAV接続
2026-09-27追記:`organizer.tasks 0.10.0`で[MKCALENDARによるVTODOリスト作成](caldav-task-lists.md)を追加。共通直下のUUIDはVEVENTまたはVTODOリストを表し、種別でハンドラを選ぶ。予定本文の読み込みで判定しない。
2026-09-26実装。`organizer.calendar 0.2.0`/`organizer.tasks 0.9.0`。01:33 JSTに本番両インスタンスへ反映済み。[反映・検証記録](../../operations/deployments/2026-09-26-common-caldav.md)。
2026-09-27追記:[OneTask向け互換モード](caldav-onetask-compatibility.md)を実装、10:01 JSTに本番反映。通常は引き続き`calendar-access`を宣言せず、明示ONで発行した資格情報のみ接続判定互換のため宣言する。完全なRFC準拠ではない。実機の同期完了は未確認。
## 1. 接続と認証
共通接続先は `/_/api/v1/caldav/`。slash有無ともリダイレクトせずPROPFINDを処理する。rootはcurrent-user-principal/principal-URL/calendar-home-setを返す。Depth 1は許可された予定カレンダー(VEVENT)と個人タスクコレクション(VTODO)を返す。予定とタスクを同じICSリソースに混在させない。
| 対象 | URI |
| --- | --- |
| 共通home/principal | `/_/api/v1/caldav/` |
| 予定 | `/_/api/v1/caldav/{calendarUUID}/` |
| 個人タスク | `/_/api/v1/caldav/tasks/default/` |
| 既存タスク専用home(維持) | `/_/api/v1/caldav/tasks/` |
| 接続資格情報の発行/一覧 | `POST/GET /_/api/v1/caldav-credentials` |
| 失効 | `DELETE /_/api/v1/caldav-credentials/{id}` |
予定/タスク両画面の⇄ボタンから共通設定を開く。発行時に対象 `tasks`(タスクのみ)、`calendar`(予定のみ)、`all`(両方)とread/write、有効期間を明示選択する。初期選択は開いた画面の対象で、両方を自動付与しない。
Basicのユーザー名は資格情報ID、パスワードは発行時のみ表示するランダム値。従来と同様、秘密値はSHA-256のみ保存し、閉じたダイアログからも消去する。通常ログインセッションでのみ発行/失効できる。DAV資格情報をWikiやJSON APIのログインとして利用することはできない(匿名公開APIを非ログイン扱いで参照できることとは別)。HTTPSを使う。
既存の資格情報と旧管理APIは維持。`organizer_dav_scopes`に対応行がない資格情報は必ず`tasks`として扱う。既存資格情報を予定アクセス付きに更新するAPIは設けず、新規発行と旧資格情報の明示失効で切り替える。新規の予定/共通秘密値は`pw_dav_`、既存互換タスク秘密値は`pw_taskdav_`。古いバイナリは`pw_dav_`を拒否するため、切り戻し時にも予定限定資格情報がタスク権限へ化けない。
## 2. 認可
- 資格情報をドメイン/所有者/期限/失効状態に結び、principalとdomain membershipの状態も確認する。
- Scopeは対象種別の上限であり、他人のカレンダーを共有する権限ではない。既存のカレンダーowner/member/publicポリシーを別途適用する。
- 共通homeの探索結果はscopeと有効な拡張機能で絞る。片方の拡張が無効でも、他方の接続設定・探索は動く。
- 予定の書き込みは資格情報のwrite許可、active状態、カレンダーのCanWriteを全て必要とする。外部ソース由来カレンダーは従来通り書き込み不可。
- 予定変更時はカレンダーmetadata lock内でACLと資格情報の失効・期限・write・scopeを再確認する。タスクは既存のDB transaction内再確認を維持する。
- API・DAV応答はprivate/no-store。通常の匿名公開予定の読取経路は維持し、無効なBasic資格情報を匿名扱いにフォールバックしない。
## 3. 予定のresource名と正本
Web作成分は従来の`base64url(UID).ics`を維持。CLIが作成する任意の安全なASCIIファイル名(英数字・`.`・`_`・`-`、末尾`.ics`、最大700 bytes)を受理する。UIDとファイル名は別の識別子。
CLI由来のUID→filenameは`calendar.json`の`dav_names`として保持する。新規resource予約と月別ICSを同じmanifest置換でcommitする。DBだけのaliasにはせず、索引再構築・カレンダー設定変更でも保持する。削除時は対応aliasも削除する。同じcalendar内のUID重複、filename衝突、既存UIDの差替えを拒否する。
既存のWeb用canonical URLも引き続き解決する。WebビューアはfilenameをUIDとしてbase64 decodeする前提を廃止し、既存JSONイベントAPIの`?resource={filename}`でサーバー側解決する。`?uid=`も維持。ETag一致時には本文を取り直さない。
新規PUTは`If-None-Match: *`を推奨するが、chroncal互換で完全な新規identityに限り条件なし作成も可能。既存更新にはIf-Match(未指定428、不一致412)を必須とし、並行作成でも上書きしない。DELETEは予定の削除、タスクのアーカイブという既存の意味を維持する。
## 4. 取得上限と互換性
- 従来のtime-range付きREPORTとETagを利用したWeb表示を維持する。
- 初回同期用の全件calendar-query(VCALENDAR全体、またはVEVENT指定)を追加。metadata lock内で完全なsnapshotを取得し、最大10,000 resource/32 MiBを超えれば507で全体を失敗させる。黙った切り捨てはしない。
- PROPFIND Depth 1による予定resource列挙、全項目calendar-data選択にも対応。無制限Depth、部分projection、未対応filterは拒否する。
- 1 resource上限256 KiB、multigetは200 href、DAVリクエストは20秒上限。既存のtime-range上限370日も維持する。
- sync-collection/sync-tokenは未対応。405を返し、chroncalはcalendar-queryへフォールバックする。完全なCalDAV適合を示すcalendar-accessは広告しない。
- 新規予定のiCalendar対応範囲は従来のまま:単独VEVENT、UTC DATE-TIMEまたはDATE。TZID/VTIMEZONE、浮動時刻、繰り返し、VALARMは書き込み未対応。chroncalの`--timezone`で出力されるVTIMEZONEも拒否する。情報を黙って欠落させる変換はしない。
## 5. 検証と運用
隔離したfixture、HOME、XDG、DB、資格情報のみでchroncal 0.10.0の共通探索、予定とタスクの取得、予定作成/CLI編集/Web変更取得/削除、タスク作成/完了/Web変更取得/アーカイブを検証。vdirsyncer/Todomanの既存タスク回帰も実施する。
権限scope、旧資格情報の移行既定値、read-only、失効、アカウント停止、拡張無効化、既存上書き拒否、並行作成、ACL取消、DB索引再構築、取得上限、Webのresource解決を試験する。ブラウザでは予定・タスク両方の設定UI、発行/失効/秘密値消去と、CLI filenameの予定描画を確認する。
本番の外部取り込みカレンダーの本文を調査・試験に利用しない。反映確認はhealth、未認証の経路、静的資産、エラーログ件数に限定する。
## 6. 後続TODO・切り戻し
- [ ] losslessなTZID/VTIMEZONE、繰り返し・例外、アラームのDAV書込契約。正本を拡張する際は外部ICS取り込みの投影と混同しない。
- [ ] sync-token/tombstone/変更journalによる大規模差分同期。
- [ ] 資格情報ごとのcalendar ID制限、カレンダー作成・属性変更のDAV対応。
- [ ] macOS/iOS等のGUI CalDAVクライアント実機検証。
- 旧バイナリは追加DB表を無視できるが、calendar manifestを更新すると未知の`dav_names`を失う。**CLIで予定を作成した後は、旧バイナリでカレンダーを編集しない**。緊急切り戻しではカレンダー拡張を無効化して修正版を待つか、整合したDBと正本のバックアップ復元を別途承認の上で行う。DBのみ復元し正本を過去に戻したつもりにしない。
@@ -0,0 +1,34 @@
# DAVx⁵/Tasks.orgの互換性調査
2026-09-27。利用者指定の公開ソース`bitfireAT/davx5-ose`を読み、commit `167dc85f211119db6a98596c08f64e5df32ca679`(2026-09-25)を基準に確認。インストール済みアプリのバージョンとは同一と仮定しない。Android実機でDAVx⁵を実行した検証ではない。
## 同期方式
- `core/src/main/kotlin/at/bitfire/davdroid/sync/TasksSyncManager.kt`:タスクは`PROPFIND_REPORT`固定。
- `resource/remote/BaseWebDavCollection.kt`:Depth 0のPROPFINDでsupported-report-set、sync-token、getctag等を取得する。
- `resource/remote/ResponseExt.kt`:sync-tokenを優先し、なければgetctagを使う。両方なければnull。
- `sync/SyncManager.kt`のsyncRequired:状態nullは「同期必要」と判断する。両方ないこと自体で同期を拒否しない。
- `sync/TaskSyncer.kt`と`resource/remote/CalDavCollection.kt`:VTODOのcalendar-queryを送り、ETagで比較し、必要なリソースをcalendar-multigetする。タスクqueryにtime-rangeは指定していない。
- カレンダーは、期間制限やsync-collection未対応の場合PROPFIND_REPORTを選ぶ。未対応のsync-collectionを広告してはいけない。
したがって、提示された`XTaskApp / CodeSpark / SyncTaskListsSend`の「sync-token or ctag必須」というエラーは、調査したDAVx⁵の同期コードの挙動とは異なる。端末上でどのアプリが直接CalDAV接続しているかを区別する必要がある。
## PagesWikiとの差分と残件
- 既存リストの単純VTODO同期に必要なREPORT、ETag、multigetは既存実装にある。実機確認は未完了。
- DAVx⁵のリスト新規作成は`DavCollectionRepository.generateMkColXml`でDAV:resourcetypeを常に付ける。現在のPagesWikiの限定MKCALENDARはこれを未対応propertyとして拒否する。VTODO専用で正しいresourcetypeを受理する回帰試験と実装が必要。
- 作成時には説明・色・タイムゾーンや複数component-setも付く場合がある。受理する項目は正本保存・PROPFIND往復まで設計し、未知の意味を黙って破棄しない。
- 時刻付きVTODO、繰り返し、VALARMはPagesWikiの現行限定プロファイルでは未対応。DAVx⁵が生成できる全タスク形式に対応済みではない。
- リストの削除/属性編集、差分sync-tokenも別途残件。
## 今回の状態
追加途中だったctagのローカル変更(追加表・更新フック・応答property)は利用者の指示を受け取り下げた。本番には反映していない。取り下げ後の名前付きリスト・認可・原子性テストは成功。本番は0.10.0のまま。
利用者はTasks.orgを試す予定。DAVx⁵経由ではDAVx⁵のタスクアプリにTasksを選択し、同期対象の既存リストを有効化する。同じアカウントをTasks.org側に直接CalDAVとしても追加すると、Tasks.orgはDAVx⁵データを無視するため二重設定に注意する。初回は件名・日付だけのタスクで両方向を確認し、複雑な形式は分ける。
## 参照
- [調査対象の固定commit](https://github.com/bitfireAT/davx5-ose/tree/167dc85f211119db6a98596c08f64e5df32ca679)
- [Tasks.org公式:DAVx⁵連携](https://tasks.org/docs/sync_davx5/)
- [DAVx⁵公式:タスク連携](https://manual.davx5.com/tasks_notes.html)
@@ -0,0 +1,73 @@
# OneTaskのcalendar-access判定への互換対応
最新追記:利用者のスタックトレースにより`SyncTaskListsSend → CreateCalendar → MKCALENDAR`のVTODOリスト新規作成失敗と特定した。[複数タスクリスト実装](caldav-task-lists.md)で対応する。以下の「MKCALENDAR未対応」は0.9系時点の調査記録。0.10.0で明示的なVTODOリスト作成を追加するが、全機能のRFC準拠は引き続き宣言しない。
## 2026-09-27 接続診断の追記
利用者のcurl応答で、共通入口の認証済みOPTIONSが204、`DAV: 1, calendar-access`、`X-PagesWiki-CalDAV-Profile: limited-compatibility`となることを確認した。資格情報の再発行を繰り返す段階ではない。別アプリOneCalendarでの追加成功に続いて、**OneTask側も追加できたと利用者から確認された**。追加順序によるものか再試行によるものかは未確定であり、カレンダーを先に追加することを必須手順にしない。アカウント追加とVTODOの同期・更新成功は分けて扱う。
従来のアクセスログにはHTTPステータスがなく、10:13 JST頃の共通入口PROPFINDが成功したか判定できなかった。CalDAV経路だけ、次の固定分類による診断ログへ置き換える。
- method、target(home/task_home/task_collection/task_resource/calendar_collection/calendar_resource/well_known)、status、duration、request_id。
- auth(none/basic/malformed_basic/bearer/other/multiple)、outcome(未判定・Basic成功/拒否・session成功・匿名・欠落・不正・複数Authorization)。
- challengeの有無、限定互換宣言の有無。
- 資格情報ID・ユーザ名・秘密値・Authorization値・Cookie・User-Agent・リソース名・query・XML/ICS本文は出力しない。認証拒否の詳細な秘密値照合結果も公開しない。
認証・ACL・プロトコル応答は変更しない。401なら認証、403なら権限/未対応要求、207ならクライアントの探索解釈を切り分ける。ただし207でも個別propstatは404になり得るため、207だけで必要プロパティが全て成功したと判断しない。認証省略や権限緩和で迂回しない。外部取り込み予定にはアクセスせず、実データ本文なしのログと隔離fixtureで調査を継続する。
診断ログ追加は10:16〜10:17 JSTにapp/app2へ反映。認証済み共通入口PROPFINDの207を確認。MKCALENDARの404も観測したが、実行アプリはログから特定できない。外部クライアントからのコレクション新規作成は未対応であり、既存タスクの作成・更新とは別課題として残す。未実装のMKCALENDARを成功扱いしない。
その後「追加はできたが、タスクの中身が見えず`calendar or event not found`」との報告。10:17:27のMKCALENDAR404と同じ文言を既存カレンダーハンドラが返す。10:18:28にはtask_collectionへのPROPFIND207/basic_acceptedも観測。現在のログにタスク本文取得のREPORT/GET失敗はなく、一覧表示の不具合とMKCALENDAR未対応の関係は未確定。「PagesWiki Tasks」が見えるか、エラーが同期時かリスト新規作成時かを利用者へ確認中。未対応メソッドを存在確認より前で拒否して誤解を招く404を避けること、VTODO探索の階層とクライアント相互運用を検証することを残件とする。接続成功のみで同期解決済みにしない。
2026-09-27実装、10:01 JSTに本番両インスタンスへ反映済み。[反映記録](../../operations/deployments/2026-09-27-onetask.md)。**OneTaskアカウント追加は利用者確認済み。ただしタスク表示は未解決。互換ONで発行した資格情報が必要。**
## 原因と今回の境界
OneTaskはOPTIONS応答に`calendar-access`がない場合、`This CalDav server does not support calendar-access`として探索を停止する。従来のPagesWikiは限定的なCalDAV実装のため`DAV: 1`のみを返していた。これは認証失敗やPROPFINDの失敗ではない。
[RFC 4791 §5.1](https://www.rfc-editor.org/rfc/rfc4791.html#section-5.1)では`calendar-access`は全MUST要件への対応宣言である。現在のPagesWikiはその全てを実装していない。従って本変更は**RFC完全準拠への変更ではなく、利用者が明示的に選ぶ接続判定の互換モード**である。付加的な独自ヘッダでこのRFC上の意味を変えられるわけではない。通常モードは今まで通り宣言しない。
## 設定と保存
カレンダー/タスクの⇄設定で認証情報を発行するとき、「OneTask互換モード(calendar-accessを宣言)」を選ぶ。初期値はOFF。限定実装と未対応機能を同じ画面に明示し、一覧にもOneTask互換と表示する。
管理API `POST /_/api/v1/caldav-credentials` の追加フィールド:
```json
{"label":"Android OneTask","scope":"tasks","write":true,"days":90,"calendar_access_compat":true}
```
従来APIも同じフィールドを受理する。scope(対象)とwrite(権限)は別のまま。互換設定で予定への権限や書き込み権限を追加しない。既存認証情報は自動変更しない。公開してしまった秘密値を再利用せず、旧認証情報を失効して新規発行する。
DBには`organizer_dav_compat(credential_id)`を追加。発行と互換フラグを同じtransactionで保存し、資格情報削除triggerで関連行を削除する。secret自体は従来どおりhashのみ。通常のログインセッション以外で発行できない。
## 応答
認証成功後、対象domain/owner/資格情報ID/有効期限に対応する互換行がある場合のみ、共通home・task home・task collection/resource・calendar collection/resourceで以下を返す。
```http
DAV: 1, calendar-access
X-PagesWiki-CalDAV-Profile: limited-compatibility
```
既存資格情報・互換OFF・Bearer/Cookieセッション・匿名要求は通常のまま。User-Agentから推測して勝手に互換設定を有効にしない。DB確認エラー時は通常応答に倒す。認証・scope・期限の失敗は401のままであり、失敗を成功応答に偽装しない。
OPTIONSの204は維持。Allowには実装しているメソッドのみを示し、MKCALENDAR・calendar-schedule・sync-token等を追加宣言しない。task collectionのPROPFINDにはcurrent-user-privilege-setとcurrent-user-principalを追加し、読み取り専用をwriteとして返さない。
## データと検証
未対応のTZID/日時付きVTODO期限/繰り返し/VALARM/部分projectionやfilterは引き続き明示的に拒否する。互換設定はこれらを変換・削除して保存を通す機能ではない。ETag必須の既存更新とACLも維持する。
`TestDAVCalendarAccessCompatibility`は報告されたOPTIONSの判定を再現し、その後の探索・取得・更新・削除、通常モード維持、User-Agent偽装無効、read-only、scope、失効・期限、管理API隔離、未対応繰り返し拒否を確認する。これはOneTaskアプリ自体の実行試験ではない。
chroncalの共通接続fixtureは互換ONで、旧タスク接続とvdirsyncer/Todomanは通常モードで回帰試験する。Chromiumでは設定の初期OFF、明示ON発行、一覧反映、失効、秘密値消去、タスク画面4サイズを確認する。すべて隔離された合成データを用い、本番資格情報・外部カレンダー本文は使わない。
2026-09-27の結果:全Goテスト成功。chroncal 0.10.0/vdirsyncer実行fixture成功。`TestCalendarChromiumSmoke`と`TestTasksChromium`(1440/1024/390/320幅)成功。JS構文・`git diff --check`成功。本番へのデプロイ、認証情報の発行・失効はこの作業では行っていない。
## 運用・残件
- [x] 本番への反映(2026-09-27 10:01 JST)。DBは追加表とtriggerのみで破壊的変更なし。旧バイナリは互換設定を無視するため、切り戻すとOneTaskの接続判定は再び失敗する。
- [x] OneTask実機でアカウント追加(利用者報告)。
- [ ] OneTask実機で一覧表示・作成・完了・変更・削除を確認する。一覧表示に問題報告があり、接続成功を全操作成功と扱わない。
- [ ] 実機で次の要求が失敗する場合は、Authorization・本文を除去したメソッド、対象経路、status、画面エラーを基に限定fixtureを追加する。
- [ ] 完全なRFC対応を目指す作業は共通CalDAVの残件(時間帯・繰り返し・filter等)として別途継続する。
+53
View File
@@ -0,0 +1,53 @@
# CalDAVタスクリスト作成とプロジェクト同期
**後続仕様**:0.11.0で通常プロジェクトも個別リストへ統一し、条件なし上書きを受理する。以下は0.10.0の記録。[新しい契約・移行と競合通知TODO](task-dav-project-unification.md)を参照。
2026-09-27。`organizer.tasks 0.10.0`。OneTaskの`SyncTaskListsSend`が`MKCALENDAR`でVTODOリストを作成する実例に対応する。認証エラーではなく、旧実装が未知のリストUUIDをVEVENTカレンダーとして検索し404にしたことが原因。従来の固定リストと認証・ETag・権限を維持し、成功応答だけを偽装しない。
10:44〜10:45 JSTに本番両インスタンスへ反映済み。[反映記録](../../operations/deployments/2026-09-27-task-lists.md)。OneTask実機での再同期確認は利用者待ち。
## 1. 正本とURI
- `MKCALENDAR /_/api/v1/caldav/{UUID}/`でVTODO専用リストを作成する。要求URIをそのまま維持し、別URIへのリダイレクトは行わない。
- 1リストを1つの個人プロジェクトに対応付ける。表示名はプロジェクトのTitle、所属タスクはProjectUUIDで決まる。外部作成分はWebでも同じプロジェクトとして管理できる。
- URIのUUIDとプロジェクトのUUIDは別の識別子。正本プロジェクトの`dav_collection`にURIのUUIDを保存し、通常の版・CAS文書・変更journalを使用する。
- DBの`organizer_dav_collections`はdomain/pathの一意性・routing索引。プロジェクト正本の情報から再構築できる。通常JSON APIから`dav_collection`を注入・変更できない。
- 従来の`/_/api/v1/caldav/tasks/default/`は維持。DAV専用リストを持つプロジェクトのタスクだけを除外し、それ以外の既存タスク・通常プロジェクトは引き続きここに表示する。既存プロジェクトを勝手に外部リスト化しない。
- 名前付きリスト内のリソースは`/_/api/v1/caldav/{UUID}/{filename}.ics`。Webで所属プロジェクトを変更すると、元のリストから消え、移動先に同じUID/filenameで現れる。これはCalDAV MOVE対応ではない。
- 共通homeおよび旧task homeの探索で、利用者自身の有効な名前付きリストを返す。タスク拡張だけが有効な場合も使える。
## 2. 作成契約
[RFC 4791 §5.3.1](https://www.rfc-editor.org/rfc/rfc4791.html#section-5.3.1)に従い、作成は一括成功か全拒否とする。対応範囲は明示的なVTODO component-setとdisplayname。VEVENT/混在/未対応property/不正XMLを黙って無視しない。
- UTF-8 XML、application/xmlまたはtext/xml。64 KiB、既存XMLの深さ・要素数制限を適用。
- component-setはVTODOのみを必須とする。displayname省略時はURI末尾を用いる。表示名の長さ等は既存プロジェクト検証を適用。
- 作成成功201+Location、private/no-cache/no-store。既存URIへの再作成は405+DAV:resource-must-be-null。再送で別プロジェクトを増殖させない。
- domain/pathは他利用者分も含めて一意。既存VEVENTカレンダーのディレクトリを上書きしない(存在だけ検査し、予定本文は読まない)。
- 1利用者・1ドメインにつき最大256リスト。上限超過507。削除済みとしてURIを再利用する契約はまだ設けない。
- タスクscopeまたはall、write権限、active membership/principal、拡張有効状態を検証。作成transaction内でも資格情報の失効・期限・権限を再確認する。
- 匿名作成・匿名リスト閲覧は不可。他利用者のリストは404、カレンダーだけの資格情報はタスクへ使えない。
## 3. タスク同期
PROPFIND/calendar-query/calendar-multiget/GET/HEAD/PUT/DELETEを既存実装と共通化し、リストごとに完全なsnapshotを返す。別リストのhrefをmultigetに混ぜても個別404にする。
UIDとfilenameの一意性はリスト内。異なるリストでは同名・同UIDを許可する。DB検索索引のみ、正本上のUIDでは許可しないNULを区切りにプロジェクトUUIDを付与し、既定リストの任意UIDと衝突しないようにする。外部ICSや返却URLに索引用の接頭辞は出さない。
Web編集時もプロジェクトのリストidentityとタスクのDAVメタデータを保持する。既存更新はETagとversion CAS、DELETEはタスクのアーカイブ。write transaction内で所属リストを再確認する。リストをアーカイブした場合、タスクを既定リストへ漏らさず、DAV上ではそのリストを非表示にする。
## 4. 検証
- OneTaskの提供したMKCALENDARのXML形状を合成fixtureで再現。201→探索→PUT→GET→REPORT→完了→Web変更→削除。
- 二つのリストの同UID/filename、相互混入防止、multigetの範囲制限、Web所属変更、索引再構築。
- 他利用者、read-only、calendar-only、匿名、拡張無効、VEVENT URI衝突、並行二重作成、失敗時にプロジェクトを残さないこと。
- chroncalの名前付きリスト往復同期。旧固定リスト・共通VEVENT接続・vdirsyncer/Todomanの回帰。
- Webタスク/カレンダー画面のブラウザ回帰。実機OneTaskでの最終同期確認は利用者の再試行で行う。
- 本番の資格情報・タスク・外部取り込み予定をfixtureに使用しない。観測は秘匿済みログとhealthに限定する。
## 5. 残件と切戻し
- リストのPROPPATCHによる名称・色変更、リストDELETE、DAV MOVEは未対応。リスト名はWebのプロジェクトで変更可能。リストDELETEを全タスク削除として仮実装しない。
- DTSTART/DUEの時刻、TZID、繰り返し、VALARM、複雑filterは従来の限定VTODO契約を維持。sync-tokenによる差分同期も別件。
- OneTaskが追加のproperty・filter・日時形式を送る場合は実要求に基づき、情報を落とさない形で対応を追加する。アカウント追加成功やCLI試験だけでOneTaskの全操作成功とはしない。
- **名前付きリスト作成後、0.9系の旧バイナリでプロジェクト/タスクを編集しない。** 未知のdav_collectionが失われたり、旧既定リストへ混入し得る。緊急切戻しはタスク拡張を停止してから行う。DBだけの無断巻戻しでユーザーの新規タスクを消さない。
@@ -0,0 +1,15 @@
# カレンダー・Cityのジョブ障害是正(2026-09-22)
## City
URL自動生成はSpot名をPathの同じ深さから作るが、Spot名の一意制約はStreet内ではなくDomain全体だった。別Streetで同名のSpotやundefinedを生成するとreconcileが失敗し、outboxが未完了のまま再試行されていた。
Pathによる既存要素の再利用を優先し、新規作成時に名前が衝突する場合のみPath由来の安定hashを識別名に追加する。表示タイトルは維持する。別Pathの要素を名前だけでマージしない。QuartierとStreetの正規化名衝突も同じ方法で回避する。再実行で要素が増殖しないことを合成データで検証する。
残タスク:手動作成との同時実行競合のリトライ、繰り返し失敗の再投入backoffと警告集約。過去の失敗ログは成功へ改竄しない。
## 外部カレンダー接続
DNS検証後に先頭IPだけへ接続していたため、その経路が利用できない場合に失敗した。他の検証済みIPへ再試行し、DNS/TLS証明書/タイムアウト/安全制限をURLやトークンなしで分類する。localhostとprivate IP直指定は設定段階でも拒否する。
Outlookへの実接続・予定内容の調査は行わない。ユーザー指定により、以降の検証は合成データのみとし、取り込まれた外部予定にはアクセスしない。一般的なOutlookフィードのTZID、有限RRULE、EXDATE/RDATE、移動・取消を含むRECURRENCE-ID、ゼロ時間予定、Microsoft終日予定を有限期間へ射影する。対応外形式は前回snapshotを維持して失敗させ、通信成功と形式対応を混同しない。
+47
View File
@@ -0,0 +1,47 @@
# 14. 予定・iCalendar・CalDAV・連絡先
状態:カレンダー部分は2026-09-22にローカル正本・Plan UI・限定CalDAVを実装開始。最新の契約・実装範囲・残作業は [カレンダー詳細設計](calendar-implementation.md) を参照。2026-11-01に個人アドレス帳のCRUD・vCard入出力・neomutt検索、2026-11-02に個人CardDAV提供・履歴復元・一括exportを実装した。[個人アドレス帳の実装契約](task-productivity-contacts.md)と[CardDAV運用](../../operations/carddav-contacts.md)を以下の旧URI案より優先する。共有アドレス帳と外部CardDAV connectorは引き続き設計案。
## 四つのレイヤー
| レイヤー | 役割 | PagesWikiでの扱い案 |
| --- | --- | --- |
| Wiki URL | 人が開く説明・知識への入口 | rename可能なPage Path |
| iCalendar | 予定・タスク等の交換形式 | export/import、UIDと元データを保持 |
| CalDAV | カレンダー資源を扱うWebDAV拡張 | 読取remote connectorと限定VEVENT/VTODO提供面は実装済み |
| CardDAV / vCard | 連絡先の通信 / データ形式 | 個人address book提供は実装済み、共有・外部connectorは後続 |
iCalendarとCalDAVは同義ではなく、`.ics`を配信するだけでCalDAVサーバーにはなりません。vCardとCardDAVも区別します。[RFC 5545](https://www.rfc-editor.org/rfc/rfc5545)、[RFC 4791](https://www.rfc-editor.org/rfc/rfc4791)、[RFC 6350](https://www.rfc-editor.org/rfc/rfc6350)、[RFC 6352](https://www.rfc-editor.org/rfc/rfc6352)。
## identity対応案
local item UUID、calendar UID、remote href、Wiki Page UUIDを対応表に持ちます。Wiki renameでUIDを変更しません。remote href変更は移動・再取得として確認し、同じ題名だけで統合しません。ローカル生成UIDはDomain名変更でも維持します。
予定はevent、タスクはtodo、連絡先はcontactの型付き状態です。説明用Pageは任意の参照であり、毎回ICS/vCardをHTMLへ変換して正本を置換しません。原形式のDocument snapshotを保持し、未対応propertyの欠落を検出します。
## 予定のモデル案
開始・終了、終日フラグ、timezone/TZID、繰り返しrule、例外、参加者、状態、source versionを分離します。終日をUTCの時刻に変換せず、時間帯なしの時刻も区別します。無限recurrenceは画面の対象期間内だけ展開し、件数上限を持ちます。
例外instanceのキーは元seriesとrecurrence identityです。単発の変更・series全体の変更をUIで分けます。参加依頼、回答、メール送信などのschedulingは単純な予定保存とは別の後続機能です。
## 同期と競合
connectorはremote ETagとlocal versionを別々に持ち、更新前の値で条件付き操作します。ETagをDocument SHA-256と仮定しません。対応サーバーではcollection sync-tokenを利用する案ですが、無効なtokenからの再同期と未対応時のboundedな全量照合も必要です。[WebDAV同期仕様](https://www.rfc-editor.org/rfc/rfc6578)。
外部connectorの設計では、通信失敗を外部削除と見なさず、tombstoneは確認できた削除だけにします。双方向書込を追加する際はbase/local/remoteを保存して競合選択へ進めます。これを現在の全DAV入口の挙動とは扱いません。タスク提供面では利用者指定により無条件PUT/DELETEを上書き受理し履歴を保持します。競合候補・通知・採否選択は[所属統一の後続](task-dav-project-unification.md)で、読取外部カレンダーconnectorとは別契約です。
## 連絡先
address bookの所有者・共有範囲、contact UUID、remote UID、複数メール/電話、名称、所属、写真の固定参照を持ちます。人物Principalとcontactは別です。メール一致だけでユーザーへ自動連結しません。重複候補を示し、mergeは変更前をexportできる確認操作にします。
## URL/API案と実装順
旧案の `/v/organizer` ではなく、連絡先の実装は `/v/plan/contacts`、APIは `/_/api/v1/extension/organizer.contacts/`、CardDAV提供面は `/_/api/v1/carddav/` を使います。Wiki PathをそのままDAV collectionにしません。以下の順序案のうち、個人アドレス帳と限定DAV提供は実装済み、共有と外部connectorは後続です。
1. ICS/vCard fixture、入出力preview、警告、重複防止。
2. local calendar/agenda・連絡先一覧、権限・version試験。
3. 選定サーバーに対するread-only connector。
4. 限定された双方向同期、最後に必要性が確認できたDAV提供。
CAL-1:終日・夏時間・例外。CAL-2:UID/href/Path変更。CAL-3:412競合・token失効。CARD-1:複数propertyと未知項目。DAV-1:権限失効・大量collection・app token失効。これらを完了条件にします。
+165
View File
@@ -0,0 +1,165 @@
# カレンダー拡張:正本・CalDAV・Plan UI
更新:2026-09-22。既存の「予定・iCalendar・CalDAV・連絡先」のうちカレンダー部分を具体化する。連絡先、タスク、メール、招待配送は別サブプロジェクト。
## 1. 判断と実装状態
正本を SQLite に置かない。カレンダー単位の manifest と月別 iCalendar ファイルを正本とし、DB は消して再構築できる検索索引に限定する。Wiki Page の HTML 化や revision に予定を埋め込まない。
初期実装は **限定された CalDAV メソッド実装**。完全な CalDAV サーバーの適合性を宣言しない。`DAV: 1` のみ返し、`calendar-access`、`calendar-schedule`、sync-token は未対応のまま広告しない。
| 項目 | 初期実装 |
| --- | --- |
| 正本 | カレンダーの manifest、月ごとの VCALENDAR/VEVENT |
| 検索 | SQLite に domain/calendar/UID・開始/終了・ETag を索引化 |
| 日時 | UTC DATE-TIME、終日 DATE、DTEND は排他的 |
| 保存 | 未知の VEVENT property を保持。UI編集は対象フィールドのみ更新 |
| 並行更新 | 強い ETag の条件付き更新、競合時412、無条件更新428 |
| UI | 年・月・週・日・起点31日一覧・水平タイムライン、詳細・作成・編集・削除 |
| 共有 | 所有者、ユーザー単位 read/write、明示的な匿名公開 |
| 無効化 | system extension `organizer.calendar` の既存スイッチ |
ローカルCRUDと正本の1予定表現では、RRULE/RDATE/EXDATE/RECURRENCE-ID、TZID/VTIMEZONE、floating時刻、DURATION、VALARM、VTODO、METHOD による scheduling を**入力拒否**する。外部購読に限り、検証済みのVTIMEZONE/TZIDと単一RRULEを有限期間のUTC/DATE予定へ射影する。受理後に検索から消えたり、繰り返しが単発予定に化けたりしないことを優先する。例外予定などの未対応が残るため、一般のカレンダーアプリを無制限に接続できる段階ではない。
## 2. URI
### UI
- `/v/plan`:最後に開いた表示種別の現在日。表示履歴がなければ現在週。
- `/v/plan/cal/yyyy`:年。
- `/v/plan/cal/yyyy/mm`:月。
- `/v/plan/cal/yyyy/mm/dd`:日。
- 同上の `/week`:月曜から日曜、その日を含む週。
- 同上の `/events`:その日から31日間の一覧。
- 同上の `/timeline`:その日の時間を左から右に配置。
- 同上の `/new`:その日を初期値に新規予定ダイアログ。
- `/v/plan/event/{UID}`:予定詳細。
- `/v/event/new`、`/v/event/{UID}/edit`:指定どおり作成・編集。
- `/v/calendar`:`/v/plan` への互換入口。
複数カレンダー選択は `?calendar=work,personal` のように `pretty_name` をコンマで連結する。各名は個別にURLエンコードし、区切りのコンマはそのままとする。表示設定ダイアログで複数選択・自分の既定値を保存できる。既定値はドメイン内のユーザー単位で、内部UUIDの配列をファイル `preferences/{principalUUID}.json` に保存する。無指定時は可視な既定値を使い、未設定・すべて失効した場合のみ可視な先頭カレンダーへフォールバックする。既定値による移動ではクエリを追加しない。匿名利用は個人設定を持たない。
`pretty_name` はドメイン内で一意(大文字小文字だけ違う名前も重複)。Unicode NFCの文字/数字、`-`、`_` が使用でき、空白・コンマ・スラッシュ・UUIDそのものは不可。最大120バイト。別途 `name` は自由な表示タイトルとして保持する。月別iCalendar正本の `NAME`([RFC 7986 §5.1](https://www.rfc-editor.org/rfc/rfc7986#section-5.1))、CalDAV collectionのdisplaynameにはpretty_nameを使う。VEVENT単体のUID/ETagやDAV資源hrefは名前変更の影響を受けない。名称の重複チェックはドメイン単位のlockで直列化する。
旧データのpretty_nameは `calendar-{UUID}` を安全な初期値とし、歯車から変更可能。旧UUIDクエリは互換受付してpretty_nameへ置換する。UUIDを保持する既定設定と予定所属はrenameに追従する。外部に配布した旧pretty_nameリンクのリダイレクトは未提供。
UIDはカレンダー内で一意。別カレンダー間に同じUIDがあり得るので、生成する詳細リンクはその予定のpretty_nameを1つだけ含める。複数表示の新規予定では、書込可能なカレンダーから作成先を選ぶ。Wiki Pathのrenameは予定UIDに影響しない。
### プロトコルと管理API
CalDAV入口:`/_/api/v1/caldav/`。既存のジョブ `/_/api/v1/event/` と混同しない。
2026-09-26更新:`organizer.calendar 0.2.0`で専用Basic認証・共通principal探索・初回全件同期・CLI resource名を追加。[予定・タスク共通CalDAV](caldav-common.md)が追加契約。以下の旧仕様に対しcollectionのDepth 1、上限付き全件REPORT、CLI指定filename、JSONイベント取得の`?resource=`が追加される。UTC/DATEに限定する書込プロファイルは維持する。
| URI | 操作 |
| --- | --- |
| DAV root | OPTIONS、PROPFIND Depth:0/1(可視カレンダー) |
| `/{calendarUUID}/` | PROPFIND Depth:0、REPORT calendar-query / calendar-multiget |
| `/{calendarUUID}/{base64url(UID)}.ics` | GET/HEAD、PUT、DELETE |
| `/_/api/v1/extension/organizer.calendar/calendars` | GET一覧、POST作成 |
| 同上 `/{calendarUUID}/settings` | PUT 設定/DELETEカレンダー、manifest version必須 |
| 同上 `/{calendarUUID}/events` | GET `?uid=`、POST作成/編集 |
| `/_/api/v1/extension/organizer.calendar/event?uid=` | 可視カレンダーからUID解決。複数一致は409 |
UIDは任意のテキストなのでDAVのファイル名へ直接使わずbase64urlにする。絶対URLのhrefや別カレンダーへのmultigetは受理しない。全件無制限のDepth:infinityやREPORTは提供しない。現在は作成を管理JSON APIで行い、MKCALENDAR/PROPPATCH/ACLは後続。
## 3. ファイルと障害復旧
```
documents/calendars/{domainUUID}/{calendarUUID}/
calendar.json
.lock
2026-09.{sha256}.ics
2026-10.{sha256}.ics
```
`calendar.json` は名前・所有者・公開設定・メンバー・version・月→ファイルSHA-256対応を持つ。月別ファイルは世代ごとに不変。VEVENTは開始月へ収納し、月をまたぐイベントは分割しない。
変更順序:カレンダーのプロセス間lock取得 → 現manifestとETagを確認 → 変更月の新ファイルを書きfsync → manifestをatomic renameしてディレクトリfsync → 未参照の旧世代を清掃。
9月から10月へ予定を移動する場合も、両月の新ファイルを準備してからmanifestを一度だけ置換する。中断した未参照ファイルは次の成功時に清掃する。過去世代を無限に保持する履歴設計にはしない。正本バックアップはmanifestとその参照先を一組で取得し、稼働中はlockと協調する必要がある。
DB表は `calendar_index_state`、`calendar_month_cache`、`calendar_event_cache`。manifest versionが変わったときだけ、hashが変わった月を索引へ反映する。DB反映前の中断は次回アクセスで修復する。DBだけが最新になりファイルが古い状態を正本とはしない。復旧時は3表を**一緒に**破棄し、正本から再構築する。
手動でICSを書き換える運用は未提供。hash名のファイルを直接編集しない。キャッシュ再構築時にはSHA-256を検証する。正本ファイルのACLは0600、ディレクトリは0700。HAの複数ノード同時書込やネットワークFSのロック保証は別途検証する。
## 4. 取得・キャッシュ・負荷
フロントエンドは対象期間の `calendar-query` でgetetagのみ取得。タブ内メモリのETagと一致する予定は再取得しない。変更分だけJSONの予定投影を取得し、同時取得は最大4件。独自にREPORTへ304を返す実装はしない。ICS GETではIf-None-Match/304を利用可能。
月をまたぐ予定は開始月のファイル列挙ではなく、DBの開始/終了の重なりで取得する。ゼロ時間の予定は開始時点が期間内なら一致。DATEとUTC日時を表示時に混同しない。UIの時刻入力・表示・日境界は**ドメイン設定のタイムゾーン**を用い、正本はUTCで保存する。DSTで存在しない時刻や二重になる時刻をドラッグ操作で暗黙補正せず、編集画面で再指定させる。DATEの検索境界差を補うためUIのREPORTは前後1日を加え、描画時に絞る。
週・日ビューは67〜400%の6段階で時間軸を拡縮する。100%以下は15分、200%は10分、300〜400%は5分単位で移動・リサイズする。予定中央のドラッグは長さを保った移動、上端は開始、下端は終了の変更である。同時刻帯で重なる予定は最大同時数分の等幅laneへ配置する。短い予定には操作可能な最低表示高を確保し、その表示領域同士も衝突しないlaneへ分ける。保存成功時はAPI応答の予定だけをタブ内集合へ反映してlaneを再計算し、期間REPORTや予定全件を再取得しない。時間列の空白をドラッグすると設定タイムゾーンの選択範囲を入力済みの新規予定ダイアログを開き、クリックだけでは作成しない。
上限:ドメイン256カレンダー、1資源256KiB、XML64KiB/深さ12、月5000件/16MiB、期間370日、結果5000件、multiget200件、lock待機5秒。超過は部分成功にせずエラー。ETagだけのREPORTは本文をparseしない。永続ブラウザキャッシュや共有CDNへの予定保存は行わない。権限を確認してから304・本文を返す。
## 5. 権限
カレンダー所有者が名前・公開設定・collaboratorを管理する。collaboratorは同じドメインの有効なユーザーで、`read` または `write`。writeには予定作成・変更・削除を含むが、共有設定の変更は含まない。readonlyアカウントの書込は不可。グループ共有・所有権移管・管理者による復旧は後続。
カレンダー全体の削除も所有者だけが行う。設定ダイアログで全予定を含む完全削除を確認し、`DELETE .../settings` に現在のmanifest versionを渡す。古いversionは412とし、削除対象の外部同期ジョブはqueuedなら取消、runningなら取消要求を付ける。正本ディレクトリを原子的に不可視化して削除し、SQLiteの3索引表を消す。ユーザー既定値に残ったUUIDは毎回の可視性検査で除外される。実行済みジョブ履歴とログは監査情報として削除しない。
匿名公開はカレンダー全体を対象とし、説明・場所を含めて公開する旨をUIで告知する。既存のドメイン公開上限がpublic、ドメイン有効、authorization_mode=enforcingのときだけ有効。上限を閉じれば次の取得から匿名アクセスを拒否する。公開範囲の狭い予定を混在させるには別カレンダーを使う。VEVENTのCLASSだけをアクセス制御として扱わない。
Page公開とは別の正本なのでCityのSpot/Pathへ暗黙に従属させない。将来Pageへの関連付けをしてもPage参照権限と予定権限は別途評価する。
## 6. RFC対応の検証と残タスク
基準:[RFC 5545](https://www.rfc-editor.org/rfc/rfc5545) / [RFC 4791](https://www.rfc-editor.org/rfc/rfc4791)。パーサーは `github.com/emersion/go-ical` を固定バージョンで使用し、アプリ側でDTSTART必須、UTC/DATE、排他的DTEND、UID、不支持property、制御文字、必須項目を追加検証する。出力はCRLF・UTF-8を分断しない75octet折返し。ライブラリ採用だけをRFC適合の証明にはしない。
- [x] 単発UTC/終日、未知property保持、月境界、同時更新、キャッシュ再構築、公開取消の回帰テスト。
- [x] Plan各ビュー、期間REPORT、差分取得、設定・予定ダイアログ、所有者限定のカレンダー完全削除。
- [x] 外部購読用のTZID/VTIMEZONE(IANA、主要Windows名、定義内のSTANDARD/DAYLIGHT)と夏時間fixture。
- [ ] ローカル正本でのTZID/VTIMEZONE/floating対応、DURATIONの暦日演算。
- [x] bounded recurrence展開、EXDATE/RDATE、RECURRENCE-ID例外、時間範囲オーバーラップ、件数・本文量・対応頻度によるCPU/メモリ上限。
- [ ] 標準XML precondition/error、PROPFIND/PROPPATCH/MKCALENDARの適合性、principal discovery、`/.well-known/caldav`、Depthと未知filterの完全な検証。
- [ ] アプリ専用の失効可能なCalDAV credential、権限のread/writeスコープ、クライアント適合試験(現状は既存session Bearer)。
- [ ] collection sync-token、tombstone保持と期限切れの再同期。現状は期間REPORTが正しい集合を返す。
- [ ] バッチmultigetの単一索引snapshot化、大量calendarの一覧ページング、UI期間ごとのvirtualization。
- [ ] 専用ジョブとしてreindex/検証/容量統計、イベント資源監視、管理者向けバックアップ・restore・所有権復旧。
- [ ] グループcollaborator、イベントごとのvisibility、承認Inbox、共有変更監査、変更通知。
- [ ] 招待・参加回答・リマインダーはCRUDとは別のevent/scheduler連携。iTIPメール配送を無断で有効にしない。
- [x] 外部ICSの条件付きGET購読、外部CalDAVの期間REPORT/差分multigetによるread-only同期(下記制約あり)。
- [ ] 外部連携のdiscovery、OAuth、sync-token、取得期間の設定、競合UI付きの双方向同期。
完全なCalDAV適合性・繰り返し対応の完了までは「一般クライアント互換」の完了チェックを付けない。
## 7. 今回の検証記録
- `go test ./internal/calendar ./internal/server -run 'TestCalendar|TestPolicyForRequest'`:成功。
- Chromium:月・週・timeline・詳細の描画、設定タイムゾーン、400%/5分ズーム、中央移動、上下端リサイズ、重複予定の等幅配置、ダイアログによる作成/編集/削除、カレンダー名の変更、ETag一致時の本文再取得抑止を確認。
- 全体テストは `TestConfigBookmarksSaveAndPanelRegistry` が既存設定画面の `config-form-table` に横スクロールコンテナがないとして失敗。カレンダー画面には当該テーブルは追加していない。別UIタスクとして調査・是正する(テストを緩めない)。
## 8. 時間軸の操作
終日の入力・詳細表示では終了日を**含む**。同日開始・同日終了で1日の予定を作成できる。保存時だけ終了日にUTC暦日を1日加算し、編集・表示時には1日戻す。正本ICSとAPIのDTENDはRFC 5545の排他的境界を維持する。既存データの移行・外部予定の書換は不要。
トップバーはCity・直近の予定・通知・プロフィールの並び。直近の予定は既定カレンダー(未設定なら先頭1件)の今日から31日間を最大20件表示する。`GET /_/api/v1/extension/organizer.calendar/upcoming?date=yyyy-mm-dd` はログイン必須、毎回権限確認、no-store、5秒上限、カレンダー毎20件の本文取得上限を設ける。モーダルを開いたときだけ取得し、閉じたら中断する。通知と予定は背景スクロールを抑止した中央モーダルで、Escape/閉じるボタンに対応する。
日・週ビューは縦の時間軸、timelineは横の時間軸に予定を表示する。従来の週リストは `/v/plan/cal/yyyy/mm/dd/week/list` とし、先頭行に曜日を表示する。日・週の尺度は40/60/90/120 px/時から選択できる。
最後に開いた年・月・週・週リスト・日・timeline・予定一覧の表示種別だけをPrincipal単位でブラウザへ保存する。予定本文、カレンダー名、選択日、資格情報は永続化しない。予定の新規作成後は詳細画面へ遷移せず、作成を開始した表示種別と基準日へ戻して再取得する。新規作成専用URLから開始した場合も、保存済み表示種別(未保存なら週)へ戻る。
書込可能な予定の移動ハンドルをドラッグすると15分単位で日時を変更する。日・週では上下キー、timelineでは左右キーでも移動可能。期間の長さを保持し、Escapeでドラッグを取消す。存在しない/曖昧なローカル時刻への日・週移動を拒否する。保存は既存ETag付きAPIを通し、競合時には上書きしない。終日・外部読取専用の予定にハンドルは表示しない。ブラウザ試験で縦横の実マウス操作とキーボード操作を検証する。
## 9. 外部カレンダーの読み取り同期
新しい空のカレンダーを作り、所有者が「外部同期」ダイアログでICS URLまたはCalDAVコレクションURLを設定する。取得間隔は1分〜7日、手動同期・停止を提供する。外部同期を停止しても、取得済み予定と読み取り専用属性は保持する。双方向同期や、既存のローカル予定への混在取込は行わない。公開・collaborator設定は取込先の設定に従うため、私用フィードは非公開カレンダーで利用する。
- API:`PUT /_/api/v1/extension/organizer.calendar/calendars/{UUID}/source` で設定、同じURIへの `POST` で手動キュー投入。所有者限定、設定更新はmanifest version必須。
- 組込イベント `calendar.source.sweep` が60秒ごとに期限を確認し、`calendar.source.sync` を低優先度・カレンダー単位の重複抑制付きでキューへ投入する。実行状態・ログは既存イベント管理に統合。UIを開く必要はない。間隔は次回取得の目安でありキュー混雑により遅れる。
- ICS:ETag/Last-Modifiedを条件付きGETに利用する。304では予定を再書込しない。
- CalDAV:過去90日〜未来275日のREPORTでgetetagを取得し、変更分のみ100資源単位のcalendar-multigetで取得する。期間外はローカル投影に保持しない。コレクションURLを直接指定し、discoveryはまだ行わない。
- 完全な応答を検証してから全月snapshotとmanifestを更新する。取得失敗、未対応形式、部分的なmultiget、途中で変わったETagでは前回の予定を保持。正常な空集合は削除として反映する。安定したUID順で保存し、応答順の変化による書換を避ける。
- 上限:ICS本文8MiB、CalDAV各応答8MiB・予定本文合計8MiB、5000件の射影予定、ジョブ50秒。Outlookで約10秒かかる応答を許容するため、response header 25秒・HTTP全体45秒とする。2.6MiB超・4500件の合成fixtureを回帰試験に含める。
- Outlook公開ICSの `METHOD:PUBLISH`、`X-WR-CALNAME`、`VTIMEZONE/TZID`、単一 `RRULE`、Microsoft固有X-propertyを受理する。主要Windows TZIDはIANAへ対応付け、それ以外はSTANDARD/DAYLIGHTとTZOFFSETTOの遷移を同期期間について解決する。RRULEは過去90日〜未来275日の各予定へ安定UIDで展開し、内部正本はUTC/DATEのまま維持する。`X-MICROSOFT-CDO-ALLDAYEVENT:TRUE` のローカル午前0時境界はDATEの終日予定へ変換する。
- Outlookが出力する `DTSTART == DTEND` の時刻予定はゼロ時間予定として保持し、同じ境界のMicrosoft終日予定だけは排他的終了を翌日に正規化する。終了が開始より前、またはDATEとDATE-TIMEが混在する場合は予定内容をログへ出さず同期全体を失敗させる。
- 外部購読の解析済みプロパティ値・パラメータに混入したC0(タブを除く)、DEL、C1制御文字は正本化の前に除去し、予定へ `X-PAGESWIKI-SANITIZED:CONTROL-CHARACTER` 相当の印を残す。ローカルCRUDの厳格な入力検証は緩和しない。NUL、不正UTF-8、制御文字を含むUIDは同一性・構文破壊を避けるため購読全体を拒否する。
- RDATEとEXDATE(DATE/DATE-TIME、複数値)、RECURRENCE-IDによる移動・内容変更・取消を親RRULEへ合成する。例外の出力UIDは元のRECURRENCE-IDから安定生成し、移動後も同一インスタンスとして扱う。EXDATEと変更例外が同一回を指す場合は明示的な変更例外を優先する。
- 購読射影が親RRULEを省略し、`RECURRENCE-ID` 付きの変更回だけを含む場合は、取消回を無視し、DTSTARTを持つ回を元のRECURRENCE-ID由来の安定UIDで単発射影する。親がないためDTEND省略時の長さは推測せずゼロ時間予定とする。
- RDATEのPERIOD、RECURRENCE-IDの`RANGE=THISANDFUTURE`、DURATION、VALARM、floating時刻はまだ未対応。部分的に欠落させずフィード全体を同期エラーにし、前回snapshotを保持する。現段階でOutlookの全機能や一般プロバイダへの完全対応は宣言しない。
- 同期失敗はcompose logへdomain UUID、calendar UUID、protocol、処理段階、安全化した理由を出す。購読URL、認証情報、UID、タイトル、説明、場所、ICS本文はログへ出さない。
- HTTPS/443の公開アドレスのみ許可する。接続時DNS検証と接続IP固定、プライベート/loopback/link-local等の遮断、リダイレクト再検査、環境proxy無効によりSSRFを防ぐ。認証付きリダイレクトは同一origin限定。
- 任意のBasic認証資格情報は既存認証鍵を用いた暗号化とdomain/calendar束縛で保存し、APIに再表示しない。URLに購読トークンを含む場合は保護された0600のmanifestに保存される。URL/認証情報をジョブpayloadに入れず、取得エラーを秘匿化する。バックアップには正本に加えて認証暗号鍵も必要。
実プロバイダ接続試験、より広いiCalendar形式、資格情報更新・失効の監査、同期専用の容量/CPU指標、取得対象の管理者allowlistは後続タスクとする。
+33
View File
@@ -0,0 +1,33 @@
# カレンダーのスマートフォンUI
2026-09-25実装。iPhone SE相当の320×568/375×667と667×375横画面を対象。**Chromiumのタッチ・画面サイズエミュレーションで検証済み。Safari/iPhone実機の確認は未実施。** 同日05:59 JSTに本番2インスタンスへ反映済み。[反映記録](../../operations/deployments/2026-09-25-active-notifications-calendar-mobile.md)。
## 操作と表示
- 小さい画面では、表示種別の7リンクをselectにまとめる。今日/前後/日付ピッカーは常時表示。
- カレンダー選択・更新・新規予定を残し、追加/設定/外部同期/表示対象は「⋯」メニューへ集約。権限による元の表示制御を引き継ぐ。
- 上部の重複ナビと重複見出しを省略。見出し自体はスクリーンリーダー用に維持する。
- 週ビューは細い7列にせず、縦画面で約1日、横画面で約2日の幅を確保。曜日/日付ボタンで同じ週の列へ横移動する。見出しの日付を押すと日ビューへ進む。PCの7列表示は維持する。
- 月ビューは日付と予定件数を表示し、詳細は日ビューで見る。狭いセルへの件名詰め込みは行わない。
- 一覧では日付/件名を優先。場所の列はスマホでは省略し、予定詳細で確認する。
## タッチ操作
coarse pointerでは通常、予定の上も空き領域もスクロールとpinch zoomを優先する。指が触れただけで時刻変更・範囲作成を開始しない。タップは予定の詳細へ進む。
時間グリッドの「編集操作」をONにした場合だけ、移動・開始/終了の伸縮・空き領域の範囲選択を有効化する。ON/OFFはaria-pressedで表現。PCのドラッグ・矢印キー操作は維持する。タイムラインの小さいドラッグハンドルはタッチでは隠し、予定詳細から時刻を編集する。
## Safariへの配慮
フォームの文字を16px以上、操作領域を40〜44pxに揃える。画面のzoomを禁止するviewport指定は追加しない。編集ダイアログはVisualViewportのheight/offsetを使い、ブラウザバーやソフトキーボードによる変化を考慮する。pinch zoom中はサイズ更新を停止する。CSSのdvh fallbackとsafe-area insetを併用する。
スマホのスクロール領域と編集ダイアログはbackdrop blurを外し、テーマの背景色でコントラストを確保する。予定データのキャッシュ/REPORT/ETag/タイムゾーン処理は変更しない。
## 検証と残TODO
- 合成ローカルカレンダーで320/375縦・667横、ページ横はみ出し、列幅、日付選択、スクロールと編集gesture切替、月の件数、設定メニュー、編集dialogの収まりを自動検証。
- 既存のPC CRUD・ドラッグ・resize・400% zoom・ETag試験を継続。
- 外部取り込みカレンダーの内容にはアクセスしない。
- TODO: Safari実機でソフトキーボード開閉、画面回転、pinch zoom、背景復帰、長い終日予定、多重重複予定を確認。端末・OSバージョンに応じた調整。
参考:[WebKitの動的viewport単位](https://webkit.org/blog/12669/new-webkit-features-in-safari-15-5/)、[MDN touch-action](https://developer.mozilla.org/en-US/docs/Web/CSS/Reference/Properties/touch-action)。
+45
View File
@@ -0,0 +1,45 @@
# 3. キャンパス・ドメイン・連携
状態:ローカル管理、外部キャンパス参加、署名付き設定配信は実装済み。コンテンツ同期・DRとは別機能。
## FQDNと配置
キャンパスの設定値はURLではなくFQDNです。ドメインのホスト指定が短い名前ならキャンパスFQDNのサブドメインとして扱い、FQDNならその値を使います。空のホスト名はキャンパスFQDN自身に対応し、未知Hostの受け皿にはしません。
表示用FQDNと配置先インスタンスは別の情報です。FQDNを指定しただけでは別インスタンスとは限りません。配置先は承認された候補から選び、同じFQDNの鍵フィンガープリント変更は同じ実体として黙認しません。
## 設定の所有者
- キャンパス管理:配下ドメイン、配置、参加トークン、キャンパスの共通情報。
- ドメイン管理:Wiki名、テーマ、記事・拡張設定、メンバー、権限。
- 外部キャンパス管理:コントローラから受け取った管理項目をローカルで独自に上書きしない。
- 秘密:実際に認証・接続するインスタンス側に保持し、一般設定manifestへ混ぜない。
ドメインの設定は個別です。同じキャンパスに属するだけで記事、グループ、認証セッションを共有しません。
## 参加と配信
コントローラ発行トークンで参加要求し、インスタンスidentityを承認します。設定manifestは署名と単調なsequenceを持ち、受信側は適用とcheckpoint更新を同一トランザクションで行います。古いsnapshotへの巻き戻しや同名の別authorityへの置換を拒否します。
入口設定を含むmanifest v2は、全承認済み配置先の対応能力を事前確認します。対応能力の申告は「適用完了」や「オンライン」の証明ではありません。反映ACKと鮮度の明示はTODOです。
`--add-external-campus` は通常経路、`--add-external-campus-http` は明示的な開発用HTTP経路です。HTTP許可を一般接続の既定値へ拡張しません。
## 継承の区別
キャンパス管理者のローカル継承には、付与元ごとの記録と失効処理があります。独立したローカル付与を消さず、継承を切ったときに派生権限だけを除きます。
Domain間PSKによるidentity・group継承は設計段階です。署名付き差分、失効、origin競合、鍵交代を含む同期契約と実装が必要です。キャンパスmanifestの配信やローカル管理者継承が完成したことをもって、任意ドメイン間の安全な権限同期が完成したとは扱いません。
## TODOと合格条件
DRの次段階は [管理主系とデータ主系の分離](dr-authority-and-rejoin.md) と
[管理manifestの世代・受理契約](dr-protocol-and-sync.md) を参照する(詳細設計のみ)。
Campusを管理するprimaryと、Domainデータに書き込むprimaryは別authority。
ただし同じDBを正本として共有する役割は物理的なwriter切替を連動させる。
保存済みroleやprobe応答だけで復帰ノードをprimaryにしない。
- 反映ACK、設定管理の解除、削除ドメインの保持・GC。
- 同名・異originのグループ衝突、PSK失効、sequence欠落、鍵交換の試験。
- 秘密の外部管理と明示的なローテーション手順。
- 別インスタンス記事射影は、取得権限・失効・障害時の閉じ方を設計してから実装する。
+216
View File
@@ -0,0 +1,216 @@
# PagesWiki のコンポーネント命名と責務
2026-11-01 採用。神名を実装アーキテクチャの部品名として使用する。
Grammaratur、Dregarnuhr、Geduldh、Mestionora、Ordoschnellic、Schutzaria、Kunstzeal、Seheweit の
8部品の初期分離を実装した。初期分離と各部品の後続機能・本番展開の完了は区別する。
## 由来と適用範囲
プロジェクト名 pages は、作者の意図として Little Glee Monster の楽曲 pages に由来する。
Knowledge City Model は『とある科学の超電磁砲』の学舎の園に着想を得た知識の配置モデル。
このモデルの Campus、Domain、Quartier、Street、Spot、Page は変更しない。
内部の機能分担には『本好きの下剋上』の神々の名前を用いる。
作品の役割とソフトウェアの役割を重ねる命名の参考は
[TOMOYO Linux 作者の説明](https://i-love.sakura.ne.jp/tomoyo/)。
名前の参照表は [プロジェクト作者の Gist](https://gist.github.com/anya-tokugawa/d3939b16298937f8b7e39a6ec3595464)。
以下の英字表記は PagesWiki 内の採用表記であり、作品の公式翻訳表記を定義するものではない。
## 採用した部品名
| 部品名 | 内部識別名 | 担当範囲 | 担当しないもの |
| --- | --- | --- | --- |
| Mestionora メスティオノーラ | `mestionora` | 索引、検索、知識間の関連性の解決 | 正本保存、認可の独自判断、編集UI |
| Grammaratur グラマラトゥーア | `grammaratur` | 文書形式変換、変換時の情報保全、変換診断 | 認可、画面用sanitizer、macro展開、Revision保存 |
| Geduldh ゲドゥルリーヒ | `geduldh` | 内容アドレス付きDocumentの保存・取得・完全性検証 | 認証やタスク等の全DB、参照なしを独断判定するGC |
| Dregarnuhr ドレッファングーア | `dregarnuhr` | 実行時刻、ジョブ投入・実行制御、リース、再試行 | カレンダーやタスクの業務ルール、通知本文・宛先 |
| Ordoschnellic オルドシュネーリ | `ordoschnellic` | イベントに応じた通知、宛先別配送、配送状態 | 業務変更の生成、正本トランザクションの所有 |
| Schutzaria シュツェーリア | `schutzaria` | 権限評価、公開範囲、共有・承認ポリシーの評価 | 個々のOIDC/LDAPプロバイダー実装、表示用HTML変換 |
| Kunstzeal キュントズィール | `kunstzeal` | 図・スライド等の描画プロバイダー実行基盤 | 独自描画エンジンの強制、文書正本の変更 |
| Seheweit ゼーエヴァイト | `seheweit` | リソース・稼働状態の収集、集約、観測 | ジョブ制御や観測対象の業務処理 |
名前はリリースごとに変えず、責務と共に維持する。バージョンは従来の SemVer 2.0.0 を継続する。
機能を増やすためだけに神名を追加せず、境界と受入条件が成立してから部品を作る。
神々の系譜や関係を依存グラフ、権限の上下関係、配置単位へ強制的に写さない。
## 実装と運用の命名規則
- パッケージ名は小文字の内部識別名、設計上の部品名は先頭大文字の表記とする。
`DocumentStore` 等の契約・インターフェースと `PutIfAbsent` 等の操作名は技術用語を使う。
- 1部品は1コンテナを意味しない。プロセス内のパッケージとして分離でき、独立配置は別判断。
- UIは検索、通知、権限設定などの通常名を維持する。神名の理解を利用条件にしない。
管理者向けの診断や設計文書には、神名と一般的な役割名を併記できる。
- URI、既存のprovider/extension ID、設定キー、DB名、保存形式、イベント型、consumer ID、
レジストリ上のイメージ名を命名のためだけに変更しない。特にconsumer IDの改名で
既存checkpointを失い、通知や処理を再実行することを防ぐ。
- 新たなログ・トレースで識別が必要なら固定の `component=grammaratur` 等を用いる。
文書本文や認証情報は記録しない。無条件に変換ごとのログを追加しない。
- 既存の `server` / `store` 一式を神名へ置換しない。呼び出し契約・テストを確立してから
部品を抽出し、移行中は一方向の互換ラッパーを残す。実装を二重に保守しない。
## Grammaratur の実装契約
初期実装は `internal/grammaratur`。HTTP、DB、ファイル、ネットワークへ直接アクセスせず、
文字列の変換だけを行う。認可と入力サイズ上限は呼び出し元が検証する。
| 関数 | 契約 |
| --- | --- |
| `HTMLToMarkdown` | 従来の通常変換。保護フェンスを理解しない外部エディタ等にも渡せる。変換損失の可能性は従来どおり |
| `MarkdownToHTML` | 従来のMarkdown方言とオプションを維持した通常変換 |
| `HTMLToMarkdownAuthoring` | 往復変換で構造・属性・文字が変わる場合は元HTMLを保護ブロックに保持 |
| `MarkdownAuthoringHTML` | 明示的にauthoringを選んだ呼び出しだけで `{=html}` ブロックをHTMLへ戻す |
出力は表示を許可されたHTMLではない。ブラウザー表示時は従来どおり認可・macro展開・
sanitizer・CSPを適用する。原文の保全とスクリプトの実行許可を混同しない。
通常変換を保護ブロック対応へ暗黙に切り替えず、hosted editor の保存形式も変更しない。
`internal/compiler` のHTML/Markdown変換関数と `render.Markdown` は Grammaratur へ委譲する。
組み込みエディタ、authoring API、Markdown出力、hosted editor のHTML入力変換は
Grammaratur を直接呼び出す。既存の通常Markdown呼び出しは `render.Markdown` 経由でも
同じ実装を使う。図のhash等、文書形式変換以外の `compiler` の処理は今回移さない。
依存方向は呼び出し元 → Grammaratur → 変換ライブラリ。
Grammaratur から server/store/db/render/compiler へ戻る依存は作らない。
新たな形式や詳細診断を追加する場合も、この境界を維持する。
## 移行順序と受入条件
1. **Grammaratur 初期分離**:通常変換と保護変換を集約。既存の関数・API・返却形式を維持。
通常変換の旧ライブラリ出力との一致、互換ラッパー、保護ブロック、エディタ、
hosted editor、表示安全性、ドキュメントexportを検証する。
2. **Dregarnuhr 初期分離**:現行schedulerから汎用の時刻・retry・dispatch契約を抽出。
preview、City、calendar、taskの実行ハンドラーは登録型の利用者として残す。
fake clockとworker競合試験で優先度・実行上限・既存claimを検証する。
期限付きlease・fencingは既存実装にもなく、初期分離には含めない。
[実装契約と後続TODO](dregarnuhr.md) を参照。
3. **Geduldh 初期分離**:既存 `documentstore.Store` 契約をaliasで維持し、
ローカル保存と検証付き一時展開の実装を `internal/geduldh` に分離。
filesystem layout、hash、Range、streaming、移行・DR手順を変えない。
大きな `store` パッケージ全体の改名はしない。[文書保存契約と残件](geduldh.md) を参照。
4. **Mestionora 初期分離**:SQLite検索・索引・City検索を分離し、
呼出し元の権限判定と既存の候補・抜粋予算を維持。[検索と索引契約](mestionora.md) を参照。
5. **Ordoschnellic 初期分離**:Web Pushの有界配送・再試行・暗号化を抽出し、
Inboxの原子的登録と購読checkpointを維持。再送が起こり得る既存の保証範囲を変更しない。
[通知配送契約と残件](ordoschnellic.md) を参照。
6. **Schutzaria 初期分離**:DB非依存の読み取り・Locality・グループ・公開範囲・承認状態評価を分離。
認証・DB読込・公開確定transactionは既存側に残す。[権限評価契約と残件](schutzaria.md) を参照。
7. **Kunstzeal 初期分離**:ビルトイン表示方式の選択、Krokiの有界描画、図hash/cacheを分離。
認可・ソース由来・CSPを維持し、redirect拒否とcache原子的置換を追加。[描画契約と残件](kunstzeal.md) を参照。
8. **Seheweit 初期分離**:リソース収集と履歴保持・bucket規則を分離。現在値APIと背景監視を接続し、
収集読込上限・容量計算・取消/read-only保存を補強。認可とSQLは既存側に残す。[観測契約と残件](seheweit.md) を参照。
この移行は配置変更、DB migration、機能追加を必須にしない。各段階でソース変更、
テスト、commit、CI、デプロイを別々に記録する。本番反映は検証後の別操作とする。
## 今回の変更範囲
- 採用済み:8部品の名称と責務、互換性と依存方向の規則。
- 実装済み:8部品の初期分離、旧入口の委譲、利用側の接続、境界・互換性テスト。
- 後続:Dregarnuhr のリース・fencing・終了処理、
Grammaratur の形式追加・詳細診断・ブロック単位の保護、Geduldh の外部backend・配置分離、
Mestionora のポリシー事前読込・現行専用索引・言語別検索、Ordoschnellic のclaim世代・購読更新競合・配送観測、
Schutzaria の権限読込の一貫性・承認継承・権限総覧、Kunstzeal のprovider版・実行予算・描画形式の拡充、
Seheweit の情報源別availability・収集予算・rollup・component別観測・HA/SNMP。
- 対象外:既存DBと本番データの変更、UIの改名、画像やロゴの利用、コンテナ分割、本番展開。
## 検証記録
2026-11-01、合成データと一時DBで以下を確認した。本番データへの操作は行っていない。
- `go test ./...` 成功。保存層・サーバー・CLI・設計書exportを含む。
- `go test -race ./internal/grammaratur -count=1` 成功。
- `go vet ./internal/grammaratur ./internal/compiler ./internal/render` 成功。
- `PAGESWIKI_BROWSER_TEST=1` の `TestEditorResilienceChromium` と
`TestHTMLProjectionChromium` 成功。編集・保存・競合・言語分岐・HTML保全と、
9つの表示/編集経路の安全性を確認。
- `git diff --check` 成功。
同日、Dregarnuhr の初期分離後にも以下を確認した。
- `go test ./...` 成功。store・serverを含め全Goパッケージの回帰テストを実行。
- `go test -race ./internal/dregarnuhr -count=1` 成功。
- store/serverの `TestDregarnuhr*`、`TestClaimDueEvent*` とコアの境界・実行制御試験を
race detector付きで実行し成功。外部カレンダーは実行せず、登録の確認だけを行った。
- `go vet ./internal/dregarnuhr ./internal/store ./internal/server` 成功。
- `git diff --check` 成功。今回ブラウザ試験の再実行はしていない。
同日、Geduldh の初期分離後にも以下を確認した。
- `go test ./...` 成功。保存・サーバー・CLI・DR・設計書exportを含む。
- `go test -race ./internal/geduldh ./internal/documentstore -count=5` 成功。
新旧入口の型・エラー・保存キー互換、並行保存、一時コピーの共有と回収を確認。
- store/server/DRの文書保存、バックフィル、Range配信、認可付き不変オブジェクト配信、
ZIPの部分取得、Materialize、snapshot/restore試験をrace detector付きで実行し成功。
- 保存失敗時のRevision非確定、DB競合時のpayload保全、Geduldhから保存したバイト列のDR復元を確認。
- `go vet ./internal/geduldh ./internal/documentstore ./internal/store ./internal/dr` 成功。
- `git diff --check` 成功。今回ブラウザ試験は再実行していない。
同日、Mestionora の初期分離後にも以下を確認した。
- `go test ./...` 成功。既存DBを読む任意診断は無効にし、一時DB・合成データのみで実行。
- Mestionoraの境界・認可・FTS実行計画をrace detector付きで3回反復し成功。
- store/serverを含む検索、City、型・エラー互換、Domain/alias分離のテストを
race detector付きで実行し成功。
- `go vet ./internal/mestionora ./internal/store ./internal/server` 成功。
- `TestSearchResultsChromium` 成功。1440pxと390pxでCity先頭表示、3表示モード、
フィルター、リンク、危険なHTMLの文字列表示と横方向のはみ出し防止を検証。画像も目視確認。
- 索引初期化・更新SQLはpackage/receiver/DBフィールドの移動以外が同一であることを確認。
- `git diff --check` 成功。ポリシーの遅延DB参照に伴う接続制約は後続TODOとして記録。
同日、Ordoschnellic の初期分離後にも以下を確認した。
- `go test ./...` 成功。既存DBを読む任意診断は無効にし、一時DB・合成データのみで実行。
- `go test -race ./internal/ordoschnellic -count=5` 成功。
バッチ上限、キャンセル後の終了保存、暗号化payload、HTTP分類、未設定clientでの送信禁止、依存境界を確認。
- store/serverの通知・Activity Subscription・Task Activity・失効済みfocus通知と新規配送試験を
race detector付きで3回反復し成功。並行claim、lease境界、旧attempt完了の拒否、
認証/Domain/受取人の不一致、focused延期、backoff、Inbox保全、checkpoint継続を確認。
- 追加した有界清掃・read-only拒否試験もrace detector付きで3回反復し成功。
- `go vet ./internal/ordoschnellic ./internal/store ./internal/server` と `git diff --check` 成功。
- 外部通知サービスへの送信・実端末Push受信・ブラウザ試験は今回実施していない。
UIとService Workerは変更せず、配送の障害競合は後続TODOとして記録。
同日、Schutzaria の初期分離後にも以下を確認した。
- `go test ./...` 成功。本番DBを読む任意診断を無効化し、一時DB・合成データで実行。
- `go test -race ./internal/schutzaria -count=5` 成功。親子制限の交差、肯定grantの範囲、
snapshotのコピー・並行参照、グループプロファイル、公開上限、予約Path、ユーザー領域、承認状態を検証。
- store/serverのPath・Locality・匿名公開・公開ナビゲーション失効・グループ継承失効・
governance・承認・endpoint判定をrace detector付きで2回反復し成功。
- 新規のstore型/JSON/エラー互換、4,096件上限、DB失敗時の不完全policy拒否試験を
race detector付きで3回反復し成功。
- `TestCityGovernanceChromium` 成功。隔離fixtureで所有者委譲・公開上限・アクセス設定を操作し、保存結果を確認。
- 抽出対象8ファイルのSQLリテラル(スキーマ・triggerを含む)が変更前と同一であることを確認。
- `go vet ./internal/schutzaria ./internal/store ./internal/server` と `git diff --check` 成功。
同日、Kunstzeal の初期分離後にも以下を確認した。
- `go test ./...` 成功。実Krokiではなくlocalhostの偽provider、一時DB・合成ソースを使用。
- `go test -race ./internal/kunstzeal -count=5` 成功。表示順・catalogコピー・kind/hash互換、
入出力上限、取消、エラー非開示、cacheの並行読取と原子的置換を検証。
- serverのKunstzeal・既存diagram認可分離・表示形式・hosted adapter設定・HTML projection等を
race detector付きで3回反復し成功。301/302/307/308の転送拒否、cache保存失敗時の表示継続も確認。
- 追加の由来情報なし/期限切れのディスクcache拒否試験もrace detector付きで3回反復し成功。
- `TestHTMLProjectionChromium` 成功。既存の9表示/編集経路で正本と表示の信頼境界を確認。
- registry定義は型名の公開化以外が同一であることを確認。nwdiagは実装済みcatalogに存在しないため、設計書の状態を訂正。
- `go vet ./internal/kunstzeal ./internal/compiler ./internal/server` と `git diff --check` 成功。
- 実Kroki・本番ネットワーク・取り込み済み外部カレンダーへのアクセスは行っていない。
同日、Seheweit の初期分離後にも以下を確認した。
- `go test ./...` 成功。本番DBの任意診断は無効化し、一時DB・合成データで実行。
- `go test -race ./internal/seheweit -count=5` 成功。JSON互換、情報源の失敗、proc読込上限、
容量のoverflow/underflow防止、I/Oの取得可否とprocess identity、並行収集、履歴windowを検証。
- store/serverのSeheweit・既存リソース監視・履歴・実行状態API試験をrace detector付きで3回反復し成功。
同一jobの重複防止、保持境界、bucket代表の互換、取消、read-only保存拒否、清掃失敗時rollback、
APIの期間fallbackと空配列、GETが履歴保存しないこと、認可・設定保持も確認。
- `TestExecutionMonitorChromium` 成功。隔離fixtureで実行状態・リソース画面と設定ダイアログを操作し、
監視間隔120秒の保存まで確認。2画面のスクリーンショットも確認。
- `go vet ./internal/seheweit ./internal/store ./internal/server` と `git diff --check` 成功。
- 本番DB・本番ネットワーク・取り込み済み外部カレンダーへのアクセスは行っていない。
8部品の初期分離はソース変更と検証まで完了。commit・push・本番展開は本作業に含めていない。
その後の明示的なリリース依頼により、同日 `v0.2.0` としてcommit・push・CI・本番反映を完了した。
[配布digest・移行・復旧点・開始ログ競合の継続課題](../../operations/deployments/2026-11-01-v0.2.0.md)
は初期分離の検証記録と分けて管理する。
+42
View File
@@ -0,0 +1,42 @@
# 21. 決定事項・矛盾の整理・未決事項
状態:既存設計の統合と、新規構想の判断台帳。設計案を実装完了に読み替えません。
## 原資料の読み替え
| 古い記述・混在 | 現在の読み方 |
| --- | --- |
| OIDCは形だけ、UserInfoだけを根拠にする | 署名・claim・state/nonce等のhardeningが入っている。詳細契約を優先 |
| CookieをやめてBearer一本にする | ブラウザCookieとAPI Bearerを目的分離して併用 |
| 匿名ナビゲーション・RSSは未実装 | 限定公開条件で実装済み。全metaや全依存物の公開ではない |
| 匿名公開はすべて毎Revision再承認 | 手動公開はそのまま。確認したStreet/Spotのみlive-head自動公開可能 |
| 個別記事の一覧登録だけが投稿方法 | 原子的な新規投稿APIと独立画面を追加済み |
| 同じCampusなら任意ノードでブログ射影 | 現行の参照型記事射影は同一インスタンスまで |
| resourceはブラウザを開いたときだけ記録 | 背景イベントで保存。表示更新と保存間隔は別 |
| groupプロフィールのnamespaceが設計のみ | 実装側にgroup画面・APIあり。機能ごとの完了度で読む |
| Asset名はinstance全体の名前 | mutable Asset名はDomain-local、Document hashは不変content identity |
| イベントキューがあるのでoutboxも完成 | queue、個別journal、transactional outbox配送は異なる |
| すべての文書はHTML正本 | 文章の正本HTMLと、バイナリ/固有形式の正本を区別 |
古い文書は削除せず、初期思想・変更理由・実装の追跡用に保持します。本書とコードが違う場合はまず差分を確認し、設計を理由に現在の安全制約を外しません。
## 新規ADRの案
| ID | 提案 | 状態・決めるための条件 |
| --- | --- | --- |
| ADR-O1 | organizerはtyped metadataと任意Page参照 | タスク初版は専用表+既存CAS Document Store(2026-09-24)。通常のObject閲覧・リンク・削除の権限境界を拡大せず、個人所有を確実に分離する。DBは捨てられる索引ではない。[実装契約](tasks-implementation.md) |
| ADR-O2 | task共通部+ticket workflow | 提案。番号・外部依頼者・遷移権限の要件確認 |
| ADR-O3 | ICS/vCard入出力→read-only DAV→双方向 | 推奨順。対象サーバー/clientを選んで互換表を作る |
| ADR-O4 | 簡易利用は内蔵正本、本格運用はFreshRSS正本+PagesWiki connector+newsboat | 2026-11-08利用者指定で採用。内蔵・FreshRSS読取・共通UI・OPML入出力まで実装。source単位で正本を分離し、自動fallbackや二重管理はしない。外部書戻しは残件。実環境の接続先・版・資格情報は別途指定 |
| ADR-O5 | IMAP read-only、Wikiコピーは明示操作 | 初期設計方針。providerとsampleが実装前提 |
| ADR-O6 | HTMLパーツを先行し全面WYSIWYGは保留 | 提案。round-tripと操作改善の評価でgo/no-go |
| ADR-O7 | Wiki Pathと外部UID/hrefを分離 | 共通の設計制約。rename fixtureを全connectorで実施 |
| ADR-O8 | 外部stateの変更は別段階の明示権限 | 共通の設計制約。既読・削除・送信の監査と確認 |
## 未決事項
利用するFreshRSSの接続先・版・資格情報、対象CalDAV/CardDAV client、IMAP provider、taskの共有単位、ticketの依頼者公開範囲、保持期限、外部接続を許可するhost、RPO/RTO、WYSIWYG採用基準です。これらが未決でもlocal fixtureと設計は進められますが、外部サービスを勝手に立ち上げたり資格情報を流用したりしません。
## 文書運用
新しい決定には理由・代案・互換影響・試験を残します。変更履歴ledgerと現状の説明を分けます。日本語統合版から詳細原資料へ辿れる対応表を維持し、英語版は同じchapter IDで追従させます。
+194
View File
@@ -0,0 +1,194 @@
# DR:単一書き込みノードと検証済み復旧世代
状態:2026-09-27、設計と第1段階の実装。自動フェイルオーバー、オンライン同期、設定画面は未実装。本番への適用・実データでの復旧訓練は別の承認を得て行う。
**次段階の詳細設計DR-D1が完了(文書のみ)**:
[管理主系・データ主系と再参加](dr-authority-and-rejoin.md)、
[通信・同期・切替protocol](dr-protocol-and-sync.md)、
[実施計画・受入試験](dr-delivery-plan.md)。
本書は既存offline実装の記録として保持する。次段階の判断・実装順はDR-D1を優先する。
DBは別件の変更対象のため、本詳細設計作業ではDB・schema・実装コードを変更していない。
## 1. 基本方針
Campus / Domain の意味、UUID、FQDN、所属関係を変えず、**そのデータを動かすインスタンスの復旧構成**を追加する。通常時は primary だけが書き込み、standby は検証済みの過去世代を読み取り専用で扱う。ネットワーク疎通失敗だけでは昇格しない。
「相手が落ちた」と「こちらから相手へ接続できない」は区別できない。2台だけで相互監視して両方が書き込みを開始すると、SQLite のトランザクションが各ノードで正常でも、全体では異なる正本が生まれる。旧 primary の電源・ストレージ・通信経路を外部で確実に隔離する **fencing** を昇格の前提とする。初版のコマンドは運用者の確認記録を残すもので、fencing 自体を実行・保証しない。
### 復旧の単位
現状の `pages.db` は複数 Campus / Domain の情報を共有する。したがって第1段階は **データディレクトリ全体=1インスタンス**を複製・昇格する。Campus の設定欄だけで1 Campusを昇格させ、同じDB内の別Campusを読み取り専用に残す操作は提供しない。
コントローラと配下の管理インスタンスはそれぞれ別の復旧単位。コントローラのスナップショットにはリモート配置 Domain の設定は含まれるが、別インスタンスにある記事の実体までは含まれない。それぞれ保全する。
## 2. 論理モデル
| 項目 | 意味・保全先 |
| --- | --- |
| Campus / Domain UUID・FQDN | 既存の論理 identity。復旧でも変更しない |
| `cluster_id` | 同じ復旧単位の primary / standby をまとめるUUID |
| `node_id` | DRノード固有のUUID。復元時に新規採番する |
| `primary_node_id` | スナップショットの作成元、または昇格後の自ノード |
| `role` | `primary` / `standby`。プロセス起動時に読み、稼働中には変更しない |
| `epoch` | 手動昇格の監査世代。**分散ロック・署名manifestのfencing tokenではない** |
| `snapshot_id`, `captured_at` | 復元世代と保全開始時刻。再起動で鮮度をリセットしない |
| `max_read_age_seconds` | 待機系の閲覧期限。初期値3,600秒、設定範囲1〜86,400秒 |
| `workers_enabled` | キュー、GC、管理Campus reconcilerの再開許可。復元・昇格直後はfalse |
| `promotion_history` | 時刻、旧primary UUID、隔離の証跡、復元世代、epoch |
設定は `data/dr-node.json`。これは**ノード固有**なのでprimaryのファイルをstandbyへ丸ごとコピーしない。`dr restore` がstandby用に生成する。`dr-node.json` を削除して昇格してはいけない。設定のない既存インスタンスは後方互換のため従来の書き込み可能起動となる。
DRホストの起動環境には `PAGESWIKI_DR_EXPECT_ROLE=standby` を指定する。設定が欠けた空volumeや、異なるroleのvolumeを誤mountした場合はDBを作らず起動を拒否する。昇格後に再起動するときだけ、配置設定も `primary` へ変更する。これは役割を変更する設定ではなく、期待した役割と実際が一致することを確認する安全装置。
既存の `instance-ed25519.key` と認証鍵は復旧する論理サービスの鍵であり、DRの物理 `node_id` とは別。鍵の複製を持つ旧ノードは確実に隔離する。
## 3. 初版の状態遷移とAPI
1. 既存 primary を停止し、`dr init` でノード設定を作成する。
2. 全writer停止中に `dr snapshot` で世代を作成する。primaryは完了後に再開できる。
3. 別ホストへ転送し、`dr verify` → `dr restore`。復元結果は常にstandby。
4. standby を別のローカルデータディレクトリから起動する。データvolumeは **read-only mount** とする。primaryとSQLiteファイルを共有しない。
5. 障害時は旧primaryを永続的に隔離し、standbyも停止して `dr promote`。
6. 復元後の権限・鍵・設定・キューを確認し、必要な修正をした上で、書き込み可能mountで再起動する。LB/DNSの切替は外部運用。
7. キューの重複実行・外部送信を検討して、停止中に `dr resume-workers`。再起動でジョブとreconcilerが動く。
`dr promote` と `dr resume-workers` はホスト運用者向けCLIに限定する。`campus_admin` や `domain_admin` が他Campusを巻き込んでインスタンス全体の役割を変えるHTTP APIは設けない。
### 待機系の動作
- SQLiteを `mode=ro` + `query_only` で開く。マイグレーション、Ensure系の初期化、認証鍵生成を行わない。
- 書き込みHTTPメソッドを503で拒否する。認証発行、エディタ起動、外部エディタ起動、外部図表レンダリング、設定・連携・インポートも拒否する。POSTを成功扱いで捨てない。
- GC、イベントスケジューラ、preview生成、外部カレンダー同期、Campus reconcilerを起動しない。
- 普通のWiki文書・RAWの読み取りは従来の認可を通す。既存sessionもスナップショット時点のものに限る。新規ログインは不可。
- **初版のorganizer/CalDAVと拡張機能APIはstandbyでは503**。既存の読み取り経路にDB索引の修復・資格情報の更新があるため。昇格後は通常の機能を使える。全機能のread replicaを実装済みとはしない。
- レスポンスは `X-PagesWiki-DR-Role: standby`、`Cache-Control: private, no-store`。RAWの長期cache指定よりも優先する。
- 古くなった復元世代からの閲覧は503で閉じる。`/health` は生存確認だけで、書き込み可能の判定に使用しない。
- `GET /health/read` はDB接続と閲覧期限を検査する。`GET /health/write` はstandbyで常に503。primary側の200も**他ノードが停止している証明ではない**。
公開範囲縮小、アカウント失効、セッション失効は保全時点より後の変更が反映されない。閲覧期限はその時間を制限するだけで即時失効を保証しない。第1段階ではstandbyは運用者の隔離された検証環境とし、通常の読み取りLBには組み込まない。実際に外部公開へ切り替える前に、復旧時点以降の権限縮小・失効を再適用する。
## 4. バックアップの内容と整合性
| 対象 | 取扱い |
| --- | --- |
| `pages.db` | SQLite Backup APIで複製。コミット済みWALを含め、rowidを保存する |
| `documents/**` | 記事・asset・taskのCASと、カレンダーの月別ICS・metadataを含む |
| `themes/**` | ローカルテーマ |
| `auth.key`, `instance-ed25519.key` | 存在する場合に保全。極めて機密性が高い |
| `federation/**` | 管理Campus参加token・endpointなどの復旧情報 |
| `dr-node.json` | **除外**。復元先でstandby用に新規生成 |
| preview・一時import・編集中adapter workspace | 除外。未保存編集は復旧対象外 |
| Docker image、Compose、環境変数、TLS鍵、外部volume | このCLIの対象外。別の保全対象台帳で管理 |
`PAGESWIKI_AUTH_KEY` を環境変数で与えている場合、それも別の暗号化されたsecret保全が必要。復旧先に**同じ値**を供給しないと既存認証・暗号化情報を復旧できない。CLIは環境変数の秘密値をmanifestに書かない。復旧に使用するアプリのimage digestと設定・secretの世代を運用記録に残し、まず同じバージョンで復元する。
### なぜ初版は停止バックアップか
SQLite Backup APIはDB内の整合性を確保するが、同時に更新中の `calendar.json` や別ファイルまで同じ論理時点へ固定しない。DBとファイル正本の境界にglobal snapshot barrierがない現状では、稼働中の `cp` / `rsync data/` を整合バックアップとはみなさない。
全てのWebサーバ、CLI、旧バイナリ、SQLite直接編集、外部writerを止める。新しいサーバ・データCLIは `.pageswiki-instance.lock` の共有ロックを保持し、保全・昇格は排他ロックで稼働中を拒否する。ただしこれは協調ロックであり、旧版や外部ツールを強制停止する機能ではない。`--writers-stopped` は実際の停止を運用者が確認したことの明示。
### 世代の作成・転送・復元
- 同じfilesystem内の一時ディレクトリへ作成し、ファイルfsync、manifest検証後にno-replaceのatomic renameで公開する。既存の保存先へ上書きしない。Linuxの `renameat2(RENAME_NOREPLACE)` を必要とする。
- manifestはsnapshot/cluster/source-node/epoch、保全開始・完了時刻、相対path・size・SHA-256の一覧を持つ。初版は最大100,000ファイル、manifest最大64MiB。
- 全payloadのSHA-256、CASファイル名、DBの `quick_check`、Revision/asset/task正本参照の存在、カレンダーmetadataから参照する月別世代のhashを確認する。カレンダーイベント本文の意味的再パースや外部同期はしない。
- symlink、絶対path、`..`、重複path、未知の保全root、manifestにないpayloadを拒否する。
- 転送はSSH等の認証された経路で行う。検証に渡す**期待manifest SHA-256を同じ未認証転送から自己申告させない**。現在のmanifestは署名付きではなく、期待hashの真正性は運用者が保証する。
- バックアップは暗号化済み保管先、暗号化transport、復旧担当者だけのアクセス権で扱う。CLI自身の暗号化・KMS連携は未実装。ファイル0600、ディレクトリ0700で生成する。
- 復元は新規ディレクトリへコピーし、コピー後もhash検証する。稼働中のデータへoverlayしない。失敗世代は公開せず、旧世代を勝手に削除しない。
- 世代更新は新規復元→standby停止→mount切替→再起動。初版の復元では物理node UUIDが毎回変わるため、その都度配置台帳を更新する。
世代保持の初期案は「直近3世代+日次7世代+週次4世代」。容量・法的保持・鍵ローテーションの要件を確認して決定する。自動削除はまだ実装しない。破損・誤削除が複製されても戻せるよう、同期先とは独立した世代バックアップを最低1系統持つ。
## 5. CLIの運用例
以下はLinux上のバイナリ例。実環境の絶対pathへ置換する。全コマンドは対象ディレクトリを明示し、保全先はデータディレクトリ外にする。コンテナでは一時CLI用コンテナへ同じdataとbackup保存先をmountし、entrypointに `/pageswiki` を指定する(image内の実際の配置を確認する)。
```sh
# 先にprimaryのサーバ・全writerを停止する。
pageswiki dr init --data-dir /srv/pageswiki/data
pageswiki dr status --data-dir /srv/pageswiki/data
pageswiki dr snapshot --data-dir /srv/pageswiki/data \
--output /srv/backup/pageswiki/generation-001 --writers-stopped
# 出力された manifest_sha256 を保管し、snapshotとともにDR側へ安全に転送する。
# この時点でprimaryは再開できる。
# DR側。期待hashは安全に受け取った値。<EXPECTED_SHA256> は置換する。
pageswiki dr verify --snapshot /srv/backup/pageswiki/generation-001 \
--manifest-sha256 '<EXPECTED_SHA256>'
pageswiki dr restore --snapshot /srv/backup/pageswiki/generation-001 \
--manifest-sha256 '<EXPECTED_SHA256>' --target /srv/pageswiki/standby-001
pageswiki dr status --data-dir /srv/pageswiki/standby-001
# standby-001を /data:ro として起動。元の論理FQDN/Hostと環境secretを維持する。
# PAGESWIKI_DR_EXPECT_ROLE=standby も起動環境に設定する。
```
昇格例:
```sh
# 旧primaryを停止・隔離し、再起動しても復帰しない外部fenceを確認する。
# standbyも停止し、失われる範囲(captured_at以降)を確認する。
pageswiki dr promote --data-dir /srv/pageswiki/standby-001 \
--fenced-node '<FORMER_PRIMARY_NODE_UUID>' \
--fence-reference 'incident-123: power fencing independently verified' \
--accept-data-loss
# 隔離環境でrw mount、PAGESWIKI_DR_EXPECT_ROLE=primaryへ切替。
# 権限失効を再適用し、起動・読み書き確認後に経路切替。
# ジョブはまだ停止。内容を確認・調整し、再びプロセスを停止してから実行する。
pageswiki dr resume-workers --data-dir /srv/pageswiki/standby-001 \
--reviewed-restored-jobs
# 再起動でスケジューラ/GC/Campus reconcilerが再開する。
```
旧primaryはそのまま再参加させない。旧データは隔離して差分調査・必要な手動救出用に保全し、新primaryの最新スナップショットから新しいstandbyを作る。古いDBで新primaryを上書きする「切り戻し」は禁止。自動の双方向マージは提供しない。
## 6. Campusコントローラに固有の注意
Campus manifestの署名、発行sequence、管理ノードの受理sequenceと参加tokenを一体で保全する。記事バックアップとmanifest配信は同義ではない。
初版では既存Campus FQDNをDR側へ向け直す方式を採り、管理ノードへ勝手に別Campusを登録しない。ホスト名・証明書・reverse proxyの扱いは復旧手順に含める。`PAGESWIKI_INSTANCE_FQDN` も復旧する論理サービスに合わせる。
**重大な未解決事項:** 保全後にprimaryが新しいmanifestを発行し、管理ノードが既に受理していた場合、復元したcontrollerのsequenceが後退して再送を拒否され得る。DRのローカル `epoch` を増やすだけでは既存署名プロトコルのsequence後退を解消しない。reconcilerを既定停止する理由の一つである。強引なDB sequence書換えで回避せず、次段階で「外部fenceで承認されたcontroller epoch+epoch内sequence」のプロトコルと管理側の受理規則を実装する。それまではcontroller配信の完全自動復旧を保証しない。
## 7. 次段階:オンライン同期と自動昇格の詳細設計
以下は初版時点の構想。詳細化後の実施順は [DR01〜DR13](dr-delivery-plan.md) を正とし、
probe/再参加gateと役割・writer境界の分離を先行する。実装済み項目ではない。
1. **オンライン整合checkpoint**:全正本writer(Web、CLI、calendar metadata、GC)で共有する短時間barrierを設ける。DB backup cutoffとcalendar metadata世代一覧を固定し、その世代が参照するCAS/monthly blobのGC pinを持つ。大きいblob転送はbarrier解除後に低優先度で行う。SQLite単独online backupをこの代替としない。
2. **転送と受理**:instance-boundの別用途資格情報、mTLSまたは短命署名、cluster/node/epoch/sequence固定。Campus参加tokenを丸用しない。CAS差分転送→hash確認→DBとmetadata世代の受理→active pointer切替の順。途中世代はservableにしない。retryは同じsnapshot IDで冪等、帯域・IO・CPU上限、foreground優先、同時転送数を設定可能にする。
3. **監視とUI**:設定画面の「運用 / DR」にノード、role、最終検証世代、保全時刻、受信時刻、遅延、容量、最終訓練結果を表示。Campus画面から復旧単位への関連リンクを出す。緑は「HTTP疎通」ではなく「検証済み世代・鮮度・役割整合」、黄は遅延、赤は破損/期限切れ。Domain設定へDRのhostnameを重複入力させない。
4. **operator権限分離**:インスタンス運用者がreplica/転送鍵/役割を管理し、campus_adminは所属Campusの復旧状態を参照する。共有DBを跨ぐ昇格・secret操作はcampus_adminだけでは許可しない。初版のCLIに後からAPIを接続する際も同じ境界とする。
5. **キュー・認証の復旧**:session/policy revocation watermarkを同期し、古いACLからのreadをfail closedにする。jobにepoch leaseとidempotency keyを導入し、外部副作用は送信先も含む重複制御または確認フローを設ける。lease期限だけで旧executorを安全とはみなさない。
6. **controller epoch**:署名対象にleader identityとepochを含め、管理インスタンスは承認された単調増加epochとsequenceを永続化する。epoch移行はout-of-bandで信頼したfencing承認だけが行えるようにする。古い鍵/epochの再送と復元後のrollbackを試験する。
7. **自動昇格**:第三のwitnessまたはconsensus storeと外部fencing backendを選択。lease失効時は旧primaryがcommitを拒否し、すべてのDB/ファイルwriterとジョブがepochを照合する。fence失敗・quorumなしでは書き込み停止を選び、二重writerを選ばない。これができるまで自動昇格ボタンや「疎通断だけでwrite ok」は実装しない。
8. **LB参照構成**:write readiness、read freshness、drain、長時間upload、WebSocket、Host検証、接続中断時の冪等再試行を統合する。read/write振り分けはHTTPメソッドだけで決めない。RO nodeをwrite routeへ入れない。
API案は `/_/api/v1/instance/dr/...` を予約候補とするが現時点で未登録。転送はユーザーのPath URIと分離し、`POST snapshots`、snapshot ID配下のmanifest/payload、`POST acknowledge` をinstance-to-instance限定で設ける。公開 `/_/api/v1/federation/enroll` にbackupの読み出し能力を追加しない。
## 8. RPO / RTOと受入基準
初版のRPOは「障害発生時刻 − 最後に**DR側で検証完了した**snapshotのcaptured_at」。保全予定間隔や転送開始時刻ではない。RTOには障害判断、外部fence、復元/検証、権限失効の再適用、サービス検査、経路切替を含む。目標値は実データ量と許容停止時間から別途決定する。RPO=0・無停止とは表明しない。
自動試験は合成データを使い、次を確認する。
- 未checkpointのコミット済みWALを取り込み、rowidを維持する。
- DB参照先の欠落・payload改変・manifest改変・path traversal・symlinkを拒否する。
- 同じsnapshotを繰り返しverifyしても書き換えない。復元先既存ディレクトリを上書きしない。
- サーバ/CLIの共有ロック中は保全・昇格を拒否する。
- 復元後はノードUUIDが別になり、role=standby、worker停止となる。
- standbyの起動とWiki読取でDBを変更せず、書込API/Store、外部起動、期限切れ閲覧を拒否する。認可をバイパスしない。
- 旧primary identity・隔離の証跡・データ損失受容のいずれかがない昇格を拒否する。
- 昇格後もjobを勝手に再送せず、明示的再開が必要。
実機の電源断、ネットワーク分断、鍵失効、容量不足、転送中断、DNS切替、controller sequence後退、復旧元と受信先のバージョン差異は別の復旧訓練が必要。訓練未完了を成功と扱わない。
## 9. 根拠資料と関連設計
- [SQLite Online Backup API](https://www.sqlite.org/backup.html):DB単位の整合したコピー。
- [SQLite WAL](https://www.sqlite.org/wal.html):WALとDBファイルの扱い、共有filesystemの制約。
- [SQLite VACUUM](https://www.sqlite.org/lang_vacuum.html):rowidが変わり得るため初版のコピー方式には採用しない。
- [Campus / Domain](../campus-domain-model.md)、[署名manifest](../federation-manifest-integrity.md)、[運用と保全](operations.md)、[Phase 8](../implementation-roadmap.md#phase-8--controller-disaster-recovery-and-high-availability)。
+285
View File
@@ -0,0 +1,285 @@
# DR詳細設計:管理主系・データ主系・再参加
設計版:DR-D1、2026-09-27。**設計のみ。記載したAPI・状態機械・設定は未実装。**
今回の作業は文書のみを変更し、DB、schema、migration、実行コード、配置設定、本番データを変更しない。
既存の停止snapshot/手動昇格は [初版の実装・運用記録](disaster-recovery.md) に残す。
## 1. 文書体系・決定の適用範囲
- 本書:責務、identity、再参加状態、書き込み条件、障害時判断、設定・UI。
- [通信・同期・主導権移譲契約](dr-protocol-and-sync.md):メッセージ、URI、検証、切替と保全。
- [実施計画・受入試験](dr-delivery-plan.md):作業単位、変更予定箇所、依存、導入・撤退手順。
DR-D1は初版文書の「次段階」の設計を具体化・更新する。初版の `role` や `epoch` の意味を
黙って変更せず、旧形式の移行は別の明示的操作とする。以下のMUST相当の条件は、
「できれば実装する」事項でなく、機能有効化の前提とする。
DB別件の完了を待たず、wire型・純粋な状態機械・認証probe・表示の実装準備はできる。
一方、writerを開く接続、online snapshot、manifest受理checkpointの永続化、
独立した主系切替は、保存層の契約確認と別途の実装承認まで無効とする。
## 2. 決定一覧
| ID | 決定 |
| --- | --- |
| D01 | Campusの管理primaryとData replica groupの書き込みprimaryを分離する |
| D02 | Instanceに全体共通のprimaryフラグを持たせず、役割はgroupごとに表現する |
| D03 | 同じ物理writer境界を共有する役割は連動切替する。論理分離だけで独立writerにしない |
| D04 | DR対象プロセスは再起動時に必ずrecoveringから入る。保存済みroleではwriterを開かない |
| D05 | discovery/probeは情報収集。参加許可、昇格承認、正本の決定とは別 |
| D06 | 発見したpeerの数や、最大の自己申告epochだけでは主系を選ばない |
| D07 | 主系選定の既定はmanual、preempt既定はoff。自動モードは別の能力認定が必要 |
| D08 | 自動裁定の最初の実装先は外部etcd v3 APIとする。PagesWiki独自Raftは作らない |
| D09 | lease切れ・HTTP不通だけでは旧writer停止の証拠にならない。新grant前にfencingを要する |
| D10 | 元primaryの未同期更新は隔離保存し、現primaryへ自動マージしない |
| D11 | 共有DB環境での最初の同期単位はデータディレクトリ全体。Domain単位のSQL差分複製はしない |
| D12 | バックアップ世代、データ書込世代、管理manifest世代を混同しない |
| D13 | 状態収集/authority監視は通常のイベントキューに依存させない。大容量転送は低優先度 |
| D14 | 主系復帰は同期→drain→最終checkpoint→権限移譲の協調的切替とする |
etcd採用は自動モードの設計上のprovider選択であり、この変更でサービスを導入しない。
運用でetcdを用意しない場合もmanualの保全・復元を使える。実際のprovider配置、電源/保存先の
fence手段、容量と目標RPO/RTOは導入時の入力であり、推測して本番設定に書かない。
## 3. 三つの境界
### 3.1 論理サービス
| 対象 | 不変のidentity | 主系の責務 | 待機系に複製するもの |
| --- | --- | --- | --- |
| Controller group(CG) | group UUID、Campus UUID | Campus設定、Domain配置、管理側継承情報、設定manifest発行 | 管理正本、発行履歴、承認済み構成、復旧用鍵参照 |
| Data replica group(DG) | group UUID、dataset UUID | Domainのページ/asset/カレンダー/タスク/ローカル権限等の更新 | metadata、CAS、可変正本、失効情報、outbox/checkpoint |
| Instance | node UUID、ノード固有鍵 | 上記役割を実行する場所 | Instance自体を業務正本のownerとしない |
1 Campusにつき管理authorityは1 CG。1 DGは複数Domainを持てる。1 Domainは同一時点では
1 DGへ配置する。複数Campusのデータが1 DGに共存することも、既存DB構成の間は許容する。
CGのprimaryはDGのprimaryである必要はない。ただし次節の物理境界制約に従う。
Campus/DomainのUUID、FQDN、Path、ユーザー領域のURIは維持する。
Domain配置の将来形式は `replica_group_id` を正本にし、現在primaryのFQDNは解決結果とする。
物理ノード移動をDomainの改名として記録しない。旧 `placement_instance_fqdn` は移行中の
互換表示に限り、二つの値を別々のauthorityが更新する方式は採らない。
### 3.2 保存領域のwriter境界
`WriterResource` はDB、可変正本ストア等の実際の排他単位を表す。ローカルpath文字列だけで
なく保存層が提供する安定したresource identityで識別する。複製コピー同士も同じ論理正本の
writer resourceに属する。NFS上のSQLiteを別resourceと見せるような登録を拒否する。
各役割は更新するresourceの集合を宣言し、共有resourceを持つ役割の推移的なまとまりを
`coupling_bundle` とする。同じbundleのwriter許可は一括で一つのnodeにだけ与える。
例:CG1とDG1がpages.dbを共有、CG2も同じDBを更新するなら三役割すべてを連動させる。
テーブル名が異なっていても、DB全体を物理snapshotで複製する間は分離しない。
独立切替を許可する条件は、保存担当が次の能力を明示すること。
- 対象の正本、派生projection、署名発行状態に独立した所有者とwriter境界がある。
- 役割間の更新はversion付きのprotocolとして受理でき、別役割のDBへ直接書かない。
- 対象resourceへの旧writerのアクセスを実際に停止できる。
- role/resource対応の構成世代が全参加者で一致する。
能力が未申告なら `shared-instance` とし、独立切替要求は `WRITER_BOUNDARY_COUPLED` で拒否する。
不足能力を理由にDB分割・DDLを自動実行しない。
### 3.3 障害・隔離の影響範囲
role単位で保存領域が分かれても、fenceがホスト電源断なら同居する全groupへ影響する。
切替planは「writer連動範囲」と「fenceの影響範囲」を別々に計算する。
他groupのprimaryまで停止するplanは、自動preemptしない。運用者の明示承認と、影響groupの
サービス継続計画を必要とする。group単位のfence能力があると偽装して回避しない。
## 4. identity、保存位置、世代
以下は論理レコードの契約であり、RDBのテーブル定義ではない。
| レコード/値 | 保存主体 | 更新ルール |
| --- | --- | --- |
| Membership | AuthorityStore | group種別、member/learner、鍵、resource対応。expected revisionでCAS |
| NodeIdentity | ノードローカル保護領域 | node UUIDと専用鍵。業務snapshotから他ノードへ複製しない |
| `boot_id` | 各プロセス | 起動ごとに乱数UUID。過去のgrant/probe応答を再利用しない |
| `trust_generation` | AuthorityStore+運用保全 | authority自身の災害復旧時に明示更新するtrust世代 |
| `control_epoch` | CG authority | CGの主導権交代ごとに単調増加 |
| `data_epoch` | DG authority | DGの書込権交代ごとに単調増加 |
| `resource_epoch` | writer resource authority | 共有保存境界の許可更新。bundle全体で矛盾なく確定 |
| `manifest_sequence` | CGの管理正本 | control_epoch内の管理配信順序。data_epochとは比較しない |
| Checkpoint | 保存層+受信側 | dataset、epoch、cut ID、親checkpoint、内容hash、検証ACK |
| Operation | AuthorityStore | 永続したintent、phase、expected世代、証跡、結果。timeoutで削除しない |
| Node runtime state | メモリ+ノードローカル記録 | 観測状態。書込権の正本にはしない |
初版 `dr-node.json` の `epoch` は監査値なので、control/data epochへコピーしてauthorityと
みなさない。旧 `cluster_id` は移行mapで既存の復旧単位へ対応させる。
DR-D1では物理ノードを維持した世代更新でnode UUIDを変えず、`dataset_generation`を更新する。
新しい物理ノードは新規identityを登録する。初版のrestoreによる採番との差は移行計画に明記する。
単に最大のepochを採用しない。有効なのは、信頼済みauthorityが登録したtrust世代・group・
membershipに結び付いた証明だけ。timestampは監視用で、履歴の新旧や正本選定に使わない。
## 5. 設定契約
設定案の中心は次のGroupPolicy。ノード一覧に秘密値を埋め込まず `secret_ref` を用いる。
| 設定 | 既定/検証 | 意味 |
| --- | --- | --- |
| `kind` | 必須:control/data | 二つの役割を混同しない |
| `coordination_mode` | manual | manual / coordinated。Campusの既存authority_modeとは別項目。自動モードは必要能力の全検査を要求 |
| `members` / `learners` | 明示登録、重複不可 | 発見しただけのpeerをmemberへ追加しない |
| `preferred_node_id` | 未指定 | 期待する主系。実際のprimaryを上書きする値ではない |
| `preempt` | off | off / manual / automatic。controlとdataで別設定 |
| `stable_for_seconds` | 300、範囲60〜86400 | preferred nodeが継続して正常・同期済みである必要時間 |
| `switch_cooldown_seconds` | 1800、stable以上 | 交代直後の再切替を抑制。障害対応は別の明示操作 |
| `preempt_window` | 空=時間制限なし | 任意の許可時間帯。タイムゾーン必須、DSTは計画時にUTCへ解決 |
| `probe_interval_seconds` | 5、範囲2〜60 | ±20% jitter。leaseの更新周期とは別 |
| `probe_timeout_seconds` | 2、範囲1〜10 | 接続・応答の合計deadline |
| `suspect_after_failures` | 3、範囲2〜10 | 表示をsuspectにするだけ。これで昇格しない |
| `read_policy` | disabled | disabled / fresh-authorized。失効watermark未対応なら後者は選べない |
| `max_read_staleness_seconds` | 300、範囲1〜3600 | read replica能力実装後に適用。初版3600秒設定とは別version |
| `rpo_target_seconds` | 必須・導入時入力 | 目標。同期の成功や無損失を保証するフラグではない |
| `rto_target_seconds` | 必須・導入時入力 | fence・起動・routingを含む実測比較の目標 |
| `data_loss_policy` | require-approval | 自動failoverで不明/未複製の損失を黙認しない |
| `fence_provider_ref` | coordinatedでは必須 | 対象identity・resource・影響範囲を検証できるprovider |
上記は試験開始用の既定値。実環境のRTTやI/Oに応じて変更できるが、同期完了、現主系の隔離、
相互排他、未知世代拒否といった安全条件は設定で無効化できない。
preempt=automaticはcoordinated、全target能力、writer境界、適合fence、失効同期を必須とする。
manualでは自動選挙も自動preemptも行わず、復帰しただけではwriterを再開しない。
## 6. 再参加の状態機械
設定上の優先順位、現在の役割、実行状態を別フィールドにする。
状態遷移の入力は検証済みイベントのみ。単なるHTTP 200やpeerの文字列申告では遷移しない。
| 状態 | 入る条件 | 許可するもの | 次状態の条件 |
| --- | --- | --- | --- |
| `recovering` | 全DR対象プロセスの起動、許可失効、重大な構成変更 | ローカル検証、認証済みprobe | identity/構成が有効ならlearning |
| `learning` | 書込gateを閉じてauthority/peerを照合中 | probe、checkpoint一覧、同期plan | authoritative state一致→syncing。矛盾→conflict |
| `syncing` | 正しいsourceとbase世代が確定 | 不変世代の受信・検証 | 内容・schema・ACL等が一致→standby |
| `standby` | 検証済み世代、最新membership | 同期追従、能力認定済みreadのみ | 書込grant取得→activating |
| `activating` | 新grantとfence証跡が確定 | root切替、互換性検証、gate接続 | 二重確認+readiness成立→primary |
| `primary` | group/resource/bootすべて一致した有効grant | 許可範囲のread/write | 計画移譲→draining。許可失効→recovering |
| `draining` | operationが停止cutを要求 | 既存処理の完了待ち、checkpoint | writer停止証明後standby/停止。失敗ならoperation停止 |
| `conflict` | 履歴分岐、同一世代で異hash、trust不一致 | 証拠保全、管理状態参照 | 運用者のreseed/解決planのみ |
| `blocked` | 同期先不在、容量不足、fence不明、互換性不足 | 状態・監視 | 原因解消後、learningから再検査 |
learning/syncing中は一般のreadも公開しない。復元されたprivate文書と古いACLをうっかり公開しない。
standbyでreadを許す場合も、内容の鮮度とは別に必要な失効watermarkを確認する。
複数groupを持つInstanceは各groupの状態を表示する。単一の緑ランプで全役割をprimaryと見せない。
### 起動順序
1. ノードの専用identity、deployment期待値、承認済みmembershipを読む。
2. 通常HTTP handler、migrations、worker、CLI接続より先にすべてのwriter gateを閉じる。
3. read-only互換性検査と管理用control listenerだけを起動する。
4. 新boot_idで認証probeし、authorityの現在値と照合する。誰にも到達できなければblocked。
5. 必要なら旧ローカルデータを隔離保全し、正しいsourceから新generationへ同期する。
6. 同期済みでもまずstandby。新bootに対するgrant取得を別処理で行う。
正常終了時にprimaryだった場合もこの順序を省略しない。単一ノードの非DR運用は従来起動を
維持するが、DR設定の欠落を非DRへ自動フォールバックする経路はdeployment guardで拒否する。
## 7. 書き込み許可の条件
次の論理式をすべて満たす場合だけwriterを開く。
```text
CanWrite(role, resource, boot) =
approved_membership AND validated_trust_generation
AND active_group_grant(holder=node, boot=boot, epoch=current)
AND active_resource_grant(holder=node, bundle=current)
AND verified_dataset AND schema_compatible AND revocations_applied
AND local_gate_open AND NOT draining AND NOT expired
```
Web/APIだけでなくCalDAV、CLI、managed設定受理、migrations、GC、asset処理、calendar保存、
外部同期、outbox配信もwriter一覧に含める。HTTP GETでも修復/発行をする経路はwriteである。
RO standbyへのmanifest適用は、そのreplicaのauthoritative DBへ直接行わず、現DG primaryへ
適用するか同期対象のprojectionとして受け取る。ローカル状態報告の保存先と業務正本を混ぜない。
許可喪失時は新規処理を拒否し、実行中処理をキャンセル・停止する。しかしプロセス停止や
ネットワーク遅延によりキャンセルが間に合わない可能性があるため、新primaryのgrant発行は
外部fence確認または認定済み保存側fencingの完了まで待つ。local mutex/lease確認だけでは代替不可。
## 8. 裁定・信頼モデル
発見peer、複製peer、authority参加者は別。既存Campus参加トークンにbackup読出しや昇格権を
追加しない。ノード間通信は専用identityとgroup限定の許可を使う。
coordinatedの初段providerは外部etcdの通常整合性readと比較更新transactionを使う。
watchは再確認のきっかけで、watch結果だけから新grantを発行しない。epoch、membership、
operation intentは期限切れで消さず、leaseは生存・短期許可にのみ用いる。
etcdのKV transactionとwatchの保証は異なるためである。[etcd API guarantees](https://etcd.io/docs/v3.5/learning/api_guarantees/)
authorityは業務snapshotと別の保全・失効境界に置く。旧authority backupを戻してepochを
巻き戻すことは禁止。authority全損時は全writerをfenceし、新trust_generationを
out-of-bandで承認・再配布するbreak-glass手順を要求する。単にepochを大きくする操作ではない。
authority quorumが失われた場合は新grantを発行しない。既存writerも許可更新不能なら閉じる。
2台のPagesWikiを相互監視するだけではcoordinated modeを認定しない。
etcdは別途適切な障害領域に配置する。配置数やfence backendの接続情報は導入計画で確定する。
manual modeの運用者証跡は自動検証済みfenceと区別し、UIと監査に表示する。
## 9. コントローラとデータ側の障害マトリクス
| 事象 | 管理側 | データ側 | 判定 |
| --- | --- | --- | --- |
| CG停止、DG正常 | 配置変更・新manifestを停止 | 有効なdata grantと許可済み設定で継続 | CGへの疎通だけをDGの書込条件にしない |
| DG停止、CG正常 | 管理操作は継続可 | 対象DGのfailover判定 | CGが配置文字列を書き換えるだけでは昇格不可 |
| 旧CG復帰 | learning→管理checkpoint同期 | DGの切替は不要 | CG preempt設定のみ評価 |
| 旧DG復帰 | 配置group identityを維持 | learning→現DG primaryから同期 | 古いデータを優先しない |
| 同じDBでCG/DGが別nodeを要求 | 連動planが必要 | 連動planが必要 | 独立要求を拒否 |
| primaryとreplicaが分断 | authoritative grantとfenceを照合 | 同左 | 応答者だけの多数決はしない |
| authorityのみ停止 | 更新許可の有効期間内だけ処理 | 同左 | 新grantなし、期限到達でwriter停止 |
| 最新受理manifestより復元内容が古い | 自動配信しない | 古い権限を再適用しない | 管理復旧の明示承認が必要 |
| role fenceが同居別groupも停止する | impact planを提示 | 対象groupも判定 | 自動preemptを拒否 |
## 10. UI・権限・監視
設定の「運用 / DR」を親にし、「概要」「グループ」「同期・保全」「切替履歴」を子パネルにする。
概要は1group 1行、kind / Campus・Domain範囲 / 実primary / preferred / state / 同期点 /
遅延 / 書込許可 / 原因を表示。ノード追加・ポリシー編集はアイコンからダイアログに収める。
フォルダ名やpathではなくnode・groupを選択させ、秘密は参照名だけ表示する。
キャンパス設定にはCGの役割・状態とDR画面へのリンクを追加する。Domain設定では配置先を
DG単位で選び、現在のnodeを補助表示する。旧配置方式は移行前であることを示す。
| 権限 | 許可 |
| --- | --- |
| instance operator(新しいホスト運用者権限) | node登録、trust/fence、backup秘密、bundle全体の切替・復元 |
| campus_admin | 自Campusの管理状態参照、ポリシー変更/切替の申請 |
| domain_admin | 自Domainの配置・同期状態参照。ホスト/鍵/他Domain情報は非表示 |
| replica service identity | 明示groupのprobe、許可された世代の同期。operator API不可 |
instance operatorはcampus_adminへの暗黙追加ではない。初段はホストCLIだけでこの権限を
実現し、既存認証DBに新roleを勝手に加えない。Webでのapplyは認可契約が完成するまで非表示。
切替UIはplan→影響範囲・損失・fence表示→承認→進行状態。操作中も平常画面をoverlayしない。
通知Inboxへ失敗/blocked/完了を集約し、probe失敗を毎回broadcastしない。
メトリクス:role state、grant残時間、最後のprobe成功、同期cut時刻、検証ACK時刻、
pending bytes、転送速度/制限、hash失敗数、fence結果、operation phase、未複製の更新有無。
UUID別の無制限時系列は避け、ノード/group別集計と履歴画面を分ける。秘密・文書本文をログに出さない。
緑=primary許可あり/検証済みstandby、黄=learning/syncing/suspect、赤=conflict/blocked。
色だけで判断させずラベルと原因コードを併記し、日本語・英語キーを用意する。
## 11. 保存担当との接続条件
DB変更の担当へ渡す成果物は、テーブル要求ではなく以下の能力・整合性契約。
- `DescribeResources`:各正本とwriter排他境界、共有するrole、schema対応範囲。
- `InspectCheckpoint`:正本を変更せず、適用世代・内容digest・schema fingerprintを取得。
- `FreezeWriters/ReleaseBarrier`:全writerの停止、未完了処理、最終cutの確定。
- `CaptureCheckpoint/Pin/Unpin`:DBと可変正本を同じcutへ固定し、必要blobをGCから保護。
- `StageGeneration/Verify/ActivateGeneration`:非稼働rootへ復元し、検証後に切替。
- `ApplyControlManifest`:管理内容と受理checkpointを同じ永続化境界で確定。
- `CheckWritePermit`:対応できる場合のみ保存側fencingとして能力宣言。
詳細な入出力と失敗時条件は [protocol](dr-protocol-and-sync.md) に定義する。
契約を満たす実装がなければoffline capture/shared-instance/manualのままとし、DB別件を
妨げるDDL、直接SQL、独自の並行正本ファイルを追加しない。
## 12. 参考と非目標
Learnerという考え方は、同期前のノードを選挙・サービス提供から外すための参考である。
PagesWikiの状態名をetcdの状態と同一視しない。[etcd learner design](https://etcd.io/docs/v3.5/learning/design-learner/)
非目標:active-active更新、複製DBのSQL行単位マージ、任意LAN参加、gossipによる独自投票、
DNSだけによるfencing、DB製品選択の変更、既存Campus/FQDNの改名、未同期更新の黙った破棄。
同期とバックアップは別であり、replicaの存在を履歴世代の保全の代替としない。
+231
View File
@@ -0,0 +1,231 @@
# DR詳細設計:実施計画・受入試験・導入判定
設計版:DR-D1、2026-09-27。**詳細設計完了、実装作業は未着手。**
ここでいう実装作業はDR-D1の再参加・役割分離protocol。既存のoffline DR CLIは別の初版。
今回の成果物は文書のみであり、DB・migration・本番配置を変更する作業指示ではない。
## 1. 設計成果物と完了条件
| 成果物 | 本版で定義した範囲 |
| --- | --- |
| [役割・再参加](dr-authority-and-rejoin.md) | CG/DG/Instance、writer連動境界、identity、状態機械、設定・UI・権限 |
| [protocol・同期](dr-protocol-and-sync.md) | 認証probe、API、authority契約、checkpoint、manifest v3案、切替・失敗時動作 |
| 本書 | 実施順、予定変更箇所、DB別件への接続条件、試験、導入/撤退手順 |
| [初版記録](disaster-recovery.md) | 既存機能と制約を保持し、DR-D1と区別 |
- [x] primaryを管理/データに分け、物理保存境界による連動条件を記述。
- [x] 起動・probe・Learning・同期・昇格・preempt・失敗からの再参加を定義。
- [x] 身元、世代、許可、同期内容、承認、未知損失の扱いを定義。
- [x] API入出力、サイズ上限、再試行、冪等性、切替phase、認可主体を定義。
- [x] DBを変更せずに準備できる範囲と、保存担当との合意待ち範囲を分離。
- [x] 実装タスク、依存、合格条件、障害注入試験、導入/撤退条件を定義。
- [ ] 実装・実機でのfencing認定・復旧訓練・本番反映。
「詳細設計完了」は実装方針と判断規則を定義した状態であり、未知の本番設備の能力を保証した
意味ではない。RPO/RTO値、ノード数/FQDN、fence機器、容量、鍵保管先は導入時の必須入力として残す。
これらを未設定のまま自動運転へ移行することはできない。
## 2. DB別件との作業境界
本タスクでは以下を行わない。
- アプリDBへのSELECTを含むアクセス、DBのコピー・バックアップ・復元、schema inspectの実行。
- SQL/DDL/migration/repositoryの編集、コード生成、サーバ起動、CLIによるデータ操作。
- compose、認証鍵、federation token、node設定、稼働コンテナ、外部カレンダーへのアクセス。
- 文書を本番Wikiへアップロードする操作(アップロードもDBへの書き込みになるため)。
読み取る対象は設計書と必要最小限のソースコードだけ。ドキュメントの静的整合性確認を実施する。
別件の変更を前提にDBの物理分割方法やテーブル名を確定させない。
保存担当へ渡すinterface要求は次の通り。応答はSQLの実装ではなく、能力の宣言とテスト結果で受け取る。
| 契約 | 保存担当に確認する事項 | 未提供時の挙動 |
| --- | --- | --- |
| Resource map | metadata/可変正本/署名発行状態が共有するwriter境界 | shared-instance連動のみ |
| Schema inspection | 非破壊・read-onlyでの互換性/世代取得 | 復旧世代をservableにしない |
| Consistent cut | DBとcalendar等を同じ論理時点で固定できる範囲 | 全writer停止snapshotのみ |
| Durable pin | snapshot/GC/転送/再起動を跨ぐ参照保持 | online同期不可 |
| Restore activation | 新generationの検証、root切替、途中停止からの復旧 | 自動復元不可、既存手動手順のみ |
| Manifest apply | 管理内容と受理位置のatomic commit | manifest v3配信不可 |
| Write fencing | 旧許可を実際の書込み先で拒否できるか | 外部fence必須 |
| Durability receipt | client成功応答済み更新がreplica上で永続化された境界 | 自動無損失failover不可 |
| Identity/secret inventory | logical service keyとphysical node keyの区別 | nodeを自動参加させない |
合意前に作業できるのは仕様型、pure state machine、mock provider、観測専用probe、
静的UI/認可契約の設計まで。runtimeへのwrite gate接続も、別件の保存経路を網羅できることを
確認してから行う。この順序は「DBが共有のままでもroleのflagだけ先に動かす」ことを禁止する。
## 3. 実装タスクと依存順
以下は将来の予定。行が存在しても対象ソースを今回変更しない。
| ID | 実施内容 | 主な変更予定箇所 | 依存 | 完了判定 |
| --- | --- | --- | --- | --- |
| DR01 | version付き型、ID/epoch/構成検証、resource coupling planner | `internal/dr`の新model/config/planner | 設計D1 | 不正構成・独立不可・未知版を拒否。DB依存なし |
| DR02 | pure state machineとoperation phase reducer、fake clock | `internal/dr`のstate/operation | DR01 | 不変条件N1〜N10の遷移試験。失敗時はwriterを開かない |
| DR03 | 専用node identity、mTLS/署名/nonce、観測専用probe | `internal/dr`のtransport、server管理route | DR01 | 未認証/別group/replayを拒否。probeでroleが変わらない |
| DR04 | runtime startup gateの接続、非DR互換、旧CLI禁止 | `cmd/pageswiki/main.go`、server middleware、store/CLI全writer入口 | DR02/03+保存経路合意 | 再起動で必ずrecovering。起動修復/workerも許可前にwriteしない |
| DR05 | AuthorityStore interface、manual provider、etcd providerと限定service | `internal/dr`のauthority、専用運用cmd | DR01/02 | CAS競合/timeout/再起動でgrantが二重にならない |
| DR06 | checkpoint adapter、世代/pin台帳、schema/read-only検証 | 既存`internal/dr/snapshot.go`と保存担当adapter | DR01+保存契約合意 | offline fixtureから再開可能。online未対応を明示拒否 |
| DR07 | 認証済みmanifest/Range転送、recipient ACK、負荷制御 | DR transport、documentstore adapter、scheduler表示 | DR03/06 | 中断再開/破損/別groupアクセス/容量不足を検出 |
| DR08 | v1/v2 anchor移行、管理manifest v3発行/受理/ACK | federation manifest、enroll/reconcile、受理repository | DR03/05/06+受理永続化合意 | 同内容再送・旧epoch・欠落履歴・rollbackを正しく処理 |
| DR09 | fence provider契約、impact planner、証跡照合 | DR fence/operation、運用connector | DR05 | 隔離未確認/誤node/別boot/別group影響で昇格拒否 |
| DR10 | 手動switchover/failover/reseedをoperationへ統合 | DR CLI/API、root activation、routing adapter | DR04〜09 | 完全fixtureでroleごとの切替と共有DB連動が成立 |
| DR11 | preempt planner、自動coordinated gate、cooldown | DR policy executor | DR10+設備認定 | flapping抑制、0損失の計画復帰、不可条件時の提案のみ |
| DR12 | DR管理UI、i18n、health、Inbox、監査集約 | config panels、status APIs、metrics | DR03/10+operator認可 | group別表示、権限境界、秘密非表示、plan/applyが一致 |
| DR13 | v0移行/本番runbook、復旧訓練、負荷/障害試験 | docs/operations、隔離検証環境 | DR10/12 | 受入matrix全通過、RPO/RTO実測、明示承認後のみ有効化 |
DR04以降のコード接続は、DB別件の差分が落ち着き接続点が合意された後。
DR08のschema実装は保存担当の変更計画に載せ、DRタスクが独自migrationを追加しない。
DR09の機器connectorは任意shell入力ではなく、許可された設備・resource IDへのtyped操作とする。
### 段階リリース
| 段階 | 提供するもの | 明示的に提供しないもの |
| --- | --- | --- |
| L0:現行初版 | offline snapshot、検証、手動復元・昇格 | 再参加判定、オンライン同期、自動fencing |
| L1:観測 | DR01〜03、状態の参照UI | 自動昇格、既存writerのruntime変更 |
| L2:再参加gate | DR04/05、operator承認下の起動、shared bundle判定 | 未接続の保存経路があるprofileの有効化 |
| L3:同期・手動切替 | DR06〜10、認証転送、group別の計画操作 | 未認定providerの自動切替 |
| L4:自動主系復帰 | 認定済みDR11/12、coordinated preempt | snapshotのみを根拠にした無人・無損失failover |
| L5:自動障害復旧 | replica durability receipt等が揃うprofileのみ | 未複製更新や未知損失の黙認 |
各段階で有効化フラグは既定off。capabilityの揃わない段階へUI操作で飛び越せない。
L1の観測機能が動作しても「Learningから自動で昇格できる」と表示しない。
## 4. CLI・API・UIの実施内容
将来CLI案(すべて未実装、DB直接更新用コマンドではない):
```text
pageswiki dr config validate --file <operator-config>
pageswiki dr groups status --group <id>
pageswiki dr rejoin plan --group <id> --node <id>
pageswiki dr switchover plan --group <id> --to <id>
pageswiki dr failover plan --group <id> --to <id>
pageswiki dr operations apply --plan <id> --plan-sha256 <hash>
pageswiki dr operations show --id <id>
pageswiki dr operations cancel --id <id>
pageswiki dr reseed plan --group <id> --node <id> --checkpoint <id>
```
現行の `dr promote` はL0専用とする。D1管理されたdatasetでは旧epochの手動書換えに
ならないよう拒否し、operationへ案内する。旧binary/旧CLIもwriterとして動けない配置制約を
設ける。新設定ファイルの存在を古いbinaryが無視する可能性は、file markerだけでは解決しない。
online CLI、Web、policy executorは同じplan/operation serviceを使う。
dry-run/planはfence・permission・同期実体を変更しない。operatorがCLIから秘密付きbackupを
扱えることと、campus_adminがWebから全DBをダウンロードできることを同一にしない。
UIは最初に観測だけを実装し、operationが完成した段階で切替ボタンを追加する。
コンパクトなgroup一覧、control/dataの別ラベル、preempt状態、同期点、ブロック理由を優先する。
確認dialogには「どのCampus/Domain/同居groupが止まるか」「失う可能性のある期間」
「隔離手段」「未達条件」を列挙する。UNIX時刻入力や、色だけの安全判定は使わない。
## 5. 受入試験マトリクス
テストはfake clock/fake authority/fake fence/一時保存領域から開始し、
実カレンダー・個人文書・本番ノードには接続しない。各テストは最終状態だけでなく、
途中のwriter許可集合を記録し、同一resourceで2 holderになっていないことを検査する。
| ID | 入力・障害注入 | 期待結果 |
| --- | --- | --- |
| T01 | primary記録を持つnodeが再起動、peer応答なし | recovering→blocked、migration/worker含めwrite 0 |
| T02 | 非DR instance起動 | 従来の起動を維持。DR期待環境で設定欠落なら起動拒否 |
| T03 | 別group、未知鍵、同FQDNで別fingerprint、失効nodeからprobe | 認証/認可拒否、WARNING。membership変化なし |
| T04 | 古いboot/nonce、応答遅延、重複JSON key、未知protocol、大きすぎるbody | 状態遷移なし。入力種別に応じたerror |
| T05 | peerが大きいepoch/primaryを自己申告、authorityは異なる | 再確認・conflict、安全側停止。自己申告で昇格なし |
| T06 | 発見peerの半数以上が同じ内容、登録authorityに到達不能 | 観測の多数決でgrantを出さない |
| T07 | 同一DBのCG/DGが別holderを要求、alias resourceも混在 | WRITER_BOUNDARY_COUPLED、連動planを提示 |
| T08 | 独立storeのCG停止、DGは有効grantと設定を保持 | CGだけ停止。DGは権限鮮度条件の範囲で継続 |
| T09 | CGのpreempt on、DGのpreempt off | 独立profileならCGのみ戻る。sharedなら連動影響を承認前に提示 |
| T10 | primaryの停止・再開、時計逆行、VM pause、permit応答喪失 | stale permitでwrite再開なし。新grantにはfenceを要求 |
| T11 | 2 executor/2 candidateが同時に同group操作、response消失、manual agent二重起動 | 一つのoperation/grantだけ確定。同じIDで結果照合、manual agent排他 |
| T12 | fence要求送信済みだが確認不明、別node/古いbootの証跡 | FENCE_UNCONFIRMED、grantなし |
| T13 | ホストfenceが別DG primaryにも影響 | 自動preempt禁止、impactに全groupを列挙 |
| T14 | capture中にDB更新・calendar世代更新・GC・schema変更 | barrier/capabilityに従い排他。保証不能ならcapture中止 |
| T15 | 転送中断/Range再開、chunk改変、symlink/path traversal、未登録file ID | 検証失敗、他データを読ませずactive root不変 |
| T16 | blobの一部欠落、calendar参照世代欠落、schema不適合、secret欠落 | verified/applicable ACKなし、standby公開なし |
| T17 | pin後にsource再起動、transfer期限切れ、GC/保持処理 | 必要世代を消さない。残存pinを照合して回収 |
| T18 | 同期中に新しいwrite、表示上lag=0だが最終cut未確定 | switchover不可。final checkpointの一致を必須とする |
| T19 | metadataのみ最新、ACL/session失効は古い | read/write readinessとも不成立 |
| T20 | 旧primaryに現系列にない更新 | 隔離保全、conflict。自動マージ/削除なし |
| T21 | manifest旧epoch、同sequence異hash、欠落親、receiptなし | 適用拒否。v3導入後のv1/v2 downgradeも拒否 |
| T22 | 内容適用と受理checkpointの間で失敗、publication receipt応答喪失 | 一体rollback/同内容照合。別payloadで再発行しない |
| T23 | controller復元点が管理ノード受理位置より古い | blocked、epochを増やして古い権限を再配信しない |
| T24 | 各operation phaseの前後でprocess kill | 文書化したphaseから再開。REVOKING以降は旧epoch復活なし |
| T25 | GRANTED後root切替失敗、RO filesystem、disk full、fsync失敗 | 新primaryを公開せず、次の復旧operationへ |
| T26 | routing更新失敗、DNS cacheに旧宛先、既存keep-alive | 新primaryは一つ。旧宛先はwrite拒否、routing修復 |
| T27 | 同期済みpreferred nodeが繰り返しflap、時間帯/DST境界 | 安定時間リセット、cooldown遵守、不要な再切替なし |
| T28 | snapshotの鮮度はRPO内だがclient成功応答の反映が不明 | 自動障害昇格不可。損失確認をoperatorへ要求 |
| T29 | grant lease失効/authority quorum喪失/watch欠落 | 新grantなし。local gate期限で停止、外部fence前は他holder不可 |
| T30 | AuthorityStoreを古いbackupへ復元、manual台帳の破損/消失 | 全writerを隔離、新trust世代の手動復旧。epochの黙認・自動初期化なし |
| T31 | high IO/CPU/foreground要求増、巨大snapshot | stream/帯域/hash上限を維持。制御通信が転送に埋もれない |
| T32 | campus_admin/domain_adminが他scope切替/backup秘密を要求 | 拒否。read UIも他Campus・secretを漏らさない |
| T33 | restart後workerが過去jobを拾う、外部通知結果が曖昧 | 許可と再送条件を確認。曖昧な副作用は自動で成功/再送しない |
| T34 | operatorがcancel/disable/旧binaryへのrollbackを要求 | 安全点だけ取消。active grantを消して旧系を復活させない |
T24はPLANNED〜COMPLETEDの全境界で行う。ネットワークを切るタイミングも
「要求送信前」「相手がcommit後、応答前」「応答後、ローカル永続化前」に分ける。
fixtureの全writer経路を一度ずつ実行し、HTTP guardだけで満たしたと判定しない。
## 6. 性能・運用の合格条件
固定のRTO保証値を現時点で作らず、次の測定結果を導入記録に残す。
- 通常foreground p50/p95/p99と、同時同期時の差。目安の許容悪化率は10%以内、超過なら転送予算を縮小。
- DB/可変正本のbarrier時間、checkpoint作成時間、初回/増分転送時間、検証時間、必要空き容量。
- suspect検知、fence確認、grant、起動、routing完了の各所要時間と合計RTO。
- 障害時点と最後のDR側検証cutの差、client成功応答境界の確認可否、未複製損失の有無。
- スナップショット10万/100万file相当のmanifest paging、RSS/heap、descriptor数の上限。
初版v1の10万file上限を、v2の性能試験なしに解除しない。
- 一日継続のprobe/転送/再起動試験で、timer・goroutine・scratch・pin・operationが無制限に増えない。
10%は候補性能予算であり、実機測定前に達成と表明しない。別件の保存層の性能評価と同じ
fixture条件を使用する。外部サービス/実データにアクセスする実機試験は別途承認を得る。
## 7. 導入時に入力・承認する事項
設計として未決のアルゴリズムではなく、設備・運用に依存する導入パラメータを列挙する。
| 入力 | 決定する主体 | 未入力時 |
| --- | --- | --- |
| node FQDN/管理origin、node identity、対象Campus/DG、resource対応 | 運用者+保存担当 | 登録・同期を開始しない |
| RPO/RTO、retention、容量/帯域予算 | 運用者 | 本番enable不可 |
| authority配置、CA、専用service key保管/更新、quorum障害領域 | 運用者 | manualのみ |
| fence機器/credential参照、再起動抑止、影響group | 運用者 | coordinated enable不可 |
| backup/secret暗号化と復号担当、別障害領域の保全先 | 運用者 | 秘密付き転送不可 |
| CG/DGごとのpreferred、preemptと許可時間帯 | 運用者+管理者 | preempt off |
| 保存profile能力と対応app/image/schema | 保存担当 | offline/shared-instanceのみ |
## 8. 本番導入手順
1. 運用者が対象と入力を確定し、保存担当がresource map・能力契約を承認する。
2. 本番とは通信しない隔離fixtureで対応段階のT01〜T34と負荷試験を実施する。
3. 既存環境の保全・image digest・schema互換性・復旧手順をレビューする。実データ保全は別途承認。
4. L1観測のみを配置し、比較表示を確認する。旧roleに基づくサービス挙動はこの段階では変えない。
5. 全writerと旧CLIを停止する保守枠を取り、L2への移行planを確認する。
6. 旧cluster/node/roleを新group/resourceに対応させ、fresh authorityをbootstrapする。
旧primaryの確認記録を付け、元のepochを新authorityの根拠にしない。
7. 新startup gate下で起動し、他nodeはlearnerとして参加。sourceと同期内容を検証する。
8. L3の手動switchoverと復帰を1往復実施し、RPO/RTO・監査・権限失効・queueを検証する。
9. 保存分離済みprofileならCG/DGの独立切替、共有profileなら連動拒否/連動切替を確認する。
10. 能力が揃ったgroupに限り、明示承認を得てL4を有効化する。L5は別の認定を要求する。
ローリング移行中、旧ノードをwriter候補に残さない。mixed protocolで書込み可能nodeを
増やす移行はしない。旧nodeからのprobeは対応版を表示し、新grant候補から除外する。
管理manifest v3は全対象が移行anchorをACKした後に切替える。
## 9. 撤退・失敗時
DRの「無効化」は設定ファイル削除ではない。新operation受付を止め、active operationを照合し、
一つのwriterを運用者が確定・隔離確認してから非DR運用への移行planを実行する。
authorityを停止しただけで旧nodeに書き込みを許可しない。
L1だけなら観測プロセスを停止して戻せる。L2以降はrole/許可/DB互換性の記録を保持する。
新epochでwriteが発生した後は、旧snapshotを起動することでrollbackしない。
旧binaryに戻す場合は、最新データとschema互換性を検証し、DRのtrust/gateを無視するbinaryが
同時に動かないことを外部で保証する。保証できなければ停止を継続する。
実施結果には、適用していない段階、満たしていない能力、最後に検証した世代、
次に必要な承認を必ず残す。設計完了・実装完了・復旧訓練完了・本番稼働を同じチェックにしない。
Loaded 100 of 1032 files, more files were not shown because too many files have changed in this diff. Show more