Capture accumulated application, UI, CLI, regression tests and design documentation changes, including task project mapping and default CalDAV list naming. Keep deployment data, backups, local credentials and build artifacts out of Git. Move the Compose enrollment token to an ignored local environment file.
92 lines
4.0 KiB
Markdown
92 lines
4.0 KiB
Markdown
# Event Scheduler
|
|
|
|
## Purpose
|
|
|
|
CPU-, memory-, and decoder-heavy work must not execute in Viewer requests.
|
|
PagesWiki uses a persistent SQLite event queue for deferred work.
|
|
|
|
`event_jobs` records:
|
|
|
|
- event UUID and type
|
|
- idempotency key
|
|
- JSON payload
|
|
- `queued`, `running`, `succeeded`, or `failed` status
|
|
- scheduled time and attempt count
|
|
- latest error
|
|
- creation and update timestamps
|
|
|
|
The current implementation is a single scheduler loop that may run up to the
|
|
configured concurrency in goroutines. It checks foreground request activity
|
|
between claims and keeps one reserved background slot so a long-lived editor or
|
|
stream cannot starve all jobs. Cross-instance leases are still future work. The
|
|
transactional Domain outbox now records Page/Revision creation and Path/Object
|
|
mutations. Domain-local consumer watermarks are durable and monotonic;
|
|
notification delivery remains future work.
|
|
External preview tools run through bounded
|
|
commands with timeouts.
|
|
|
|
## Event log retention and trends
|
|
|
|
`event_logs` remains the recent, drill-down log. Each record is associated with
|
|
its event type when the job is known. `event_log_policies` provides per-event
|
|
retention, aggregation window, bucket size, and maximum raw-row settings; `*`
|
|
is the safe default. `event_log_buckets` stores compact counts and first/last
|
|
timestamps by event type, level, and time bucket, so trend charts do not scan
|
|
the raw log table. The API exposes `/event/logs/trends` and `/event/policies`.
|
|
|
|
Retention removes old rows in place and keeps bounded aggregate buckets. A
|
|
future partition/rotation implementation may replace the physical table, but
|
|
must use an atomic rename-and-recreate procedure; an online `DROP TABLE` is
|
|
never used while workers can append logs.
|
|
|
|
## Preview Events
|
|
|
|
`preview.generate` uses the immutable Document hash as its idempotency key.
|
|
Viewer never generates a thumbnail. A children projection:
|
|
|
|
1. checks the immutable preview cache;
|
|
2. emits the cached thumbnail URL when present;
|
|
3. otherwise enqueues the event and immediately renders a format icon.
|
|
|
|
The browser polls only the lightweight cache URL for visible entries. Once the
|
|
job succeeds, it replaces the icon with the cached image without reloading the
|
|
Page.
|
|
|
|
Current generators:
|
|
|
|
| Input | Generator | Cached form |
|
|
| --- | --- | --- |
|
|
| PNG/JPEG/GIF | bounded Go image resampler | JPEG |
|
|
| PDF | `pdftoppm`, first page | JPEG |
|
|
| Video | `ffmpeg`, one frame | JPEG |
|
|
| Excalidraw | bounded server SVG renderer | sandboxed SVG |
|
|
|
|
Individual PDF pages use the same queue. A cache miss returns `202` with
|
|
`Retry-After`; the browser retries until the scheduler writes the immutable
|
|
`<document-hash>-p<page>.jpg` file. PDF page counts and non-ZIP archive listings
|
|
are persisted output of `projection.metadata.generate`, not Viewer work. See
|
|
[deferred projection metadata](projection-metadata-cache.md) for cache quotas,
|
|
pending/retry responses, authorization, and decoder limits.
|
|
|
|
Generated files live beside the Document Store in `previews/` and are addressed
|
|
by immutable Document hash. Cache responses are immutable for one year.
|
|
|
|
## Scheduling
|
|
|
|
- Missing previews are enqueued when an icon projection encounters them.
|
|
- A periodic sweep runs every 15 minutes.
|
|
- The first sweep runs shortly after startup.
|
|
- Configuration → Execution monitor combines resource charts with running jobs
|
|
only. Job history & logs contains paginated completed jobs and per-job logs.
|
|
Event definitions exposes the manual “Generate missing previews” trigger.
|
|
|
|
See [Configuration panels](configuration-panels.md) for polling, indexed queue
|
|
listing, lazy log retrieval and legacy resource-panel URL compatibility.
|
|
|
|
Failures remain visible in scheduler statistics and logs. Retry/backoff and
|
|
cancellation are persisted in the queue; a later sweep or manual trigger may
|
|
enqueue a missing cache again. Preview requests never execute the generator in
|
|
the normal Viewer request path. Non-ZIP entry extraction remains a bounded
|
|
foreground stream pending its separate cached-entry design; metadata listing
|
|
and PDF page counts have migrated to deferred projections.
|