Capture accumulated application, UI, CLI, regression tests and design documentation changes, including task project mapping and default CalDAV list naming. Keep deployment data, backups, local credentials and build artifacts out of Git. Move the Compose enrollment token to an ignored local environment file.
98 lines
5.1 KiB
Markdown
98 lines
5.1 KiB
Markdown
# Object metadata list: bounded cursor pagination
|
|
|
|
Implemented: 2026-09-14. Endpoint: `GET` / `HEAD /_/api/v1/object/list`.
|
|
|
|
## Compatible response and continuation
|
|
|
|
The body remains a JSON array of the existing `Object` representation
|
|
(`UUID`, `Kind`, `CreatedAt`). No Path, revision body, aggregate count, or
|
|
unreadable Object metadata is added. Empty results serialize as `[]`.
|
|
|
|
Query parameters:
|
|
|
|
- `limit`: default 100, positive integers, capped at 200.
|
|
- `cursor`: opaque continuation token from the preceding response.
|
|
|
|
When continuation is possible, responses include both `X-Next-Cursor` and an
|
|
RFC-style `Link: <...>; rel="next"` header containing the same cursor. HEAD
|
|
returns the corresponding metadata/continuation headers without an entity
|
|
body. Responses are `private, no-store`; unsupported methods return 405.
|
|
|
|
Clients **must continue when a next cursor exists, even if the current array
|
|
is empty**. A page may examine only unreadable Objects and still have later
|
|
readable Objects. Absence of a next cursor is the end-of-scan condition. There
|
|
is deliberately no `total`, denied count, or public scanned-row count.
|
|
|
|
Ordering is `(COALESCE(created_at, 0) DESC, uuid ASC)`. The keyset position does
|
|
not require the boundary Object to continue existing, and handles identical
|
|
timestamps and legacy NULL timestamps without OFFSET or duplicates. New
|
|
Objects/links created between requests may appear before or after the current
|
|
position; pagination is not a multi-request immutable snapshot.
|
|
The continuation query separately seeks the same-timestamp UUID suffix and
|
|
then older timestamps, keeping large same-second imports from repeatedly
|
|
rescanning UUIDs before the cursor.
|
|
|
|
## Cursor confidentiality and authority
|
|
|
|
The cursor contains its purpose/version, Domain, principal, fifteen-minute
|
|
expiry, and `(created_at, UUID)` position. The existing instance authentication
|
|
key encrypts/authenticates it with AES-GCM; the transport encoding is base64url.
|
|
A base64-only bookmark would disclose the last denied Object's UUID and time
|
|
on an empty page, so it is explicitly not used.
|
|
|
|
Tampered, expired, wrong-purpose, cross-Domain, and cross-principal cursors
|
|
return a generic 400. Cursors do not carry permissions: current session,
|
|
membership and grants are reevaluated on every request. They survive a process
|
|
restart when the authentication key is unchanged. Replicas need the same key
|
|
to interchange cursors; key distribution itself is not implemented here.
|
|
|
|
## Bounded selection and authorization
|
|
|
|
The API retains its existing scope: only Objects with a **live semantic Path
|
|
binding in the current Domain** are candidates. Unbound Objects and Objects
|
|
bound exclusively in another Domain do not become listable merely because
|
|
their global UUID or creation origin is known. Direct Object grants authorize
|
|
metadata among these candidates; they do not widen this listing's scope.
|
|
|
|
The handler installs a two-second request context, loads exact Object / broad
|
|
Domain-or-Object read grants in one bounded query, and uses the shared
|
|
request-scoped Path-read filter. Principal and Domain membership must be active
|
|
(readonly Principals may read). Candidate metadata and binding inspection share
|
|
a read-only SQLite transaction.
|
|
|
|
- At most 400 candidates, plus one lookahead row, are selected in SQL.
|
|
- Direct Object and broad read grants are applied before looking at Paths.
|
|
- At most 8,000 binding rows are inspected in one batch for the remaining
|
|
candidates. No per-Object SQL authorization loop remains.
|
|
- Exact Object / Domain grant loading is capped at 4,096 grants.
|
|
- If a binding/grant budget is exceeded, the request fails closed with a
|
|
generic 503. It does not incorrectly skip insufficiently inspected Objects.
|
|
- A visible page ending early resumes after its last returned Object; a
|
|
denied-only candidate batch resumes after its last inspected Object.
|
|
|
|
Schema initialization adds metadata-order and Object/Domain/Path indexes once.
|
|
The helper tolerates legacy Domain-only migrations where Object tables have
|
|
not yet been installed. No foreground listing performs DDL.
|
|
|
|
## Remaining scaling work
|
|
|
|
Very many cross-Domain Objects can still require scanning the metadata-order
|
|
index before finding in-Domain candidates; the deadline bounds this rather than
|
|
a claim of constant query time. Very highly aliased Objects can exceed the
|
|
binding budget for a Path-only reader. Domain-specific listing projections and
|
|
SQL-side joins to a compiled, bounded ACL projection would address these cases.
|
|
The older trusted store listing methods remain available to their other callers;
|
|
this change specifically bounds the HTTP metadata-list endpoint.
|
|
|
|
## Regression tests
|
|
|
|
Tests cover default/max limits, stable tied/NULL timestamps, deleted boundary
|
|
Objects, HEAD behavior, malformed/tampered/expired/scoped cursors, empty pages
|
|
with continuation, exact Object and Path grants, grant revocation, hidden Path
|
|
and cursor confidentiality, Domain/unbound exclusion, cancellation, extreme
|
|
binding budgets, and the legacy Domain-only migration fixture.
|
|
|
|
```sh
|
|
go test -race ./internal/server ./internal/store -run 'TestObjectList|TestObjectMetadata|TestLegacyDistrictsMigrateToDomains' -count=1
|
|
```
|