2 Commits
Author SHA1 Message Date
Anya Tokugawa 4049acf76e fix(tasks): recover deleted captures without reviving tombstones
Release container images / release (push) Successful in 19s
Build and publish container images / images (push) Successful in 12m0s
2026-11-08 19:26:50 +09:00
Anya Tokugawa 43a2d8c5a1 docs: record v0.14.0 production deployment [skip ci] 2026-11-08 18:21:06 +09:00
11 changed files with 639 additions and 34 deletions

No files matched your search

+19 -1
View File
@@ -34,7 +34,9 @@
一覧GETは `kind=task|project`、`view`、`project`、`q`、`limit`、`before` を受け付けます。既定60件、上限100件。作成はPOST、更新はPUT、`If-Match: "{version}"` 必須。ヘッダーなしは428、古い版は412。UUIDをクライアントで生成し、同内容の作成再送は同じ版を返します。異内容の再送は競合です。成功時は `item` と現在読める `page_path`、ETagを返します。一覧はメモ本文を省略するため、編集やクイック完了前に必ず詳細を取得します。
## タスクの削除(2026-11-08実装、未デプロイ)
## タスクの削除
2026-11-08、v0.14.0で本番反映済み。[配備記録](../../operations/releases/2026-11-08-task-deletion.md)。
UUID検索は追加せず、既存のタスク編集画面で「アーカイブ/復元」の隣に「削除」を置きます。未保存の新規タスク、readonly、プロジェクト編集には表示しません。件名と削除範囲を確認してから実行し、削除失敗時は編集内容を保持します。アーカイブ済みタスクも同じ操作で削除できます。
@@ -49,6 +51,22 @@ UUID検索は追加せず、既存のタスク編集画面で「アーカイブ
`internal/server/task_delete_test.go` で条件付き削除・権限・再送・DBロールバック・通知取消・作業履歴保全・DAV索引再構築/再送拒否を検証します。既存のChromiumテストに、削除ボタンの位置、確認キャンセル、版競合、成功後の一覧更新を追加しています。本番データの削除は実施しません。
### 削除済みメール取り込みの再作成
PagesWiki v0.14.1/pages-cli v0.4.2向け実装。サーバーとCLIの両方の更新が必要です。DBスキーマ変更や既存データの削除はありません。
`POST /_/api/v1/extension/organizer.tasks/capture` は、受付記録に対応するタスクの状態を本文ハッシュより先に確認します。削除済みなら同じ内容・異なる内容とも410 `deleted`。タスクと削除記録が両方欠落していれば409 `capture_missing` とし、自動修復・再作成はしません。存続中のタスクへの異内容再送は従来どおり412です。
削除済み要求の保留下書きは `pages task retry REQUEST_UUID --recreate-deleted` で明示的に再取り込みできます。CLIは新しい要求を送る前に保存し、以降の再送用IDを表示します。通信失敗後は表示された新IDで通常の `task retry NEW_REQUEST_UUID` を使います。メール本文・Wiki suffixは保持し、以前のタスクの状態・作業時間・子タスクを復元する操作ではありません。
- APIには `replaces_deleted` に元のrequest UUIDを渡します。新しい `request_id` はURL名前空間のUUIDv5で、名前は `pages:task-recreate:` と元のrequest UUIDを連結したもの。任意の後継IDを拒否し、同じ再取り込みの二重作成を防ぎます。
- 同じDomain・所有者の受付記録と削除記録を要求し、元タスクが存続中なら412、他人の受付記録や不存在の要求は404。writer transaction内でも検査します。受付記録のない直接作成タスクには使えません。
- 新しいタスク・受付記録・変更履歴・通知予約を原子的に保存します。旧受付記録、削除記録、DAV識別子の予約は残し、旧端末の再送による復活を防ぎます。
- Wiki保存先にページが残っていれば、そのUUIDを新しいタスクに関連付けます。本文・リビジョンは上書きしません。存在しなければ保留本文から作成します。read/create/writeの現在権限と入力上限を検査し、ファイルなどページ以外との衝突は409です。保存先は保留要求と現在の既定値から解決するため、移動済みのWikiや既定Path変更を追跡して復元する機能ではありません。
- 再作成済み要求の再送は同じ新タスクを返します。そのタスクも削除済みなら410で停止し、さらに復活させません。元のメールを再入力しただけでは削除済み状態を解除しません。
`internal/server/task_capture_recreate_test.go` でライフサイクル・並行再送・所有者境界・ロールバック・Wiki保持を検証し、CLIの単体テストとHTTP結合テストでも再送IDの保持を検証します。
## 保存と復旧の境界
`organizer_items` はDomain・所有者・UUID・種類・検索用メタデータ・現在版を保持します。本文を含む型付きJSONは既存Document StoreへSHA-256で保存し、同じJSONをDBにも保持します。`organizer_revisions` は版からDocumentへの参照、`organizer_changes` は後続処理用の小さな変更ジャーナルです。head更新・履歴・変更ジャーナルは同じSQLiteトランザクションです。
@@ -14,7 +14,7 @@ Geckoの追加整備やRSSの追加ショートカットなど、並行作業中
- CalDAV一覧/REPORTから削除対象を除外。削除済みのUUIDとcollection内のDAV UID/hrefを予約し、古い端末からの再作成を拒否。
- 従来のDAV DELETEをアーカイブへ変換する動作は維持。
詳細は[タスクの実装契約](../../design/ja/tasks-implementation.md#タスクの削除2026-11-08実装未デプロイ)を参照してください。
詳細は[タスクの実装契約](../../design/ja/tasks-implementation.md#タスクの削除)を参照してください。
## DBと切り戻し
@@ -27,7 +27,44 @@ Geckoの追加整備やRSSの追加ショートカットなど、並行作業中
## 検証と配備
配備前に、リリース対象だけを切り出したソースの全体テストとCIビルド、
本番DBの隔離コピーを使った追加スキーマの整合性検査を実施します。
停止時点のデータ・設定と旧イメージを保全し、配備後にhealth、認証境界、静的ファイルの配信を検証します。
本番のタスク削除や外部カレンダー本文を用いた検証は行いません。
対象commitは `38a4fcf5301e0c7c6cad12070e040dc62a03d41b`。
2026-11-08 18:20:25 JSTに配備後の検証を完了しました。
- [ビルドCI run 68](https://git.anya.jp/anya/pages/actions/runs/68)、[リリースCI run 69](https://git.anya.jp/anya/pages/actions/runs/69)はともにsuccess。
- [Gitea Release v0.14.0](https://git.anya.jp/anya/pages/releases/tag/v0.14.0)と注釈付きGitタグを公開。
- リリース対象だけの独立ソースで `go test -p 1 ./...`、server/storeのgo vet、配布ツール等15テスト、PC・モバイル幅のChromium回帰試験が成功。
- 両本番DBのオンラインバックアップを隔離し、`EnsureTasksSchema`を各2回実行。task head・版参照・DAV resource/collection識別子のdigestが移行前後で一致し、削除記録は0件、`integrity_check`はok。
- app・app2・adaptor-gateway・excalidraw-viewerを同じv0.14.0へ更新。他サービス・Gecko設定・利用者設定は変更なし。
本番のタスク削除や外部カレンダー本文を用いた検証は行っていません。
### 復旧点
`.deployment-backups/release-v0.14.0-20261108/` の `config/` に旧構成、
`cutover/data`・`cutover/data2`に停止時点のデータを保存しています。
18:18:53 JSTに停止を開始、18:19:57 JSTにchecksum同期・再走査の差分ゼロ・DB一致を確認しました。
稼働元とinodeを共有しません。旧バックアップは削除していません。
旧3イメージは `rollback-before-v0.14.0-20261108` タグで保持しています。
`.env.release` は `PAGESWIKI_IMAGE_TAG=v0.14.0`。
新版の4サービスは18:20:12〜18:20:18 JSTに起動しました。
### 配備後検証
- 4サービスのrevisionが対象commitと一致、running、再起動回数0。
- app・app2のhealthと管理スキーマ検査が正常。legacy task領域は上記の隔離コピー移行試験で別途確認。
- 両DBの `quick_check` がok、削除記録0件。
- 外部HTTPSのhealth・ログイン画面200。匿名のタスク/ダッシュボード/RSS画面は303、タスクAPIは401。
- 実在しないUUIDへの匿名DELETEを401で拒否。
- tasks.js、tasks.css、organizer-dashboard.js、feeds.js、feed-gecko.jsの配信SHA-256がリリースcommitと一致。未完了Gecko変更の混入なし。
- 起動直後のfatal/panic、DB lock、event job failed、schema/migration errorは両アプリとも各0件。
### 配布イメージ
SHAタグから再ビルドせず昇格し、本番ホストで取得したdigestとCIの昇格結果が一致しています。
| イメージ(`git.anya.jp/anya/`、タグ`v0.14.0`) | manifest digest |
| --- | --- |
| pageswiki-app | `sha256:46edb2640bfda3be73d8bed354bb1e014b050d68b6c17af8036d65d7732378af` |
| pageswiki-adaptor-gateway | `sha256:696f2d0bcdf1a3dce8821c061a78ad9f3d6fcaadda32cc1cb4e5d5a117028fc0` |
| pageswiki-excalidraw-viewer | `sha256:4d72abb4c1964553458af2b95e19722e18ae05fcfa2b02ff086407ad962ca62c` |
@@ -0,0 +1,226 @@
package server
import (
"context"
"encoding/json"
"fmt"
"strings"
"sync"
"testing"
"github.com/google/uuid"
"pageswiki/internal/store"
)
func TestTaskCaptureDeletedRecreation(t *testing.T) {
for _, wiki := range []bool{false, true} {
t.Run(fmt.Sprint(wiki), func(t *testing.T) {
s, token, domain, actor := taskFixture(t)
ctx := context.Background()
id := uuid.NewString()
input := map[string]any{"request_id": id, "title": "Original"}
if wiki {
input["wiki_body"], input["wiki_prefix"] = "Original private body", "/mail"
}
w := taskRequest(s, token, "POST", "capture", input, "")
if w.Code != 201 {
t.Fatal(w.Code, w.Body)
}
var result struct {
Item store.TaskRecord `json:"item"`
Path string `json:"page_path"`
}
if err := json.Unmarshal(w.Body.Bytes(), &result); err != nil {
t.Fatal(err)
}
page, path := result.Item.PageUUID, result.Path
var revision string
if wiki {
r, err := s.store.GetLatestRevision(page)
if err != nil {
t.Fatal(err)
}
revision = r.UUID
}
if w = taskRequest(s, token, "DELETE", "items/"+id, nil, `"1"`); w.Code != 204 {
t.Fatal(w.Code, w.Body)
}
if w = taskRequest(s, token, "GET", "items/"+id, nil, ""); w.Code != 404 {
t.Fatal(w.Code, w.Body)
}
for _, title := range []string{"Original", "Changed"} {
input["title"] = title
if w = taskRequest(s, token, "POST", "capture", input, ""); w.Code != 410 || !strings.Contains(w.Body.String(), `"deleted"`) {
t.Fatal(w.Code, w.Body)
}
}
input["replaces_deleted"], input["request_id"] = id, store.TaskRecreationID(id)
if wiki {
input["wiki_body"] = "Must not overwrite the retained page"
for _, invalidBody := range []string{"bad\x00body", strings.Repeat("x", 256<<10+1)} {
input["wiki_body"] = invalidBody
if w = taskRequest(s, token, "POST", "capture", input, ""); w.Code != 422 {
t.Fatal("reused page bypassed body validation", w.Code, w.Body)
}
}
input["wiki_body"] = "Must not overwrite the retained page"
}
// A lost successful response, including concurrent retries, creates one
// successor. It neither revives the original nor revises the Wiki.
var wg sync.WaitGroup
for i := 0; i < 4; i++ {
wg.Add(1)
go func() {
defer wg.Done()
v := taskRequest(s, token, "POST", "capture", input, "")
if v.Code != 201 {
t.Errorf("concurrent recreation: %d %s", v.Code, v.Body)
}
}()
}
wg.Wait()
w = taskRequest(s, token, "POST", "capture", input, "")
if w.Code != 201 {
t.Fatal(w.Code, w.Body)
}
if err := json.Unmarshal(w.Body.Bytes(), &result); err != nil {
t.Fatal(err)
}
if result.Item.UUID != store.TaskRecreationID(id) || result.Item.UUID == id || result.Item.Title != "Changed" || result.Item.Version != 1 || result.Item.PageUUID != page || result.Path != path {
t.Fatal(result)
}
if wiki {
r, err := s.store.GetLatestRevision(page)
if err != nil || r.UUID != revision || !strings.Contains(r.Body, "Original private body") {
t.Fatal("Wiki changed", r, err)
}
}
for table, expected := range map[string]int{"organizer_items": 1, "organizer_capture_receipts": 2, "organizer_task_deletions": 1, "organizer_revisions": 1} {
var count int
if err := s.store.DB.QueryRow("SELECT count(*) FROM " + table).Scan(&count); err != nil || count != expected {
t.Fatal(table, count, err)
}
}
if _, err := s.store.SaveTask(ctx, domain, actor, store.TaskRecord{UUID: id, Kind: "task", Title: "Offline resurrection", Status: "todo", Bucket: "inbox", Priority: 1}, 0); err != store.ErrTaskDeleted {
t.Fatal("old UUID revived", err)
}
input["title"] = "Different committed recovery"
if w = taskRequest(s, token, "POST", "capture", input, ""); w.Code != 412 {
t.Fatal(w.Code, w.Body)
}
// Deleting the successor cannot be undone by replaying the recovery.
if w = taskRequest(s, token, "DELETE", "items/"+result.Item.UUID, nil, `"1"`); w.Code != 204 {
t.Fatal(w.Code, w.Body)
}
if w = taskRequest(s, token, "POST", "capture", input, ""); w.Code != 410 {
t.Fatal(w.Code, w.Body)
}
})
}
}
func TestTaskCaptureRecreationGuards(t *testing.T) {
s, token, domain, actor := taskFixture(t)
id := uuid.NewString()
input := map[string]any{"request_id": id, "title": "Original"}
if w := taskRequest(s, token, "POST", "capture", input, ""); w.Code != 201 {
t.Fatal(w.Code, w.Body)
}
recreate := map[string]any{"request_id": store.TaskRecreationID(id), "replaces_deleted": id, "title": "New"}
if w := taskRequest(s, token, "POST", "capture", recreate, ""); w.Code != 412 {
t.Fatal("live capture replaced", w.Code, w.Body)
}
if w := taskRequest(s, token, "DELETE", "items/"+id, nil, `"1"`); w.Code != 204 {
t.Fatal(w.Code, w.Body)
}
other, err := s.store.CreatePasswordAccount(domain, "recreateother", "long fixture password", "Other", "")
if err != nil {
t.Fatal(err)
}
otherToken, err := s.store.CreateAuthSession(domain, other.UUID)
if err != nil {
t.Fatal(err)
}
if w := taskRequest(s, otherToken, "POST", "capture", recreate, ""); w.Code != 404 {
t.Fatal("other owner", w.Code, w.Body)
}
if _, err := s.store.RecreateCapturedTask(context.Background(), "other-domain", actor, store.TaskRecreationID(id), "hash", newTaskRecord("task", "Wrong domain"), "", "", id); err != store.ErrAccessDenied {
t.Fatal("wrong domain", err)
}
if err := s.store.SetPrincipalState(domain, actor, "readonly"); err != nil {
t.Fatal(err)
}
if _, err := s.store.RecreateCapturedTask(context.Background(), domain, actor, store.TaskRecreationID(id), "hash", newTaskRecord("task", "Readonly"), "", "", id); err != store.ErrAccessDenied {
t.Fatal("readonly", err)
}
if err := s.store.SetPrincipalState(domain, actor, "active"); err != nil {
t.Fatal(err)
}
token, err = s.store.CreateAuthSession(domain, actor)
if err != nil {
t.Fatal(err)
}
if err := s.store.SetConfigInDomain(domain, "system_extension_states", `{"organizer.tasks":false}`); err != nil {
t.Fatal(err)
}
if _, err := s.store.RecreateCapturedTask(context.Background(), domain, actor, store.TaskRecreationID(id), "hash", newTaskRecord("task", "Disabled"), "", "", id); err != store.ErrAccessDenied {
t.Fatal("extension disabled", err)
}
if err := s.store.SetConfigInDomain(domain, "system_extension_states", `{"organizer.tasks":true}`); err != nil {
t.Fatal(err)
}
for _, invalid := range []string{uuid.NewString(), id, "not-an-id"} {
recreate["request_id"] = invalid
if w := taskRequest(s, token, "POST", "capture", recreate, ""); w.Code != 422 {
t.Fatal("arbitrary recovery ID", w.Code, w.Body)
}
}
recreate["request_id"] = store.TaskRecreationID(id)
// Failure during receipt insertion rolls back the task, page and journals.
if _, err := s.store.DB.Exec(`CREATE TRIGGER reject_recreation BEFORE INSERT ON organizer_capture_receipts BEGIN SELECT RAISE(ABORT,'fixture rejection'); END`); err != nil {
t.Fatal(err)
}
recreate["wiki_body"], recreate["wiki_prefix"] = "New body", "/recreated"
if w := taskRequest(s, token, "POST", "capture", recreate, ""); w.Code != 500 {
t.Fatal(w.Code, w.Body)
}
var count int
if err := s.store.DB.QueryRow(`SELECT count(*) FROM organizer_items`).Scan(&count); err != nil || count != 0 {
t.Fatal("orphan task", count, err)
}
if err := s.store.DB.QueryRow(`SELECT count(*) FROM paths WHERE path LIKE '/recreated/%'`).Scan(&count); err != nil || count != 0 {
t.Fatal("orphan page", count, err)
}
if _, err := s.store.DB.Exec(`DROP TRIGGER reject_recreation`); err != nil {
t.Fatal(err)
}
if w := taskRequest(s, token, "POST", "capture", recreate, ""); w.Code != 201 {
t.Fatal(w.Code, w.Body)
}
if _, err := s.store.GetTask(context.Background(), domain, actor, store.TaskRecreationID(id)); err != nil {
t.Fatal(err)
}
}
func TestTaskCaptureOrphanReceipt(t *testing.T) {
s, token, domain, actor := taskFixture(t)
id := uuid.NewString()
input := map[string]any{"request_id": id, "title": "Original"}
if w := taskRequest(s, token, "POST", "capture", input, ""); w.Code != 201 {
t.Fatal(w.Code, w.Body)
}
// Simulate preexisting corruption, not a supported user deletion operation.
if _, err := s.store.DB.Exec(`DELETE FROM organizer_items WHERE domain_uuid=? AND owner_uuid=? AND uuid=?`, domain, actor, id); err != nil {
t.Fatal(err)
}
for _, title := range []string{"Original", "Changed"} {
input["title"] = title
if w := taskRequest(s, token, "POST", "capture", input, ""); w.Code != 409 || !strings.Contains(w.Body.String(), "capture_missing") {
t.Fatal(w.Code, w.Body)
}
}
input["request_id"], input["replaces_deleted"] = store.TaskRecreationID(id), id
if w := taskRequest(s, token, "POST", "capture", input, ""); w.Code != 409 || !strings.Contains(w.Body.String(), "capture_missing") {
t.Fatal(w.Code, w.Body)
}
}
+32 -16
View File
@@ -56,20 +56,21 @@ func applyTaskReminderDefaults(item *store.TaskRecord, p store.TaskPreferences)
}
type taskCaptureInput struct {
RequestID string `json:"request_id"`
Title string `json:"title"`
Notes string `json:"notes"`
ProjectUUID *string `json:"project_uuid"`
AssigneeUUID *string `json:"assignee_uuid"`
DueOn *string `json:"due_on"`
StartOn *string `json:"start_on"`
PlannedOn *string `json:"planned_on"`
ReviewOn *string `json:"review_on"`
Estimate *int `json:"estimate_minutes"`
Complexity *int `json:"complexity"`
WikiBody *string `json:"wiki_body,omitempty"`
WikiPrefix *string `json:"wiki_prefix,omitempty"`
WikiSuffix *string `json:"wiki_suffix,omitempty"`
ReplacesDeleted string `json:"replaces_deleted,omitempty"`
RequestID string `json:"request_id"`
Title string `json:"title"`
Notes string `json:"notes"`
ProjectUUID *string `json:"project_uuid"`
AssigneeUUID *string `json:"assignee_uuid"`
DueOn *string `json:"due_on"`
StartOn *string `json:"start_on"`
PlannedOn *string `json:"planned_on"`
ReviewOn *string `json:"review_on"`
Estimate *int `json:"estimate_minutes"`
Complexity *int `json:"complexity"`
WikiBody *string `json:"wiki_body,omitempty"`
WikiPrefix *string `json:"wiki_prefix,omitempty"`
WikiSuffix *string `json:"wiki_suffix,omitempty"`
}
func (s *Server) handleTaskProductivity(w http.ResponseWriter, r *http.Request, actor string, active bool, kind string) {
@@ -197,6 +198,13 @@ func (s *Server) handleTaskProductivity(w http.ResponseWriter, r *http.Request,
taskError(w, store.ErrTaskInvalid)
return
}
if input.ReplacesDeleted != "" {
id, err := uuid.Parse(input.ReplacesDeleted)
if err != nil || id.String() != input.ReplacesDeleted || input.RequestID != store.TaskRecreationID(input.ReplacesDeleted) {
taskError(w, store.ErrTaskInvalid)
return
}
}
raw, _ := json.Marshal(input)
hash := sha256.Sum256(raw)
requestHash := hex.EncodeToString(hash[:])
@@ -261,7 +269,11 @@ func (s *Server) handleTaskProductivity(w http.ResponseWriter, r *http.Request,
taskError(w, store.ErrTaskInvalid)
return
}
target = store.TaskCaptureWikiPath(prefix, item.Title, input.RequestID)
pathID := input.RequestID
if input.ReplacesDeleted != "" {
pathID = input.ReplacesDeleted
}
target = store.TaskCaptureWikiPath(prefix, item.Title, pathID)
if input.WikiSuffix != nil {
if !store.ValidTaskMailSuffix(*input.WikiSuffix) {
taskError(w, store.ErrTaskInvalid)
@@ -280,7 +292,11 @@ func (s *Server) handleTaskProductivity(w http.ResponseWriter, r *http.Request,
taskError(w, store.ErrTaskInvalid)
return
}
item, err = s.store.CaptureTaskWithWiki(r.Context(), domain, actor, input.RequestID, requestHash, item, target, body)
if input.ReplacesDeleted != "" {
item, err = s.store.RecreateCapturedTask(r.Context(), domain, actor, input.RequestID, requestHash, item, target, body, input.ReplacesDeleted)
} else {
item, err = s.store.CaptureTaskWithWiki(r.Context(), domain, actor, input.RequestID, requestHash, item, target, body)
}
if err != nil {
taskError(w, err)
return
+2
View File
@@ -62,6 +62,8 @@ func taskError(w http.ResponseWriter, err error) {
status, code = 422, "invalid"
case errors.Is(err, store.ErrTaskConflict):
status, code = 412, "conflict"
case errors.Is(err, store.ErrTaskCaptureMissing):
status, code = 409, "capture_missing"
case errors.Is(err, store.ErrTaskDeleted):
status, code = 410, "deleted"
case errors.Is(err, store.ErrTaskHasChildren):
+54
View File
@@ -0,0 +1,54 @@
package store
import (
"context"
"database/sql"
"errors"
"github.com/google/uuid"
)
// A receipt without a live task or a deletion marker needs repair, not an
// implicit create that could silently revive an inconsistent record.
var ErrTaskCaptureMissing = errors.New("captured task is missing without a deletion record")
func TaskRecreationID(request string) string {
return uuid.NewSHA1(uuid.NameSpaceURL, []byte("pages:task-recreate:"+request)).String()
}
func deletedCapture(ctx context.Context, q interface {
QueryRowContext(context.Context, string, ...any) *sql.Row
}, domain, actor, request string) error {
var id string
if err := q.QueryRowContext(ctx, `SELECT item_uuid FROM organizer_capture_receipts WHERE domain_uuid=? AND owner_uuid=? AND request_uuid=?`, domain, actor, request).Scan(&id); err != nil {
return err
}
var live, deleted bool
if err := q.QueryRowContext(ctx, `SELECT EXISTS(SELECT 1 FROM organizer_items WHERE domain_uuid=? AND uuid=?), EXISTS(SELECT 1 FROM organizer_task_deletions WHERE domain_uuid=? AND owner_uuid=? AND item_uuid=?)`, domain, id, domain, actor, id).Scan(&live, &deleted); err != nil {
return err
}
if live {
return ErrTaskConflict
}
if !deleted {
return ErrTaskCaptureMissing
}
return nil
}
// RecreateCapturedTask never removes old receipts, tombstones or DAV identity
// reservations. A deterministic successor ID makes a lost response retry safe.
// The caller authorizes read/create/write for target, including retained pages.
func (s *Store) RecreateCapturedTask(ctx context.Context, domain, actor, request, hash string, item TaskRecord, target, body, replaces string) (TaskRecord, error) {
id, err := uuid.Parse(replaces)
if err != nil || id.String() != replaces || request != TaskRecreationID(replaces) {
return TaskRecord{}, ErrTaskInvalid
}
if err = taskActor(ctx, s.DB, domain, actor, true); err != nil {
return TaskRecord{}, err
}
if err = deletedCapture(ctx, s.DB, domain, actor, replaces); err != nil {
return TaskRecord{}, err
}
return s.captureTaskWithWiki(ctx, domain, actor, request, hash, item, target, body, replaces)
}
+5 -1
View File
@@ -77,8 +77,12 @@ func ValidTaskMailSuffix(s string) bool {
return mailListSuffix.MatchString(s) || (mailMonthSuffix.MatchString(s) && !strings.HasPrefix(s, "0000/"))
}
func validCapturedTaskWiki(target, body string) bool {
return target != "" && ValidTaskWikiPrefix(path.Dir(target)) && path.Clean(target) == target && len(body) <= 256<<10 && utf8.ValidString(body) && strings.IndexByte(body, 0) < 0
}
func (s *Store) createCapturedTaskWiki(ctx context.Context, tx *sql.Tx, domain, actor string, item *TaskRecord, target, body string) error {
if target == "" || !ValidTaskWikiPrefix(path.Dir(target)) || path.Clean(target) != target || item.PageUUID != "" || len(body) > 256<<10 || !utf8.ValidString(body) || strings.IndexByte(body, 0) >= 0 {
if !validCapturedTaskWiki(target, body) || item.PageUUID != "" {
return ErrTaskInvalid
}
var exists bool
+39 -3
View File
@@ -227,10 +227,22 @@ func (s *Store) CapturedTask(ctx context.Context, domain, actor, request, hash s
if err != nil {
return TaskRecord{}, err
}
// Check lifecycle before payload equality. A retained receipt must not
// disguise a deleted task as an editable version conflict.
item, err := s.GetTask(ctx, domain, actor, id)
if errors.Is(err, sql.ErrNoRows) {
if err = deletedCapture(ctx, s.DB, domain, actor, request); err != nil {
return TaskRecord{}, err
}
return TaskRecord{}, ErrTaskDeleted
}
if err != nil {
return TaskRecord{}, err
}
if stored != hash {
return TaskRecord{}, ErrTaskConflict
}
return s.GetTask(ctx, domain, actor, id)
return item, nil
}
func (s *Store) CaptureTask(ctx context.Context, domain, actor, request, hash string, item TaskRecord) (TaskRecord, error) {
@@ -240,6 +252,10 @@ func (s *Store) CaptureTask(ctx context.Context, domain, actor, request, hash st
// CaptureTaskWithWiki commits the task, receipt and optional Wiki page together.
// The HTTP caller must authorize read/create/write for the exact target path.
func (s *Store) CaptureTaskWithWiki(ctx context.Context, domain, actor, request, hash string, item TaskRecord, target, body string) (TaskRecord, error) {
return s.captureTaskWithWiki(ctx, domain, actor, request, hash, item, target, body, "")
}
func (s *Store) captureTaskWithWiki(ctx context.Context, domain, actor, request, hash string, item TaskRecord, target, body, replaces string) (TaskRecord, error) {
if _, err := uuid.Parse(request); err != nil {
return TaskRecord{}, ErrTaskInvalid
}
@@ -252,11 +268,31 @@ func (s *Store) CaptureTaskWithWiki(ctx context.Context, domain, actor, request,
}
item.UUID = request
out, err := s.saveTask(ctx, domain, actor, item, 0, func(tx *sql.Tx, current *TaskRecord, _ TaskRecord) error {
if target != "" {
if e := s.createCapturedTaskWiki(ctx, tx, domain, actor, current, target, body); e != nil {
if replaces != "" {
if e := deletedCapture(ctx, tx, domain, actor, replaces); e != nil {
return e
}
}
if target != "" {
if !validCapturedTaskWiki(target, body) || current.PageUUID != "" {
return ErrTaskInvalid
}
// Explicit recreation may reuse a surviving Wiki page. Its content
// and revisions are independent of task deletion and never replaced.
reused := false
if replaces != "" {
e := tx.QueryRowContext(ctx, `SELECT p.object_uuid FROM paths p JOIN objects o ON o.uuid=p.object_uuid WHERE p.domain_uuid=? AND p.path=? AND o.kind='page'`, domain, target).Scan(&current.PageUUID)
if e != nil && !errors.Is(e, sql.ErrNoRows) {
return e
}
reused = e == nil
}
if !reused {
if e := s.createCapturedTaskWiki(ctx, tx, domain, actor, current, target, body); e != nil {
return e
}
}
}
_, e := tx.ExecContext(ctx, `INSERT INTO organizer_capture_receipts VALUES(?,?,?,?,?)`, domain, actor, request, hash, item.UUID)
return e
})
+26 -4
View File
@@ -32,7 +32,7 @@ import urllib.parse as url
import urllib.request as http
import webbrowser
VERSION = '0.4.0'
VERSION = '0.4.2'
class Failure(Exception):
def __init__(self, message, status=0, payload=None):
@@ -569,7 +569,7 @@ def tasks(client, profile, words):
for name in ('done','reopen','archive','start'):
p=sub.add_parser(name);p.add_argument('uuid',type=task_uuid)
sub.add_parser('stop');sub.add_parser('status')
p=sub.add_parser('retry');p.add_argument('request_id',type=task_uuid)
p=sub.add_parser('retry');p.add_argument('request_id',type=task_uuid);p.add_argument('--recreate-deleted',action='store_true',help='Explicitly recreate a deleted capture with a new UUID; preserve and relink any existing Wiki page')
p=sub.add_parser('report');p.add_argument('--from',dest='start');p.add_argument('--to',dest='end')
p=sub.add_parser('preferences');p.add_argument('--wiki-prefix');p.add_argument('--project');p.add_argument('--assignee');p.add_argument('--due-days',type=int);p.add_argument('--start-days',type=int);p.add_argument('--planned-days',type=int);p.add_argument('--review-days',type=int);p.add_argument('--estimate',type=int);p.add_argument('--complexity',type=int);p.add_argument('--digest',choices=['on','off']);p.add_argument('--at');p.add_argument('--timezone')
for child in sub.choices.values():child.add_argument('--json',action='store_true')
@@ -578,10 +578,19 @@ def tasks(client, profile, words):
def capture(payload):
pending=profile.data.setdefault('task_outbox',{})
if len(pending)>=20 and payload['request_id'] not in pending:raise Failure('20 pending captures retained. Retry them before adding more.')
if payload['request_id'] in pending and pending[payload['request_id']] != payload:raise Failure('A different draft with this request ID is retained. Use pages task retry '+payload['request_id'])
replaced=payload['request_id'] in pending and pending[payload['request_id']] != payload
pending[payload['request_id']]=payload;profile.save()
if replaced:print('Replaced the retained draft with the current input.',file=sys.stderr)
try: result=client.json(TASK_API+'capture',method='POST',data=payload)
except Failure as e:
if e.status==410 and e.payload.get('error')=='deleted':
raise Failure('Draft retained. This task was deleted; normal retry will not restore it. To explicitly create a new task (preserving any existing Wiki body), use: pages task retry '+payload['request_id']+' --recreate-deleted',e.status,e.payload) from e
if e.status==409 and e.payload.get('error')=='capture_missing':
raise Failure('Draft retained. The server has a capture receipt but no task or deletion record. Server-side consistency repair is required; retry will not fix it.',e.status,e.payload) from e
if e.status==412:
raise Failure('Draft retained. This request ID conflicts with a server capture or task. An unchanged retry will not resolve it; inspect with: pages task show '+payload['request_id'],e.status,e.payload) from e
if e.status==409 and e.payload.get('error')=='note_exists':
raise Failure('Draft retained. A Wiki page already occupies the destination; it was not overwritten.',e.status,e.payload) from e
print('Draft retained; retry with: pages task retry '+payload['request_id'],file=sys.stderr)
if e.payload.get('error') == 'wiki_prefix_required': raise Failure('Set a private Wiki destination first: pages task preferences --wiki-prefix /YOUR/MAIL/PATH',e.status) from e
raise
@@ -606,6 +615,19 @@ def tasks(client, profile, words):
if action=='retry':
payload=profile.data.get('task_outbox',{}).get(a.request_id)
if not payload:raise Failure('No retained task capture with this request ID.')
if a.recreate_deleted:
pending=profile.data['task_outbox']
replacement=str(uuid.uuid5(uuid.NAMESPACE_URL,'pages:task-recreate:'+a.request_id))
payload=dict(payload,request_id=replacement,replaces_deleted=a.request_id)
if replacement in pending and pending[replacement]!=payload:
raise Failure('A different recreation draft already exists; inspect or retry it first: '+replacement)
previous=pending.copy()
pending[replacement]=payload;del pending[a.request_id]
try:profile.save()
except Exception:
profile.data['task_outbox']=previous
raise
print('Recreation draft saved. Existing Wiki content will be preserved. Retry ID: '+replacement,file=sys.stderr)
emit(capture(payload));return
if action in ('add','from-mail'):
payload={'request_id':getattr(a,'request_id',None) or str(uuid.uuid4()),'title':a.title,'notes':a.notes or ''}
@@ -622,7 +644,7 @@ def tasks(client, profile, words):
value=getattr(a,attr)
if value is not None:payload[key]=task_project(client,value) if attr=='project' else value
# A retry of the same pending mail must not drift across a month boundary
# when Date was missing. Other changed options still fail closed.
# when Date was missing. Changed input replaces the pending draft below.
old=profile.data.get('task_outbox',{}).get(payload['request_id'])
if action=='from-mail' and old and {k:v for k,v in old.items() if k!='wiki_suffix'}=={k:v for k,v in payload.items() if k!='wiki_suffix'}:
payload=old.copy()
+148 -4
View File
@@ -9,6 +9,74 @@ import unittest
m=runpy.run_path(str(Path(__file__).with_name('pages')))
class ProductivityTests(unittest.TestCase):
def test_deleted_capture_message_retains_draft_without_retry_loop(self):
ident='b5550cab-22d4-54eb-b06d-3b432d3f5101'
original={'request_id':ident,'title':'Private title','notes':'Private notes'}
for status,code,message in [(410,'deleted','--recreate-deleted'),(409,'capture_missing','consistency repair'),(412,'conflict','unchanged retry'),(409,'note_exists','not overwritten')]:
profile=SimpleNamespace(data={'task_outbox':{ident:original.copy()}},save=lambda:None)
calls=[]
def send(*args,**kw):
calls.append(kw)
raise m['Failure']('old generic message',status,{'error':code})
output=io.StringIO()
with contextlib.redirect_stderr(output),self.assertRaises(m['Failure']) as error:
m['tasks'](SimpleNamespace(json=send),profile,['retry',ident])
self.assertIn(message,str(error.exception))
self.assertNotIn('Private',str(error.exception)+output.getvalue())
self.assertNotIn('retry with:',output.getvalue())
self.assertEqual(profile.data['task_outbox'][ident],original)
self.assertEqual(len(calls),1)
def test_recreate_deleted_persists_successor_and_retries_same_id(self):
import uuid
ident='b5550cab-22d4-54eb-b06d-3b432d3f5101'
replacement=str(uuid.uuid5(uuid.NAMESPACE_URL,'pages:task-recreate:'+ident))
payload={'request_id':ident,'title':'Private title','wiki_body':'Private body','wiki_suffix':'2030/01/'+ident}
saved,calls=[],[]
profile=SimpleNamespace(data={'task_outbox':{ident:payload.copy()}})
profile.save=lambda:saved.append(json.loads(json.dumps(profile.data)))
def send(route,**kw):
calls.append(kw['data'].copy())
self.assertEqual(saved[-1]['task_outbox'][replacement],kw['data'])
self.assertNotIn(ident,saved[-1]['task_outbox'])
if len(calls)==1:raise m['Failure']('Connection lost')
return {'item':{'uuid':replacement}}
output,error=io.StringIO(),io.StringIO()
with contextlib.redirect_stdout(output),contextlib.redirect_stderr(error):
with self.assertRaises(m['Failure']):m['tasks'](SimpleNamespace(json=send),profile,['retry',ident,'--recreate-deleted'])
self.assertEqual(len(profile.data['task_outbox']),1)
m['tasks'](SimpleNamespace(json=send),profile,['retry',replacement])
self.assertEqual(calls[0],calls[1])
self.assertEqual(calls[0]['request_id'],replacement)
self.assertEqual(calls[0]['replaces_deleted'],ident)
self.assertEqual(calls[0]['wiki_suffix'],payload['wiki_suffix'])
self.assertEqual(calls[0]['wiki_body'],payload['wiki_body'])
self.assertEqual(profile.data['task_outbox'],{})
self.assertIn(replacement,error.getvalue())
self.assertNotIn('Private',error.getvalue())
def test_recreate_deleted_save_failure_keeps_original_and_sends_nothing(self):
ident='b5550cab-22d4-54eb-b06d-3b432d3f5101'
payload={'request_id':ident,'title':'Draft'}
def fail():raise m['Failure']('disk full')
profile=SimpleNamespace(data={'task_outbox':{ident:payload.copy()}},save=fail)
calls=[]
with self.assertRaises(m['Failure']):m['tasks'](SimpleNamespace(json=lambda *a,**kw:calls.append(kw)),profile,['retry',ident,'--recreate-deleted'])
self.assertEqual(calls,[])
self.assertEqual(profile.data['task_outbox'],{ident:payload})
def test_recreate_deleted_never_clobbers_different_successor_draft(self):
import uuid
ident='b5550cab-22d4-54eb-b06d-3b432d3f5101'
replacement=str(uuid.uuid5(uuid.NAMESPACE_URL,'pages:task-recreate:'+ident))
pending={ident:{'request_id':ident,'title':'Original'},replacement:{'request_id':replacement,'title':'Different'}}
before=json.loads(json.dumps(pending))
calls=[]
profile=SimpleNamespace(data={'task_outbox':pending},save=lambda:calls.append('save'))
with self.assertRaises(m['Failure']):m['tasks'](SimpleNamespace(json=lambda *a,**kw:calls.append('send')),profile,['retry',ident,'--recreate-deleted'])
self.assertEqual(calls,[])
self.assertEqual(pending,before)
def test_mail_path_priority_and_dates(self):
import datetime,hashlib
ident='06cb3a24-61f8-4a9f-a0bf-89eea9632f25'
@@ -24,7 +92,7 @@ class ProductivityTests(unittest.TestCase):
raw=b'Subject: [../unsafe 123] Title\r\n'+date+b'\r\nbody'
self.assertEqual(suffix(raw,ident,now),'2030/02/'+hashlib.sha256(raw).hexdigest())
def test_pending_mail_month_and_options_are_preserved(self):
def test_pending_mail_keeps_month_but_replaces_changed_options(self):
from unittest.mock import patch
raw=b'Subject: Test\r\nMessage-ID: <same@example.invalid>\r\n\r\nbody'
calls=[]
@@ -37,10 +105,86 @@ class ProductivityTests(unittest.TestCase):
with patch.dict(m['tasks'].__globals__,task_mail_suffix=lambda raw,ident:month+'/'+ident),patch('sys.stdin',SimpleNamespace(buffer=io.BytesIO(raw))),contextlib.redirect_stderr(io.StringIO()),self.assertRaises(m['Failure']):
m['tasks'](client,profile,['from-mail'])
self.assertEqual(calls[0],calls[1])
with patch('sys.stdin',SimpleNamespace(buffer=io.BytesIO(raw))),self.assertRaises(m['Failure']):
with patch('sys.stdin',SimpleNamespace(buffer=io.BytesIO(raw))),contextlib.redirect_stderr(io.StringIO()),self.assertRaises(m['Failure']):
m['tasks'](client,profile,['from-mail','--wiki-prefix','/changed'])
self.assertEqual(len(calls),2)
self.assertEqual(next(iter(profile.data['task_outbox'].values())),calls[0])
self.assertEqual(len(calls),3)
self.assertEqual(calls[2]['request_id'],calls[0]['request_id'])
self.assertEqual(calls[2]['wiki_prefix'],'/changed')
self.assertEqual(next(iter(profile.data['task_outbox'].values())),calls[2])
def test_changed_capture_is_saved_before_send_and_retained_on_conflict(self):
ident='b5550cab-22d4-54eb-b06d-3b432d3f5101'
saved,calls=[],[]
old={'request_id':ident,'title':'Old draft','notes':'Old private notes'}
profile=SimpleNamespace(data={'task_outbox':{ident:old}})
profile.save=lambda:saved.append(json.loads(json.dumps(profile.data)))
def send(route,**kw):
payload=kw['data'].copy();calls.append(payload)
self.assertEqual(saved[-1]['task_outbox'][ident],payload)
if len(calls)==1:raise m['Failure']('conflict',412,{'error':'conflict'})
return {'item':{'uuid':ident}}
client=SimpleNamespace(json=send)
out,err=io.StringIO(),io.StringIO()
with contextlib.redirect_stdout(out),contextlib.redirect_stderr(err):
with self.assertRaises(m['Failure']) as failure:
m['tasks'](client,profile,['add','New draft','--request-id',ident,'--notes','New private notes'])
self.assertEqual(failure.exception.status,412)
self.assertEqual(profile.data['task_outbox'][ident],calls[0])
self.assertEqual(len(calls),1) # No automatic server overwrite/retry.
m['tasks'](client,profile,['retry',ident])
self.assertEqual(calls[0],calls[1])
self.assertEqual(calls[0]['title'],'New draft')
self.assertEqual(profile.data['task_outbox'],{})
self.assertIn('Replaced the retained draft',err.getvalue())
self.assertNotIn('private notes',err.getvalue())
self.assertEqual(json.loads(out.getvalue())['item']['uuid'],ident)
def test_replacing_draft_does_not_send_when_local_save_fails(self):
ident='b5550cab-22d4-54eb-b06d-3b432d3f5101'
def fail_save():raise m['Failure']('local write failed')
profile=SimpleNamespace(data={'task_outbox':{ident:{'request_id':ident,'title':'Old','notes':''}}},save=fail_save)
calls=[]
client=SimpleNamespace(json=lambda *a,**kw:calls.append(kw))
with self.assertRaises(m['Failure']):m['tasks'](client,profile,['add','New','--request-id',ident])
self.assertEqual(calls,[])
def test_full_outbox_allows_replacement_but_not_new_id(self):
import uuid
pending={str(uuid.uuid4()):{'title':'Old'} for _ in range(20)}
ident=next(iter(pending))
profile=SimpleNamespace(data={'task_outbox':pending},save=lambda:None)
calls=[]
def send(route,**kw):
calls.append(kw['data'].copy())
raise m['Failure']('offline')
client=SimpleNamespace(json=send)
with contextlib.redirect_stderr(io.StringIO()),self.assertRaises(m['Failure']):
m['tasks'](client,profile,['add','Replacement','--request-id',ident])
self.assertEqual(len(calls),1)
self.assertEqual(pending[ident]['title'],'Replacement')
with self.assertRaises(m['Failure']):m['tasks'](client,profile,['add','Another'])
self.assertEqual(len(calls),1)
self.assertEqual(len(pending),20)
def test_reimported_mail_replaces_body_and_removes_obsolete_options(self):
from unittest.mock import patch
raw=b'Subject: Old\r\nMessage-ID: <same@example.invalid>\r\n\r\nOLD BODY'
calls=[]
def send(route,**kw):
calls.append(kw['data'].copy())
raise m['Failure']('offline')
client=SimpleNamespace(base='https://example.invalid',json=send)
profile=SimpleNamespace(data={'username':'test'},save=lambda:None)
changed=raw.replace(b'Subject: Old',b'Subject: New').replace(b'OLD BODY',b'NEW BODY')
for mail,args in [(raw,['--wiki-prefix','/old']),(changed,['--wiki-prefix','/new']),(changed,['--metadata-only'])]:
with patch('sys.stdin',SimpleNamespace(buffer=io.BytesIO(mail))),contextlib.redirect_stderr(io.StringIO()),self.assertRaises(m['Failure']):
m['tasks'](client,profile,['from-mail',*args])
self.assertEqual(len({c['request_id'] for c in calls}),1)
self.assertEqual(calls[1]['title'],'New')
self.assertIn('NEW BODY',calls[1]['wiki_body'])
self.assertNotIn('OLD BODY',calls[1]['wiki_body'])
for key in ('wiki_body','wiki_prefix','wiki_suffix'):self.assertNotIn(key,calls[2])
self.assertEqual(next(iter(profile.data['task_outbox'].values())),calls[2])
def test_mail_privacy_and_long_subject_override(self):
raw=b'Subject: Test\r\nFrom: Person <person@example.invalid>\r\nMessage-ID: <id@example.invalid>\r\nContent-Type: text/plain; charset=utf-8\r\n\r\nPRIVATE BODY'
+46
View File
@@ -5,6 +5,7 @@ import json
from pathlib import Path
import runpy
import sys
import uuid
from types import SimpleNamespace
from unittest.mock import patch
from email.message import EmailMessage
@@ -61,5 +62,50 @@ def run():
output=io.StringIO()
with contextlib.redirect_stdout(output):m['contacts'](client,['export',contact['uuid']])
assert 'UID:urn:test:cli' in output.getvalue()
# Failed captures can be replaced locally; committed receipts remain protected.
request_id=str(uuid.uuid4())
with contextlib.redirect_stderr(io.StringIO()):
try:task('add','Rejected draft','--request-id',request_id,'--estimate','-1')
except m['Failure'] as error:assert error.status==422
else:raise AssertionError('Invalid estimate accepted')
assert profile.data['task_outbox'][request_id]['estimate_minutes']==-1
corrected=task('add','Corrected draft','--request-id',request_id,'--estimate','15')['item']
assert corrected['uuid']==request_id and corrected['title']=='Corrected draft'
assert request_id not in profile.data['task_outbox']
try:task('add','Changed after commit','--request-id',request_id,'--estimate','30')
except m['Failure'] as error:assert error.status==412
else:raise AssertionError('Committed capture silently overwritten')
assert profile.data['task_outbox'][request_id]['title']=='Changed after commit'
stored=task('show',request_id)['item']
assert stored['title']=='Corrected draft' and stored['estimate_minutes']==15
# Deleted receipts must not be mistaken for editable conflicts. An
# explicit recreation preserves the old tombstone and changes identity.
client.request(m['TASK_API']+'items/'+request_id,method='DELETE',etag='"'+str(stored['version'])+'"')
with contextlib.redirect_stderr(io.StringIO()):
try:task('retry',request_id)
except m['Failure'] as error:
assert error.status==410 and '--recreate-deleted' in str(error)
else:raise AssertionError('Deleted task implicitly recreated')
recreated=task('retry',request_id,'--recreate-deleted')['item']
replacement=str(uuid.uuid5(uuid.NAMESPACE_URL,'pages:task-recreate:'+request_id))
assert recreated['uuid']==replacement and recreated['title']=='Changed after commit'
assert recreated['estimate_minutes']==30 and not profile.data.get('task_outbox')
try:task('show',request_id)
except m['Failure'] as error:assert error.status==404
else:raise AssertionError('Deleted UUID resurrected')
# Reimport a deleted mail with a changed body: reconnect the surviving
# Wiki note, without overwriting its independent content or revision.
mail_id=captured['item']['uuid']
client.request(m['TASK_API']+'items/'+mail_id,method='DELETE',etag='"1"')
changed_mail=mail.replace(b'Private mail body',b'Changed private body')
with patch('sys.stdin',SimpleNamespace(buffer=io.BytesIO(changed_mail))),contextlib.redirect_stderr(io.StringIO()):
try:task('from-mail')
except m['Failure'] as error:assert error.status==410
else:raise AssertionError('Deleted mail task recreated implicitly')
with contextlib.redirect_stderr(io.StringIO()):recreated_mail=task('retry',mail_id,'--recreate-deleted')
assert recreated_mail['item']['page_uuid']==captured['item']['page_uuid']
assert recreated_mail['page_path']==captured['page_path']
body,_=client.raw('path:'+recreated_mail['page_path'])
assert 'Private mail body' in body.decode() and 'Changed private body' not in body.decode()
if __name__=='__main__':run()