feat: add previewed atomic OPML subscription import and export
Build and publish container images / images (push) Failing after 7m31s
Build and publish container images / images (push) Failing after 7m31s
This commit is contained in:
1 parent
38daea737d
commit
7fe64113c7
13 files changed
+782
-8
No files matched your search
@@ -27,7 +27,7 @@
|
||||
| ADR-O1 | organizerはtyped metadataと任意Page参照 | タスク初版は専用表+既存CAS Document Store(2026-09-24)。通常のObject閲覧・リンク・削除の権限境界を拡大せず、個人所有を確実に分離する。DBは捨てられる索引ではない。[実装契約](tasks-implementation.md) |
|
||||
| ADR-O2 | task共通部+ticket workflow | 提案。番号・外部依頼者・遷移権限の要件確認 |
|
||||
| ADR-O3 | ICS/vCard入出力→read-only DAV→双方向 | 推奨順。対象サーバー/clientを選んで互換表を作る |
|
||||
| ADR-O4 | 簡易利用は内蔵正本、本格運用はFreshRSS正本+PagesWiki connector+newsboat | 2026-11-08利用者指定で採用。内蔵・FreshRSS読取・共通UIまで実装。source単位で正本を分離し、自動fallbackや二重管理はしない。OPML・外部書戻しは残件。実環境の接続先・版・資格情報は別途指定 |
|
||||
| ADR-O4 | 簡易利用は内蔵正本、本格運用はFreshRSS正本+PagesWiki connector+newsboat | 2026-11-08利用者指定で採用。内蔵・FreshRSS読取・共通UI・OPML入出力まで実装。source単位で正本を分離し、自動fallbackや二重管理はしない。外部書戻しは残件。実環境の接続先・版・資格情報は別途指定 |
|
||||
| ADR-O5 | IMAP read-only、Wikiコピーは明示操作 | 初期設計方針。providerとsampleが実装前提 |
|
||||
| ADR-O6 | HTMLパーツを先行し全面WYSIWYGは保留 | 提案。round-tripと操作改善の評価でgo/no-go |
|
||||
| ADR-O7 | Wiki Pathと外部UID/hrefを分離 | 共通の設計制約。rename fixtureを全connectorで実施 |
|
||||
|
||||
+32
-3
@@ -1,7 +1,7 @@
|
||||
# 15. RSSビューア・FreshRSS・newsboat
|
||||
|
||||
状態:内蔵簡易reader、FreshRSS読取connector、共通viewer、Wiki保存・タスク化、ダッシュボードを実装。
|
||||
FreshRSSへの書戻し・OPMLは残件。本番反映と実FreshRSSへの接続は未実施。
|
||||
OPMLの確認付き追加取り込み・書出しも実装済み。FreshRSSへの書戻しは残件。本番反映と実FreshRSSへの接続は未実施。
|
||||
既存のブログRSS「配信」と、本章の外部RSS「購読」は別機能。
|
||||
|
||||
2026-11-08:利用者指定のサブプロジェクト順で最初に実施する。内蔵readerは簡易利用の
|
||||
@@ -75,10 +75,14 @@ API prefixは `/_/api/v1/extension/organizer.feeds/`。Domain内の本人専用
|
||||
| `GET item?source=…&id=…` | 本人の記事本文 |
|
||||
| `POST state` | 内蔵read/starの条件付き変更。FreshRSSは拒否 |
|
||||
| `POST note` | 公開範囲確認付きのWiki単独保存。request_idと入力hashで再送を識別 |
|
||||
| `GET opml/export` | 本人の内蔵購読をOPML 2.0としてダウンロード |
|
||||
| `POST opml/preview` | ファイルを解析し、追加可能・登録済み・重複・未対応等を返す。保存・外部通信なし |
|
||||
| `POST opml/import` | 選択行を再解析・検査し、確認時の購読設定と一致する場合だけ原子的に追加 |
|
||||
|
||||
managed schema `feeds` v1に `feed_sources`、`feed_items`、`feed_states`、`feed_note_receipts`を追加し、
|
||||
v2で`feed_items.body_html`を追加します。既存のテキスト本文・既読・starは保持し、条件付きGETのvalidatorを
|
||||
一度クリアして次回背景取得でHTMLを取り直します。migration前のworkerは設定versionで無効化します。
|
||||
v3で`feed_import_receipts`を追加し、購読の一括追加と再送記録を同時保存します。
|
||||
既存テーブルを破壊・再構成しません。SQLiteのforeign_keys設定に依存せず削除を完遂するtriggerを
|
||||
持ちます。内蔵read/starを本文cacheから分離し、記事を保持窓から落としても状態は黙って消しません。
|
||||
DBは内蔵購読・状態・保存receiptの正本であり、全体を捨てられるcacheとは扱いません。
|
||||
@@ -123,7 +127,32 @@ FreshRSS APIに条件付き更新がない操作もあり得るため、対応
|
||||
指数backoff/Retry-Afterを追加します。RSS 2.0とAtomを対象とし、XML外部entityは展開しません。
|
||||
外部画像の同意付き表示は未実装で、現在は一律に取得・表示しません。
|
||||
|
||||
OPMLは購読の入出力として扱い、既読・star・全本文のバックアップと同一視しません。importはpreview・追加/置換選択・重複確認を行い、既存購読を黙って削除しません。
|
||||
### OPML入出力
|
||||
|
||||
RSSツールバーの`⇄`から、本文ペイン内でファイル選択・確認・追加を行います。書出しは本人の内蔵購読の
|
||||
名前・URL・フォルダをOPML 2.0に保存し、停止中の購読も含みます。FreshRSS接続・資格情報は含めません。
|
||||
購読URLのqueryに秘密がある場合はそのURLも書き出されるため、ファイルは本人用として扱います。
|
||||
既読・star・記事本文・有効状態・更新間隔のバックアップではありません。
|
||||
|
||||
取込はUTF-8のOPML 1.0/1.1/2.0を対象とします。OPMLは購読と階層outlineを交換する形式であり、
|
||||
この実装はfeed購読部分のみ扱います。[OPML公式仕様](https://2005.opml.org/spec2.html)。
|
||||
ネストしたフォルダは`親 / 子`という一つのフォルダ名へ投影します。名前のmarkupは文字情報にし、
|
||||
`include`・外部outline・HTMLページの参照は取得しません。コメント扱いのoutline配下も追加対象外です。
|
||||
|
||||
確認表では、追加・登録済み・ファイル内重複・未対応URL/形式・名前不正を区別し、追加可能な行を選択します。
|
||||
HTTPS/443と内部アドレス拒否の既存契約は維持し、ファイル取り込みを検証の迂回経路にはしません。
|
||||
初期版は**追加のみ**で、置換/既存購読の削除は提供しません。有効状態と更新間隔は追加時に指定します。
|
||||
保存後のfeed取得は既存の背景eventが行い、previewやimportのHTTP request中には通信しません。
|
||||
|
||||
上限はファイル512 KiB、記事購読候補500行、outline 1,000個、XML階層16段です。既存sourceと合わせた
|
||||
32件上限を確認画面とwriter transactionの両方で検査します。既存・ファイル内重複は、hostの大小文字・
|
||||
既定443 port・空pathを正規化して判定します。queryの順序、pathの大小文字、redirect先を同一視しません。
|
||||
|
||||
previewはsource UUID・設定versionのsnapshotを返し、適用時に一致しなければ412で再確認を求めます。
|
||||
選択行の再解析、本人・Domain・有効状態の確認、source追加とreceipt保存を一つのtransactionで実行します。
|
||||
同じrequest_id・同じ入力は保存結果を返し、入力が異なれば拒否します。後でsourceを削除してもreceiptを
|
||||
保持するため、古い要求を再送して購読が復活することはありません。receiptはDomain・本人ごとに1,000件まで
|
||||
保持し、上限時は新しいimportを拒否します。既存のWiki/タスク・既読/starは変更しません。
|
||||
|
||||
## Wiki保存と権利・公開
|
||||
|
||||
@@ -146,7 +175,7 @@ Wiki単独は親Path配下の `rss-{request UUID}` に新規作成します。
|
||||
- [x] RSS-2:安全な共通一覧、出典付きWiki保存、任意のタスク化、ダッシュボード。
|
||||
- [ ] RSS-3:FreshRSSへの明示的な既読/star操作と送信状態・競合・再送の契約。
|
||||
- [x] RSS-4a:内蔵簡易reader、購読設定、既読・star、背景取得。
|
||||
- [ ] RSS-4b:OPML export/importのpreview、重複確認、明示移行。
|
||||
- [x] RSS-4b:OPML export/importのpreview、選択追加、重複確認、原子的な保存と再送防止。置換・状態移行は範囲外。
|
||||
- [ ] RSS-5:FreshRSS実環境の接続先・版・API資格情報を指定してread-only受入。
|
||||
- [ ] RSS-6:大規模利用時の公平性、host別並列上限、Retry-After/backoff、資格情報rotation UI。
|
||||
- [ ] RSS-7:FreshRSSの購読一覧・カテゴリを使うnavigation。現在の左ペインはPagesWikiに登録したsourceとそのフォルダで、FreshRSSの全購読一覧ではない。
|
||||
|
||||
@@ -111,7 +111,7 @@ P0は認可・整合性・復旧の安全条件、P1は日常利用の回復性
|
||||
|
||||
| ID / 優先 | 状態・次の作業 | 着手ゲート・最初の検証 | 詳細 |
|
||||
| --- | --- | --- | --- |
|
||||
| RSS-01 / P2 | 一部実装:内蔵簡易正本、FreshRSS読取connector、3ペインUI・キー操作、Wiki保存・タスク化・dashboard | 残件はOPML、FreshRSS既読/star書戻し、実接続受入、公平性/backoff。managed schema feeds v1・隔離fixture。本番未反映。二重正本・停止時の自動切替なし | [RSS](feeds.md) |
|
||||
| RSS-01 / P2 | 一部実装:内蔵簡易正本、FreshRSS読取connector、3ペインUI・HTML本文、Wiki保存・タスク化・dashboard、OPML確認付き入出力 | 残件はFreshRSS既読/star書戻し、購読/カテゴリnavigation、実接続受入、公平性/backoff。managed schema feeds v3・隔離fixture。本番未反映。二重正本・停止時の自動切替なし | [RSS](feeds.md) |
|
||||
| IMAP-01 / P3 | 未実装・選定待ち:IMAP読取同期 | provider/認証/保持を決める。UIDVALIDITY/UID、添付量、外部画像、失効をfixture化。neomutt captureは実装済みで別 | [メール](mail.md) |
|
||||
| TICKET-01 / P3 | 未実装:queue/番号/workflow/外部依頼者 | 共有範囲と状態遷移を定義、同時採番・失効・監査。既存個人タスクをticketと呼び替えない | [タスクとticket](tasks-tickets.md) |
|
||||
| EXT-01 / P3 | built-inのみ:第三者manifest/署名/隔離runtime、停止・rollback・観測 | capability・secret・host・予算・export契約。任意コードをGoプロセス内へ直接ロードしない | [拡張](extensions.md) |
|
||||
|
||||
@@ -42,6 +42,10 @@ func (s *Server) handleFeeds(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
key := strings.TrimPrefix(r.URL.Path, feedsAPI)
|
||||
if strings.HasPrefix(key, "opml/") {
|
||||
s.handleFeedOPML(w, r, domain, actor, key)
|
||||
return
|
||||
}
|
||||
switch {
|
||||
case key == "note" && r.Method == "POST":
|
||||
var input struct {
|
||||
|
||||
@@ -0,0 +1,275 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"encoding/xml"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"pageswiki/internal/store"
|
||||
)
|
||||
|
||||
const feedOPMLMaxBytes = 512 << 10
|
||||
|
||||
type feedOPMLRow struct {
|
||||
store.FeedSubscription
|
||||
Index int `json:"index"`
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
// Parse only the uploaded document. OPML includes and HTML links are never
|
||||
// fetched. Every token participates in the depth/outline/DTD checks.
|
||||
func parseFeedOPML(raw string) ([]feedOPMLRow, error) {
|
||||
if len(raw) > feedOPMLMaxBytes {
|
||||
return nil, store.ErrTaskDAVLimit
|
||||
}
|
||||
d := xml.NewDecoder(strings.NewReader(strings.TrimPrefix(raw, "\ufeff")))
|
||||
// JSON and the browser's File.text() carry UTF-8, not the original legacy
|
||||
// bytes. Reject other XML encodings instead of silently corrupting names.
|
||||
type frame struct {
|
||||
name xml.Name
|
||||
folder string
|
||||
commented bool
|
||||
}
|
||||
stack := []frame{}
|
||||
rows := []feedOPMLRow{}
|
||||
root, head, body, outlines := 0, 0, 0, 0
|
||||
for {
|
||||
token, err := d.Token()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return nil, store.ErrTaskInvalid
|
||||
}
|
||||
switch t := token.(type) {
|
||||
case xml.Directive:
|
||||
return nil, store.ErrTaskInvalid
|
||||
case xml.CharData:
|
||||
if len(stack) == 0 && strings.TrimSpace(string(t)) != "" {
|
||||
return nil, store.ErrTaskInvalid
|
||||
}
|
||||
case xml.StartElement:
|
||||
if len(stack) >= 16 {
|
||||
return nil, store.ErrTaskDAVLimit
|
||||
}
|
||||
attrs := map[string]string{}
|
||||
seen := map[xml.Name]bool{}
|
||||
for _, a := range t.Attr {
|
||||
if seen[a.Name] {
|
||||
return nil, store.ErrTaskInvalid
|
||||
}
|
||||
seen[a.Name] = true
|
||||
if a.Name.Space == "" {
|
||||
attrs[a.Name.Local] = a.Value
|
||||
}
|
||||
}
|
||||
f := frame{name: t.Name}
|
||||
if len(stack) == 0 {
|
||||
root++
|
||||
if root != 1 || t.Name.Space != "" || t.Name.Local != "opml" || (attrs["version"] != "1.0" && attrs["version"] != "1.1" && attrs["version"] != "2.0") {
|
||||
return nil, store.ErrTaskInvalid
|
||||
}
|
||||
} else {
|
||||
parent := stack[len(stack)-1]
|
||||
f.folder = parent.folder
|
||||
f.commented = parent.commented
|
||||
if len(stack) == 1 && t.Name.Space == "" {
|
||||
switch t.Name.Local {
|
||||
case "head":
|
||||
head++
|
||||
if head != 1 || body != 0 {
|
||||
return nil, store.ErrTaskInvalid
|
||||
}
|
||||
case "body":
|
||||
body++
|
||||
if head != 1 || body != 1 {
|
||||
return nil, store.ErrTaskInvalid
|
||||
}
|
||||
default:
|
||||
return nil, store.ErrTaskInvalid
|
||||
}
|
||||
}
|
||||
inBody := len(stack) >= 2 && stack[1].name == (xml.Name{Local: "body"})
|
||||
if inBody && t.Name.Space == "" && parent.name.Space == "" && (parent.name.Local == "body" || parent.name.Local == "outline") && t.Name.Local != "outline" {
|
||||
return nil, store.ErrTaskInvalid
|
||||
}
|
||||
if inBody && t.Name.Space == "" && t.Name.Local == "outline" && parent.name.Space == "" && (parent.name.Local == "body" || parent.name.Local == "outline") {
|
||||
outlines++
|
||||
if outlines > 1000 {
|
||||
return nil, store.ErrTaskDAVLimit
|
||||
}
|
||||
f.commented = f.commented || strings.EqualFold(attrs["isComment"], "true")
|
||||
label := attrs["text"]
|
||||
if label == "" {
|
||||
label = attrs["title"]
|
||||
}
|
||||
label = feedText(label, 1024)
|
||||
kind := strings.ToLower(attrs["type"])
|
||||
if attrs["xmlUrl"] != "" || kind != "" {
|
||||
if len(rows) >= 500 {
|
||||
return nil, store.ErrTaskDAVLimit
|
||||
}
|
||||
row := feedOPMLRow{FeedSubscription: store.FeedSubscription{Title: label, Folder: f.folder, URL: strings.TrimSpace(attrs["xmlUrl"])}, Index: len(rows), Status: "add"}
|
||||
switch {
|
||||
case f.commented:
|
||||
row.Status = "commented"
|
||||
case kind != "" && kind != "rss" && kind != "atom":
|
||||
row.Status = "unsupported_type"
|
||||
case !validCalendarSourceURL(row.URL):
|
||||
row.Status = "unsupported_url"
|
||||
case label == "" || len(label) > 200 || len(f.folder) > 200:
|
||||
row.Status = "invalid_fields"
|
||||
}
|
||||
rows = append(rows, row)
|
||||
} else if label != "" {
|
||||
if f.folder != "" {
|
||||
f.folder += " / "
|
||||
}
|
||||
f.folder += label
|
||||
}
|
||||
}
|
||||
}
|
||||
stack = append(stack, f)
|
||||
case xml.EndElement:
|
||||
if len(stack) == 0 {
|
||||
return nil, store.ErrTaskInvalid
|
||||
}
|
||||
stack = stack[:len(stack)-1]
|
||||
}
|
||||
}
|
||||
if root != 1 || head != 1 || body != 1 || len(stack) != 0 {
|
||||
return nil, store.ErrTaskInvalid
|
||||
}
|
||||
return rows, nil
|
||||
}
|
||||
|
||||
func classifyFeedOPML(rows []feedOPMLRow, sources []store.FeedSource) {
|
||||
existing := map[string]bool{}
|
||||
seen := map[string]bool{}
|
||||
for _, s := range sources {
|
||||
if s.Provider == "local" {
|
||||
existing[store.FeedURLKey(s.URL)] = true
|
||||
}
|
||||
}
|
||||
for i := range rows {
|
||||
row := &rows[i]
|
||||
if row.Status != "add" {
|
||||
continue
|
||||
}
|
||||
key := store.FeedURLKey(row.URL)
|
||||
if existing[key] {
|
||||
row.Status = "existing"
|
||||
} else if seen[key] {
|
||||
row.Status = "duplicate"
|
||||
}
|
||||
seen[key] = true
|
||||
}
|
||||
}
|
||||
|
||||
type feedOPMLOutline struct {
|
||||
Text string `xml:"text,attr"`
|
||||
Title string `xml:"title,attr,omitempty"`
|
||||
Type string `xml:"type,attr,omitempty"`
|
||||
URL string `xml:"xmlUrl,attr,omitempty"`
|
||||
Children []feedOPMLOutline `xml:"outline"`
|
||||
}
|
||||
|
||||
func encodeFeedOPML(sources []store.FeedSource) ([]byte, error) {
|
||||
var doc struct {
|
||||
XMLName xml.Name `xml:"opml"`
|
||||
Version string `xml:"version,attr"`
|
||||
Title string `xml:"head>title"`
|
||||
Outlines []feedOPMLOutline `xml:"body>outline"`
|
||||
}
|
||||
doc.Version = "2.0"
|
||||
doc.Title = "PagesWiki RSS"
|
||||
groups := map[string]int{}
|
||||
for _, source := range sources {
|
||||
if source.Provider != "local" {
|
||||
continue
|
||||
}
|
||||
entry := feedOPMLOutline{Text: source.Title, Title: source.Title, Type: "rss", URL: source.URL}
|
||||
if source.Folder == "" {
|
||||
doc.Outlines = append(doc.Outlines, entry)
|
||||
continue
|
||||
}
|
||||
i, ok := groups[source.Folder]
|
||||
if !ok {
|
||||
i = len(doc.Outlines)
|
||||
groups[source.Folder] = i
|
||||
doc.Outlines = append(doc.Outlines, feedOPMLOutline{Text: source.Folder})
|
||||
}
|
||||
doc.Outlines[i].Children = append(doc.Outlines[i].Children, entry)
|
||||
}
|
||||
raw, err := xml.MarshalIndent(doc, "", " ")
|
||||
return append([]byte(xml.Header), raw...), err
|
||||
}
|
||||
|
||||
func (s *Server) handleFeedOPML(w http.ResponseWriter, r *http.Request, domain, actor, key string) {
|
||||
if key == "opml/export" && r.Method == http.MethodGet {
|
||||
sources, err := s.store.FeedSources(r.Context(), domain, actor)
|
||||
if err != nil {
|
||||
taskError(w, err)
|
||||
return
|
||||
}
|
||||
raw, err := encodeFeedOPML(sources)
|
||||
if err != nil {
|
||||
taskError(w, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/x-opml; charset=utf-8")
|
||||
w.Header().Set("Content-Disposition", `attachment; filename="pageswiki-feeds.opml"`)
|
||||
w.Header().Set("X-Content-Type-Options", "nosniff")
|
||||
w.Write(raw)
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodPost || (key != "opml/preview" && key != "opml/import") {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
var input struct {
|
||||
OPML string `json:"opml"`
|
||||
Selected []int `json:"selected"`
|
||||
RequestID string `json:"request_id"`
|
||||
Expected string `json:"expected"`
|
||||
Enabled bool `json:"enabled"`
|
||||
Interval int `json:"interval_seconds"`
|
||||
}
|
||||
if !taskJSONLimit(w, r, &input, 2<<20) {
|
||||
return
|
||||
}
|
||||
rows, err := parseFeedOPML(input.OPML)
|
||||
if err != nil {
|
||||
taskError(w, err)
|
||||
return
|
||||
}
|
||||
if key == "opml/preview" {
|
||||
sources, err := s.store.FeedSources(r.Context(), domain, actor)
|
||||
if err != nil {
|
||||
taskError(w, err)
|
||||
return
|
||||
}
|
||||
classifyFeedOPML(rows, sources)
|
||||
writeEventJSON(w, 200, map[string]any{"rows": rows, "expected": store.FeedSourcesSnapshot(sources), "available": 32 - len(sources)})
|
||||
return
|
||||
}
|
||||
// Reparse on apply; client-supplied titles, URLs or row eligibility are not trusted.
|
||||
classifyFeedOPML(rows, nil)
|
||||
seen := map[int]bool{}
|
||||
plan := store.FeedImport{RequestID: input.RequestID, Expected: input.Expected, Enabled: input.Enabled, Interval: input.Interval}
|
||||
for _, index := range input.Selected {
|
||||
if index < 0 || index >= len(rows) || seen[index] || rows[index].Status != "add" {
|
||||
taskError(w, store.ErrTaskInvalid)
|
||||
return
|
||||
}
|
||||
seen[index] = true
|
||||
plan.Entries = append(plan.Entries, rows[index].FeedSubscription)
|
||||
}
|
||||
ids, err := s.store.ImportFeedSubscriptions(r.Context(), domain, actor, plan)
|
||||
if err != nil {
|
||||
taskError(w, err)
|
||||
return
|
||||
}
|
||||
writeEventJSON(w, 201, map[string]any{"source_ids": ids})
|
||||
}
|
||||
@@ -0,0 +1,240 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"pageswiki/internal/store"
|
||||
)
|
||||
|
||||
func opmlDoc(outlines string) string {
|
||||
return `<?xml version="1.0"?><opml version="2.0"><head><title>Test</title></head><body>` + outlines + `</body></opml>`
|
||||
}
|
||||
func opmlEntry(name, link string) string {
|
||||
return `<outline text="` + name + `" type="rss" xmlUrl="` + link + `"/>`
|
||||
}
|
||||
|
||||
func TestFeedOPMLParserAndExport(t *testing.T) {
|
||||
for _, version := range []string{"1.0", "1.1", "2.0"} {
|
||||
raw := strings.Replace(opmlDoc(`<outline text="技術"><outline text="Go">`+opmlEntry("A & B", "https://example.org/feed?a=1&b=2")+`</outline></outline>`), `version="2.0"`, `version="`+version+`"`, 1)
|
||||
rows, err := parseFeedOPML(raw)
|
||||
if err != nil || len(rows) != 1 || rows[0].Title != "A & B" || rows[0].Folder != "技術 / Go" || rows[0].URL != "https://example.org/feed?a=1&b=2" || rows[0].Status != "add" {
|
||||
t.Fatal(rows, err)
|
||||
}
|
||||
}
|
||||
for _, bad := range []string{
|
||||
strings.Replace(opmlDoc(""), `version="1.0"?>`, `version="1.0" encoding="iso-8859-1"?>`, 1),
|
||||
`<!DOCTYPE opml [<!ENTITY external SYSTEM "file:///etc/passwd">]>` + opmlDoc(""),
|
||||
opmlDoc("") + opmlDoc(""), opmlDoc("") + "garbage", `<opml version="2.0"><body/></opml>`,
|
||||
opmlDoc(`<outline text="x" text="y"/>`), opmlDoc(strings.Repeat(`<outline text="deep">`, 20) + strings.Repeat(`</outline>`, 20)),
|
||||
opmlDoc(strings.Repeat(opmlEntry("x", "https://example.org"), 501)), strings.Repeat("x", feedOPMLMaxBytes+1),
|
||||
opmlDoc(`<unexpected>` + opmlEntry("x", "https://example.org") + `</unexpected>`),
|
||||
} {
|
||||
if _, err := parseFeedOPML(bad); err == nil {
|
||||
t.Fatal("accepted invalid OPML")
|
||||
}
|
||||
}
|
||||
sources := []store.FeedSource{{Provider: "local", Title: "A & B", Folder: "技術 / Go", URL: "https://example.org/feed?a=1&b=2"}, {Provider: "local", Title: "Paused", Enabled: false, URL: "https://example.org/paused"}, {Provider: "freshrss", Title: "Remote", URL: "https://private.example.org/api/greader.php", Credential: "secret"}}
|
||||
raw, err := encodeFeedOPML(sources)
|
||||
if err != nil || strings.Contains(string(raw), "private.example.org") || strings.Contains(string(raw), "secret") {
|
||||
t.Fatal("export leak", err)
|
||||
}
|
||||
rows, err := parseFeedOPML(string(raw))
|
||||
if err != nil || len(rows) != 2 || rows[0].Folder != "技術 / Go" || rows[0].Title != "A & B" {
|
||||
t.Fatal("roundtrip", string(raw), rows, err)
|
||||
}
|
||||
raw, err = encodeFeedOPML(nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if rows, err = parseFeedOPML(string(raw)); err != nil || len(rows) != 0 {
|
||||
t.Fatal("empty export", string(raw), err)
|
||||
}
|
||||
}
|
||||
|
||||
type opmlPreview struct {
|
||||
Rows []feedOPMLRow `json:"rows"`
|
||||
Expected string `json:"expected"`
|
||||
Available int `json:"available"`
|
||||
}
|
||||
|
||||
func previewOPML(t *testing.T, s *Server, token, raw string) opmlPreview {
|
||||
t.Helper()
|
||||
w := feedRequestTest(s, token, "POST", "opml/preview", map[string]string{"opml": raw})
|
||||
var preview opmlPreview
|
||||
if w.Code != 200 || json.Unmarshal(w.Body.Bytes(), &preview) != nil {
|
||||
t.Fatal(w.Code, w.Body)
|
||||
}
|
||||
return preview
|
||||
}
|
||||
|
||||
func TestFeedOPMLPreviewImportReplayAndOwnership(t *testing.T) {
|
||||
s, token, domain, actor, source := feedFixture(t)
|
||||
raw := opmlDoc(`<outline text="日本語">` + opmlEntry("New", "https://example.org/new") + opmlEntry("Existing", "https://EXAMPLE.org:443/feed.xml") + opmlEntry("Duplicate", "https://example.org/new") + opmlEntry("Private", "https://127.0.0.1/feed") + `<outline text="Skipped" isComment="true">` + opmlEntry("Hidden", "https://example.org/hidden") + `</outline><outline type="include" text="Include" url="https://example.org/external.opml"/>` + opmlEntry("", "https://example.org/empty-title") + `</outline>`)
|
||||
p := previewOPML(t, s, token, raw)
|
||||
if p.Available != 31 || len(p.Rows) != 7 {
|
||||
t.Fatal(p)
|
||||
}
|
||||
for i, want := range []string{"add", "existing", "duplicate", "unsupported_url", "commented", "unsupported_type", "invalid_fields"} {
|
||||
if p.Rows[i].Status != want {
|
||||
t.Fatal(i, p.Rows[i])
|
||||
}
|
||||
}
|
||||
all, _ := s.store.FeedSources(context.Background(), domain, actor)
|
||||
if len(all) != 1 {
|
||||
t.Fatal("preview wrote sources")
|
||||
}
|
||||
for _, index := range []int{2, 3, 4, 5, 6} {
|
||||
w := feedRequestTest(s, token, "POST", "opml/import", map[string]any{"opml": raw, "expected": p.Expected, "selected": []int{index}, "request_id": uuid.NewString(), "interval_seconds": 1800})
|
||||
if w.Code != 422 {
|
||||
t.Fatal("ineligible row accepted", index, w.Code)
|
||||
}
|
||||
}
|
||||
input := map[string]any{"opml": raw, "expected": p.Expected, "selected": []int{0}, "request_id": uuid.NewString(), "enabled": false, "interval_seconds": 1800}
|
||||
w := feedRequestTest(s, token, "POST", "opml/import", input)
|
||||
if w.Code != 201 {
|
||||
t.Fatal(w.Code, w.Body)
|
||||
}
|
||||
original := w.Body.String()
|
||||
if retry := feedRequestTest(s, token, "POST", "opml/import", input); retry.Code != 201 || retry.Body.String() != original {
|
||||
t.Fatal("retry", retry.Code, retry.Body)
|
||||
}
|
||||
input["enabled"] = true
|
||||
if retry := feedRequestTest(s, token, "POST", "opml/import", input); retry.Code != 412 {
|
||||
t.Fatal("changed retry", retry.Code)
|
||||
}
|
||||
input["enabled"] = false
|
||||
all, _ = s.store.FeedSources(context.Background(), domain, actor)
|
||||
if len(all) != 2 {
|
||||
t.Fatal("duplicate import", len(all))
|
||||
}
|
||||
for _, v := range all {
|
||||
if v.UUID != source.UUID {
|
||||
if v.Enabled || v.Title != "New" || v.Folder != "日本語" {
|
||||
t.Fatal(v)
|
||||
}
|
||||
if err := s.store.FeedSourceAction(context.Background(), domain, actor, v.UUID, "delete", v.Version); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
if retry := feedRequestTest(s, token, "POST", "opml/import", input); retry.Code != 201 || retry.Body.String() != original {
|
||||
t.Fatal("deleted retry", retry.Code, retry.Body)
|
||||
}
|
||||
all, _ = s.store.FeedSources(context.Background(), domain, actor)
|
||||
if len(all) != 1 {
|
||||
t.Fatal("resurrected source")
|
||||
}
|
||||
w = feedRequestTest(s, token, "GET", "opml/export", nil)
|
||||
if w.Code != 200 || !strings.Contains(w.Header().Get("Content-Disposition"), "attachment") || !strings.Contains(w.Header().Get("Cache-Control"), "no-store") {
|
||||
t.Fatal(w.Code, w.Header())
|
||||
}
|
||||
other, err := s.store.CreatePasswordAccount(domain, "opmlother", "long test password", "Other", "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
otherToken, err := s.store.CreateAuthSession(domain, other.UUID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
w = feedRequestTest(s, otherToken, "GET", "opml/export", nil)
|
||||
if w.Code != 200 || strings.Contains(w.Body.String(), source.URL) {
|
||||
t.Fatal("owner leak", w.Code, w.Body)
|
||||
}
|
||||
if w = feedRequestTest(s, "", "GET", "opml/export", nil); w.Code == 200 {
|
||||
t.Fatal("anonymous export")
|
||||
}
|
||||
if err = s.store.SetPrincipalState(domain, actor, "readonly"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
token, err = s.store.CreateAuthSession(domain, actor)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if w = feedRequestTest(s, token, "GET", "opml/export", nil); w.Code != 200 {
|
||||
t.Fatal("readonly export", w.Code)
|
||||
}
|
||||
if w = feedRequestTest(s, token, "POST", "opml/import", input); w.Code != 403 {
|
||||
t.Fatal("readonly mutation", w.Code)
|
||||
}
|
||||
if err = s.setSystemExtensionEnabled(domain, "organizer.feeds", false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if w = feedRequestTest(s, token, "GET", "opml/export", nil); w.Code != 404 {
|
||||
t.Fatal("disabled export", w.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFeedOPMLAtomicityLimitsAndConcurrentApply(t *testing.T) {
|
||||
s, token, domain, actor, source := feedFixture(t)
|
||||
raw := opmlDoc(opmlEntry("A", "https://example.org/a") + opmlEntry("B", "https://example.org/b"))
|
||||
p := previewOPML(t, s, token, raw)
|
||||
input := map[string]any{"opml": raw, "expected": p.Expected, "selected": []int{0, 1}, "request_id": uuid.NewString(), "interval_seconds": 1800}
|
||||
if _, err := s.store.DB.Exec(`CREATE TRIGGER fail_feed_import BEFORE INSERT ON feed_import_receipts BEGIN SELECT RAISE(ABORT,'synthetic failure'); END`); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if w := feedRequestTest(s, token, "POST", "opml/import", input); w.Code != 500 {
|
||||
t.Fatal(w.Code, w.Body)
|
||||
}
|
||||
all, _ := s.store.FeedSources(context.Background(), domain, actor)
|
||||
if len(all) != 1 {
|
||||
t.Fatal("partial source insert")
|
||||
}
|
||||
if _, err := s.store.DB.Exec(`DROP TRIGGER fail_feed_import`); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// A configuration edit invalidates the preview; neither new source is applied.
|
||||
source.Title = "Changed"
|
||||
if _, err := s.store.SaveFeedSource(context.Background(), domain, actor, source); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if w := feedRequestTest(s, token, "POST", "opml/import", input); w.Code != 412 {
|
||||
t.Fatal("stale preview", w.Code)
|
||||
}
|
||||
p = previewOPML(t, s, token, raw)
|
||||
input["expected"] = p.Expected
|
||||
var wg sync.WaitGroup
|
||||
codes := make(chan int, 2)
|
||||
for range 2 {
|
||||
wg.Add(1)
|
||||
go func() { defer wg.Done(); codes <- feedRequestTest(s, token, "POST", "opml/import", input).Code }()
|
||||
}
|
||||
wg.Wait()
|
||||
close(codes)
|
||||
for code := range codes {
|
||||
if code != 201 {
|
||||
t.Fatal("concurrent retry", code)
|
||||
}
|
||||
}
|
||||
all, _ = s.store.FeedSources(context.Background(), domain, actor)
|
||||
if len(all) != 3 {
|
||||
t.Fatal("duplicate concurrent import")
|
||||
}
|
||||
entries := ""
|
||||
selected := []int{}
|
||||
for i := 0; i < 32; i++ {
|
||||
entries += opmlEntry(fmt.Sprint(i), fmt.Sprintf("https://example.org/large/%d", i))
|
||||
selected = append(selected, i)
|
||||
}
|
||||
raw = opmlDoc(entries)
|
||||
p = previewOPML(t, s, token, raw)
|
||||
input = map[string]any{"opml": raw, "expected": p.Expected, "selected": selected, "request_id": uuid.NewString(), "interval_seconds": 1800}
|
||||
if w := feedRequestTest(s, token, "POST", "opml/import", input); w.Code != 507 {
|
||||
t.Fatal("capacity", w.Code, w.Body)
|
||||
}
|
||||
all, _ = s.store.FeedSources(context.Background(), domain, actor)
|
||||
if len(all) != 3 {
|
||||
t.Fatal("partial limit import")
|
||||
}
|
||||
for _, selection := range [][]int{{-1}, {99}, {0, 0}, {}} {
|
||||
input["selected"] = selection
|
||||
if w := feedRequestTest(s, token, "POST", "opml/import", input); w.Code != 422 {
|
||||
t.Fatal("invalid selection", selection, w.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
Vendored
+8
-1
@@ -19,6 +19,13 @@
|
||||
d.querySelector('#feed-reader .feed-actions button:last-child').click();await wait(()=>d.querySelector('#feed-reader form'));
|
||||
const form=d.querySelector('#feed-reader form'),inputs=form.querySelectorAll('input');inputs[2].value='/notes/feed-ui';inputs[2].dispatchEvent(new w.Event('input'));inputs[5].checked=true;form.requestSubmit();await wait(()=>d.querySelector('#feed-reader a[href^="/v/uuid:"]'));
|
||||
const search=d.querySelector('#feed-toolbar input');search.value='';search.dispatchEvent(new w.Event('input',{bubbles:true}));await wait(()=>d.querySelectorAll('.feed-row').length===100);d.querySelector('.feed-row').click();await wait(()=>d.querySelector('#feed-reader .feed-body'));
|
||||
assert(d.documentElement.scrollWidth<=w.innerWidth,'Reader overflow');document.body.dataset.vimReady='1';
|
||||
assert(d.documentElement.scrollWidth<=w.innerWidth,'Reader overflow');
|
||||
d.getElementById('feed-opml').click();await wait(()=>d.getElementById('feed-opml-file'));
|
||||
const opml='<opml version="2.0"><head><title>Import</title></head><body><outline text="技術"><outline type="rss" text="Import '+w.innerWidth+'" xmlUrl="https://example.org/ui-'+w.innerWidth+'"/><outline type="rss" text="Existing" xmlUrl="https://example.org/feed.xml"/><outline type="rss" text="Private" xmlUrl="https://127.0.0.1/feed"/></outline></body></opml>';
|
||||
const choose=()=>{const transfer=new w.DataTransfer();transfer.items.add(new w.File([opml],'feeds.opml',{type:'text/x-opml'}));const f=d.getElementById('feed-opml-file');f.files=transfer.files;f.dispatchEvent(new w.Event('change',{bubbles:true}));};choose();
|
||||
await wait(()=>d.querySelectorAll('#feed-opml-preview tbody tr').length===3);const checks=d.querySelectorAll('#feed-opml-preview tbody input');assert(checks[0].checked&&!checks[0].disabled&&checks[1].disabled&&checks[2].disabled,'Import eligibility');
|
||||
assert(d.documentElement.scrollWidth<=w.innerWidth,'OPML overflow');d.querySelector('.feed-import-options input[type=checkbox]').checked=false;d.getElementById('feed-opml-preview').requestSubmit();await wait(()=>!d.getElementById('feed-opml-preview'));
|
||||
assert(d.getElementById('feed-sources').textContent.includes('Import '+w.innerWidth),'Imported source missing');d.getElementById('feed-opml').click();await wait(()=>d.getElementById('feed-opml-file'));choose();await wait(()=>d.querySelectorAll('#feed-opml-preview tbody tr').length===3);assert(d.querySelector('#feed-opml-preview tbody input').disabled,'Duplicate after import');
|
||||
assert(d.documentElement.scrollWidth<=w.innerWidth,'OPML preview overflow');document.body.dataset.vimReady='1';
|
||||
}catch(e){const out=document.createElement('pre');out.id='error';out.textContent=e.stack;document.body.append(out);}
|
||||
})();
|
||||
@@ -33,6 +33,10 @@ var feedMigrations = []db.SQLMigration{{Version: 1, Name: "private_feed_reader",
|
||||
// background fetch while fencing workers started before this migration.
|
||||
`UPDATE feed_sources SET etag='',modified='',next_ms=0,lease='',lease_until=0,version=version+1`,
|
||||
}, Assertions: []string{schemaColumns("feed_items", "body_html")},
|
||||
}, {
|
||||
Version: 3, Name: "feed_import_receipts", Statements: []string{
|
||||
`CREATE TABLE feed_import_receipts(domain_uuid TEXT NOT NULL,owner_uuid TEXT NOT NULL,request_id TEXT NOT NULL,input_hash TEXT NOT NULL,result TEXT NOT NULL,PRIMARY KEY(domain_uuid,owner_uuid,request_id))`,
|
||||
}, Assertions: []string{schemaUnique("feed_import_receipts", "domain_uuid,owner_uuid,request_id")},
|
||||
}}
|
||||
|
||||
// CaptureFeedNote creates an inert Wiki note and receipt atomically. It does not
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/url"
|
||||
"sort"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
type FeedSubscription struct {
|
||||
Title string `json:"title"`
|
||||
Folder string `json:"folder"`
|
||||
URL string `json:"url"`
|
||||
}
|
||||
|
||||
// Only normalize syntactic equivalence; never reorder query parameters, change
|
||||
// path case or follow redirects to decide identity.
|
||||
func FeedURLKey(raw string) string {
|
||||
u, err := url.Parse(strings.TrimSpace(raw))
|
||||
if err != nil {
|
||||
return raw
|
||||
}
|
||||
u.Scheme = strings.ToLower(u.Scheme)
|
||||
u.Host = strings.ToLower(u.Host)
|
||||
if u.Scheme == "https" && u.Port() == "443" {
|
||||
u.Host = strings.TrimSuffix(u.Host, ":443")
|
||||
}
|
||||
if u.Path == "" {
|
||||
u.Path = "/"
|
||||
}
|
||||
u.Fragment = ""
|
||||
return u.String()
|
||||
}
|
||||
|
||||
func FeedSourcesSnapshot(sources []FeedSource) string {
|
||||
versions := make([]string, 0, len(sources))
|
||||
for _, s := range sources {
|
||||
versions = append(versions, fmt.Sprintf("%s:%d", s.UUID, s.Version))
|
||||
}
|
||||
sort.Strings(versions)
|
||||
return fmt.Sprintf("%x", sha256.Sum256([]byte(strings.Join(versions, "\n"))))
|
||||
}
|
||||
|
||||
type FeedImport struct {
|
||||
RequestID string `json:"request_id"`
|
||||
Expected string `json:"expected"`
|
||||
Entries []FeedSubscription `json:"entries"`
|
||||
Enabled bool `json:"enabled"`
|
||||
Interval int `json:"interval_seconds"`
|
||||
}
|
||||
|
||||
// Append-only import: source inserts and the durable retry receipt commit as a
|
||||
// unit. A retry after later source deletion returns its receipt, never resurrects
|
||||
// the deleted subscription. Authorization is checked even on receipt replay.
|
||||
func (s *Store) ImportFeedSubscriptions(ctx context.Context, domain, owner string, input FeedImport) ([]string, error) {
|
||||
if _, err := uuid.Parse(input.RequestID); err != nil {
|
||||
return nil, ErrTaskInvalid
|
||||
}
|
||||
if len(input.Expected) != 64 || len(input.Entries) == 0 || len(input.Entries) > 32 || input.Interval < 300 || input.Interval > 86400 {
|
||||
return nil, ErrTaskInvalid
|
||||
}
|
||||
keys := map[string]bool{}
|
||||
for _, entry := range input.Entries {
|
||||
u, err := url.Parse(entry.URL)
|
||||
if err != nil || u.Scheme != "https" || u.Hostname() == "" || u.User != nil || u.Fragment != "" || (u.Port() != "" && u.Port() != "443") || len(entry.URL) > 4096 || strings.TrimSpace(entry.Title) == "" || len(entry.Title) > 200 || len(entry.Folder) > 200 || !utf8.ValidString(entry.Title+entry.Folder+entry.URL) {
|
||||
return nil, ErrTaskInvalid
|
||||
}
|
||||
key := FeedURLKey(entry.URL)
|
||||
if keys[key] {
|
||||
return nil, ErrTaskInvalid
|
||||
}
|
||||
keys[key] = true
|
||||
}
|
||||
raw, _ := json.Marshal(input)
|
||||
hash := fmt.Sprintf("%x", sha256.Sum256(raw))
|
||||
ids := []string{}
|
||||
err := s.feedWrite(ctx, domain, owner, func(tx *sql.Tx) error {
|
||||
var oldHash, result string
|
||||
err := tx.QueryRowContext(ctx, `SELECT input_hash,result FROM feed_import_receipts WHERE domain_uuid=? AND owner_uuid=? AND request_id=?`, domain, owner, input.RequestID).Scan(&oldHash, &result)
|
||||
if err == nil {
|
||||
if oldHash != hash {
|
||||
return ErrTaskConflict
|
||||
}
|
||||
return json.Unmarshal([]byte(result), &ids)
|
||||
}
|
||||
if !errors.Is(err, sql.ErrNoRows) {
|
||||
return err
|
||||
}
|
||||
rows, err := tx.QueryContext(ctx, `SELECT uuid,version,provider,url FROM feed_sources WHERE domain_uuid=? AND owner_uuid=?`, domain, owner)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
existing := []FeedSource{}
|
||||
for rows.Next() {
|
||||
var source FeedSource
|
||||
if err = rows.Scan(&source.UUID, &source.Version, &source.Provider, &source.URL); err != nil {
|
||||
rows.Close()
|
||||
return err
|
||||
}
|
||||
existing = append(existing, source)
|
||||
}
|
||||
err = rows.Err()
|
||||
rows.Close()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if FeedSourcesSnapshot(existing) != input.Expected {
|
||||
return ErrTaskConflict
|
||||
}
|
||||
if len(existing)+len(input.Entries) > 32 {
|
||||
return ErrTaskDAVLimit
|
||||
}
|
||||
for _, source := range existing {
|
||||
if source.Provider == "local" && keys[FeedURLKey(source.URL)] {
|
||||
return ErrTaskConflict
|
||||
}
|
||||
}
|
||||
var receipts int
|
||||
if err = tx.QueryRowContext(ctx, `SELECT count(*) FROM feed_import_receipts WHERE domain_uuid=? AND owner_uuid=?`, domain, owner).Scan(&receipts); err != nil {
|
||||
return err
|
||||
}
|
||||
if receipts >= 1000 {
|
||||
return ErrTaskDAVLimit
|
||||
}
|
||||
for _, entry := range input.Entries {
|
||||
id := uuid.NewString()
|
||||
if _, err = tx.ExecContext(ctx, `INSERT INTO feed_sources(uuid,domain_uuid,owner_uuid,provider,title,folder,url,version,enabled,interval_seconds) VALUES(?,?,?,'local',?,?,?,1,?,?)`, id, domain, owner, entry.Title, entry.Folder, entry.URL, input.Enabled, input.Interval); err != nil {
|
||||
return err
|
||||
}
|
||||
ids = append(ids, id)
|
||||
}
|
||||
encoded, _ := json.Marshal(ids)
|
||||
_, err = tx.ExecContext(ctx, `INSERT INTO feed_import_receipts VALUES(?,?,?,?,?)`, domain, owner, input.RequestID, hash, string(encoded))
|
||||
return err
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ids, nil
|
||||
}
|
||||
@@ -71,3 +71,41 @@ func TestFeedMigrationAtomicRepeatAndCleanup(t *testing.T) {
|
||||
t.Fatal(result, e)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFeedImportReceiptMigrationRollbackAndRepeat(t *testing.T) {
|
||||
st := newSchemaTestStore(t)
|
||||
ctx := context.Background()
|
||||
if err := db.ApplySQLiteMigrations(ctx, st.DB, "feeds", feedMigrations[:2]); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := st.DB.Exec(`INSERT INTO feed_sources(uuid,domain_uuid,owner_uuid,provider,title,folder,url,version,enabled,interval_seconds) VALUES('s','d','u','local','title','','https://example.org',7,1,300)`); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
broken := append([]db.SQLMigration(nil), feedMigrations...)
|
||||
broken[2].Statements = append(append([]string(nil), feedMigrations[2].Statements...), `INSERT INTO no_such_feed_table VALUES(1)`)
|
||||
if err := db.ApplySQLiteMigrations(ctx, st.DB, "feeds", broken); err == nil {
|
||||
t.Fatal("expected v3 failure")
|
||||
}
|
||||
var count int
|
||||
if err := st.DB.QueryRow(`SELECT count(*) FROM sqlite_master WHERE name='feed_import_receipts'`).Scan(&count); err != nil || count != 0 {
|
||||
t.Fatal("partial migration", count, err)
|
||||
}
|
||||
if err := st.ensureSchemaComponent("feeds"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := st.DB.Exec(`INSERT INTO feed_import_receipts VALUES('d','u','request','hash','["s"]')`); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := st.ensureSchemaComponent("feeds"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := st.DB.QueryRow(`SELECT version FROM feed_sources WHERE uuid='s'`).Scan(&count); err != nil || count != 7 {
|
||||
t.Fatal("unrelated data changed", count, err)
|
||||
}
|
||||
if _, err := st.DB.Exec(`DELETE FROM feed_sources WHERE uuid='s'`); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := st.DB.QueryRow(`SELECT count(*) FROM feed_import_receipts`).Scan(&count); err != nil || count != 1 {
|
||||
t.Fatal("receipt must survive deletion", count, err)
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
.feeds-page{height:100dvh;overflow:hidden;display:flex;flex-direction:column;background:var(--bg);color:var(--fg)}
|
||||
.feed-opml{display:grid;gap:.6rem;min-width:0}.feed-opml>label{display:grid;gap:.25rem}.feed-opml input[type=file]{width:100%;box-sizing:border-box}.feed-opml table{width:100%;table-layout:fixed;border-collapse:collapse;font-size:.8rem}.feed-opml th,.feed-opml td{text-align:left;vertical-align:top;padding:.3rem;border-bottom:1px solid var(--border,#8885);overflow-wrap:anywhere}.feed-opml th:first-child{width:1.6rem}.feed-opml th:nth-child(3){width:20%}.feed-opml th:last-child{width:20%}.feed-opml small{display:block;font-size:.7rem;opacity:.7;overflow-wrap:anywhere}#feeds-app .feed-opml input[type=checkbox]{width:auto;min-height:1.2rem}.feed-import-options{display:flex;gap:.75rem;align-items:center;flex-wrap:wrap}#feeds-app .feed-import-options label:has(input[type=checkbox]){display:flex;align-items:center}.feed-opml input:disabled{opacity:.4}
|
||||
#feed-status:empty{display:none}#feed-reader .feed-body-html{white-space:normal}.feed-original{display:inline-flex;align-items:center;justify-content:center;min-width:2rem;min-height:2rem;border-radius:.2rem}.feed-original:hover{background:var(--border,#8883)}.feed-original svg{width:1.1rem;height:1.1rem;fill:none;stroke:currentColor;stroke-width:1.7;stroke-linecap:round;stroke-linejoin:round}
|
||||
.feeds-page .topbar,.feeds-page .organizer-nav{flex-shrink:0}.feeds-page #feeds-app{flex:1;min-height:0;display:flex;flex-direction:column;width:100%;max-width:none;margin:0;padding:0 .5rem .25rem;box-sizing:border-box}
|
||||
#feed-toolbar{display:flex;align-items:center;gap:.35rem;min-height:2.5rem;flex-wrap:wrap}#feed-toolbar input{flex:1;min-width:8rem;max-width:32rem}#feeds-app button,#feeds-app input,#feeds-app select{font:inherit;color:inherit;background:var(--panel,var(--bg));border:1px solid var(--border,#6666);border-radius:.2rem;padding:.2rem .45rem;min-height:1.8rem}#feeds-app button{cursor:pointer}#feeds-app button:disabled{opacity:.45;cursor:default}#feeds-app a{color:var(--accent,#659de4)}#feed-status{margin:0;min-height:1.25rem;font-size:.78rem;color:var(--muted,var(--fg));overflow-wrap:anywhere}
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
const refresh=button('↻',t('Queue source refresh (r)','背景更新を予約 (r)'),queueRefresh);refresh.disabled=!writable;
|
||||
toolbar.append(button('☰',t('Sources','購読元'),()=>layout.classList.toggle('sources-open')),sourceLabel,searchbox,filters,refresh);
|
||||
if(writable)toolbar.append(button('+',t('Add source','購読元を追加'),()=>sourceForm()));
|
||||
const transfer=button('⇄','OPML',opmlForm);transfer.id='feed-opml';toolbar.append(transfer);
|
||||
toolbar.append(button('?',t('Keyboard shortcuts','キー操作'),showHelp));
|
||||
let debounce;searchbox.addEventListener('input',()=>{clearTimeout(debounce);debounce=setTimeout(()=>{search=searchbox.value.trim();load(false).catch(report);},180);});
|
||||
const columns=document.getElementById('feed-columns');for(const s of ['',t('Date','日付'),t('Feed','配信元'),t('Subject','件名')])columns.append(el('span',s));
|
||||
@@ -54,6 +55,33 @@
|
||||
}
|
||||
async function setState(read,starred){const item=items[selected];if(!item||!writable||item.provider!=='local')return;const source=item.source,id=item.id;const data=await post('state',{source,id,read,starred,version:item.version});Object.assign(item,{read,starred,version:data.version});drawRows();if(items[selected]===item&&!editing)renderItem(item);}
|
||||
function field(form,label,value='',type='text'){const wrap=el('label',label),input=el('input');input.type=type;input.value=value;wrap.append(input);form.append(wrap);return input;}
|
||||
function opmlForm(){
|
||||
editing=true;detailSeq++;detailController?.abort();layout.classList.add('reading');reader.replaceChildren();
|
||||
const section=el('section',undefined,'feed-opml'),heading=el('div',undefined,'feed-actions');heading.append(el('h1','OPML'),button('←',t('Back','戻る'),closeReader));section.append(heading);reader.append(section);
|
||||
const exportLink=el('a',t('Export local subscriptions','内蔵購読を書き出す'));exportLink.href=api+'opml/export';exportLink.download='pageswiki-feeds.opml';section.append(exportLink);if(!writable)return;
|
||||
const fileLabel=el('label',t('Import local subscriptions','内蔵購読を取り込む')),file=el('input');file.type='file';file.accept='.opml,.xml,text/x-opml,application/xml,text/xml';file.id='feed-opml-file';fileLabel.append(file);section.append(fileLabel);
|
||||
const preview=el('div');section.append(preview);let generation=0;
|
||||
const reasons={add:t('Add','追加'),existing:t('Subscribed','登録済み'),duplicate:t('Duplicate','重複'),unsupported_url:t('Unsupported URL','未対応URL'),unsupported_type:t('Unsupported type','未対応形式'),invalid_fields:t('Invalid name/folder','名前・フォルダ不正'),commented:t('Commented','コメント扱い')};
|
||||
file.onchange=async()=>{
|
||||
const version=++generation,chosen=file.files[0];preview.replaceChildren();status.textContent='';if(!chosen)return;
|
||||
try{
|
||||
if(chosen.size>512*1024)throw Error(t('OPML must be at most 512 KiB.','OPMLは512 KiB以内で指定してください。'));
|
||||
const opml=await chosen.text(),data=await post('opml/preview',{opml});if(version!==generation||!section.isConnected)return;
|
||||
const form=el('form');form.id='feed-opml-preview';preview.append(form);const table=el('table'),head=el('thead'),tr=el('tr');for(const label of ['',t('Subscription','購読'),t('Folder','フォルダ'),t('Status','状態')])tr.append(el('th',label));head.append(tr);table.append(head);const body=el('tbody');table.append(body);form.append(table);
|
||||
const choices=[];let remaining=data.available;
|
||||
for(const row of data.rows){const tr=el('tr'),cell=el('td'),check=el('input');check.type='checkbox';check.value=row.index;check.ariaLabel=row.title;check.disabled=row.status!=='add';check.checked=!check.disabled&&remaining-->0;cell.append(check);const name=el('td');name.append(el('strong',row.title||'—'),el('small',row.url));tr.append(cell,name,el('td',row.folder||'—'),el('td',reasons[row.status]||row.status));body.append(tr);if(!check.disabled)choices.push(check);}
|
||||
const options=el('div',undefined,'feed-import-options'),enabled=field(options,t('Enable','有効'),'','checkbox');enabled.checked=true;const interval=field(options,t('Interval (min)','更新間隔(分)'),'30','number');interval.min='5';interval.max='1440';interval.required=true;form.append(options);
|
||||
const actions=el('div',undefined,'feed-actions'),save=el('button');save.type='submit';save.id='feed-opml-apply';actions.append(save);form.append(actions);
|
||||
let busy=false,pending=null;
|
||||
const update=()=>{const n=choices.filter(x=>x.checked).length;save.textContent=t('Add ','追加 ')+n;save.disabled=busy||n===0||n>data.available;save.title=n>data.available?t('Subscription limit exceeded','購読数の上限を超えています'):'';choices.forEach(check=>check.disabled=busy);enabled.disabled=interval.disabled=busy;};choices.forEach(check=>check.onchange=update);update();
|
||||
form.onsubmit=async event=>{
|
||||
event.preventDefault();if(busy)return;const selected=choices.filter(x=>x.checked).map(x=>Number(x.value));if(!selected.length||selected.length>data.available)return;
|
||||
const input={opml,selected,expected:data.expected,enabled:enabled.checked,interval_seconds:Number(interval.value)*60},fingerprint=JSON.stringify(input);if(!pending||pending.fingerprint!==fingerprint)pending={fingerprint,request_id:crypto.randomUUID()};input.request_id=pending.request_id;busy=true;file.disabled=true;update();
|
||||
try{await post('opml/import',input);if(!section.isConnected)return;await reload();status.textContent=t('Imported','取り込みました');}catch(e){report(e);}finally{busy=false;file.disabled=false;update();}
|
||||
};
|
||||
}catch(e){if(version===generation&§ion.isConnected)report(e);}
|
||||
};
|
||||
}
|
||||
function sourceForm(source){editing=true;detailSeq++;layout.classList.add('reading');reader.replaceChildren();const form=el('form'),buttons=el('div',undefined,'feed-actions');reader.append(form);form.append(el('h1',t('Source settings','購読元設定')));
|
||||
const title=field(form,t('Name','表示名'),source?.title||''),folder=field(form,t('Folder','フォルダ'),source?.folder||'');title.required=true;title.maxLength=folder.maxLength=200;
|
||||
const label=el('label',t('Provider','正本')),provider=el('select');for(const [id,name]of [['local',t('Local · RSS / Atom','内蔵 · RSS / Atom')],['freshrss','FreshRSS']]){const o=el('option',name);o.value=id;provider.append(o);}provider.value=source?.provider||'local';provider.disabled=!!source;label.append(provider);form.append(label);
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
{{ define "feeds.gohtml" }}<!doctype html>
|
||||
<html lang="{{ .Locale }}"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<title>{{ T "feeds.title" }} · {{ .WikiName }}</title>
|
||||
<link rel="stylesheet" href="/static/css/main.css"><link rel="stylesheet" href="/static/css/theme/base.css"><link rel="stylesheet" href="{{ .ThemeCSS }}"><link rel="stylesheet" href="/static/css/feeds.css?v=2"></head>
|
||||
<link rel="stylesheet" href="/static/css/main.css"><link rel="stylesheet" href="/static/css/theme/base.css"><link rel="stylesheet" href="{{ .ThemeCSS }}"><link rel="stylesheet" href="/static/css/feeds.css?v=3"></head>
|
||||
<body class="mode-view organizer-page feeds-page"><header class="topbar"><div class="brand"><a href="/">{{ .WikiName }}</a></div>{{ template "auth-menu" . }}</header>{{ template "organizer-nav" . }}
|
||||
<main id="feeds-app" data-writable="{{ .FeedWritable }}" data-tasks="{{ .TasksEnabled }}">
|
||||
<div id="feed-toolbar"></div><p id="feed-status" role="status" aria-live="polite"></p>
|
||||
<div class="feed-layout"><aside id="feed-sources" aria-label="{{ T "feeds.title" }}"></aside><section class="feed-index"><div id="feed-columns" aria-hidden="true"></div><div id="feed-rows" tabindex="0" role="listbox" aria-label="{{ T "feeds.title" }}"></div><button type="button" id="feed-more" hidden>↓</button></section><article id="feed-reader" tabindex="0"></article></div>
|
||||
</main><script src="/static/js/feeds.js?v=2" defer></script></body></html>{{ end }}
|
||||
</main><script src="/static/js/feeds.js?v=3" defer></script></body></html>{{ end }}
|
||||
Reference in new issue
Block a user