Refactor architecture into eight components and consolidate reliability work
Build and publish container images / images (push) Successful in 9m34s
Release container images / release (push) Successful in 36s

Extract Grammaratur, Dregarnuhr, Geduldh, Mestionora, Ordoschnellic, Schutzaria, Kunstzeal and Seheweit behind compatible application contracts.

Include accumulated editor resilience, navigation and theme UX, remote CLI, storage/schema and DR groundwork, tests and design/operations documentation.

Verified: uncached full Go suite; go vet ./...; race tests for all eight components; 11 targeted Chromium tests; Node suites; 19 Python CLI tests; reproducible editor build. No push, release tag or deployment in this checkpoint.
This commit is contained in:
Anya Tokugawa committed 2026-11-01 15:23:31 +09:00
1 parent 9dd425f765
commit d5d21b7841
302 files changed
+20037 -3492

No files matched your search

+108
View File
@@ -0,0 +1,108 @@
# pages-cli
**`pages` — PagesWikiをターミナルから読む、たどる、編集する。**
プロファイルや言語は初回に設定し、普段のコマンドは短く保ちます。
DBに直接アクセスする `data/pageswiki-cli` とは独立したHTTPクライアントです。
## はじめる
Python 3.9以降、Linux/macOS等のUnix系環境を使用します。外部Pythonパッケージは不要です。
リポジトリ内では `./pages-cli/pages` で実行できます。PATHへ配置する場合:
```sh
mkdir -p ~/.local/bin
install -m 755 pages-cli/pages ~/.local/bin/pages
# ~/.local/bin をPATHに含めてください。
```
```sh
pages configure --origin https://pages.example.test --username anya
pages login
pages
```
パスワードと必要なTOTPは非表示プロンプトへ入力します。`pages` 単独で対話シェル、
`pages ls /manual` のように続ければワンショット操作です。
```text
pages[default]:/> cd /manual
pages[default]:/manual> ls
pages[default]:/manual> edit start
pages[default]:/manual> exit
```
`exit` はシェルを閉じるだけです。セッションを失効させる場合は `pages logout`。
## 設定する
```sh
pages configure # 設定を表示(トークンは表示しない)
pages configure --lang ja # 現在の既定接続の編集言語
pages configure list # 接続先一覧、★が既定
pages configure set --profile work --origin https://work.example.test --username anya --lang en
pages configure use work # 以後はworkを指定不要
pages status
pages edit /guide
```
既定プロファイル名は `default`、既定言語は `und`(言語未指定)です。OSや表示言語から
書き込み先の言語を推測しません。最初に名前付きプロファイルを作った場合はそれを既定とします。
言語は接続先ごとに保持し、編集時はその言語の既存Revisionを使用します。
一度だけ変更したいときは `pages --profile personal --lang ja edit /guide` も使えます。
プロファイル自体は先にconfigureで登録します。引数の上書きは設定ファイルを書き換えません。
設定ファイルは `~/.config/pages-cli/config.json`(`XDG_CONFIG_HOME`対応)。
```json
{
"version": 1,
"default_profile": "default",
"profiles": {
"default": {
"origin": "https://pages.example.test",
"username": "anya",
"lang": "und"
}
}
}
```
セッションは別の `sessions/PROFILE.json` に保存します。設定にパスワードやBearerを入れないでください。
ディレクトリ0700/ファイル0600を要求します。セッション保持中のorigin・username変更には
事前logoutが必要です。同じプロファイルのシェルが開いている間の設定変更はロックで拒否します。
手編集で接続先を変えても、旧セッションを新接続先に送信しません。
旧試作名 `data/pageswiki-remote` は新コマンドへの案内用ランチャーです。
旧 `~/.config/pageswiki-remote/` の資格情報は自動移行・削除しません。
新しいconfigureで接続先を登録して再ログインし、不要な旧セッションはサーバのセッション管理で失効してください。
## 操作
| コマンド | 用途 |
| --- | --- |
| `ls` / `cd` / `pwd` | Semantic Pathをたどる。仮想フォルダにも移動可能 |
| `cat` / `view` | 正本の出力/`$PAGER`で閲覧 |
| `get ADDRESS LOCAL` | 正本をダウンロード。既存のローカルファイルは上書きしない |
| `edit ADDRESS` | `$EDITOR`で編集。Wikiはauthoring Markdown、他のテキストは原文 |
| `put LOCAL ADDRESS --base REVISION_UUID` | 明示した基点RevisionとのCASで更新 |
| `search QUERY` | 権限付きの検索候補 |
| `open ADDRESS` | ブラウザで閲覧。ブラウザ側ログインを使用 |
| `status` / `logout` | 接続状態表示/セッション失効 |
`pages --help`、`pages configure --help`、`pages edit --help` で引数を確認できます。
Tab補完、上下キーのプロセス内履歴に対応。複雑な引用位置の補完、Ctrl+Pのfuzzy UIは後続です。
初期版は既存Page・既存言語の更新まで。新規作成、OIDC引き継ぎ、バイナリ更新は未実装です。
競合や通信断で自動上書きせず、ローカル原稿を保持します。HTTPS必須(開発用loopbackのみHTTP可)。
詳しいURI・保存・安全性の契約は[設計書](../docs/design/ja/remote-cli.md)を参照してください。
## テスト
```sh
python3 pages-cli/test_pages.py
go test ./internal/server -run TestRemoteCLIHTTPIntegration -count=1
```
偽サーバと一時DBのみを使用します。本番資格情報や外部カレンダーは不要です。
+528
View File
@@ -0,0 +1,528 @@
#!/usr/bin/env python3
"""pages — navigate and edit PagesWiki from your terminal (pages-cli)."""
import argparse
import contextlib
import difflib
import fcntl
import getpass
import hashlib
import json
import os
from pathlib import Path
import posixpath
import re
import shlex
import stat
import subprocess
import sys
import tempfile
import time
import unicodedata
import urllib.error
import urllib.parse as url
import urllib.request as http
import webbrowser
class Failure(Exception):
def __init__(self, message, status=0, payload=None):
super().__init__(message)
self.status, self.payload = status, payload or {}
def safe_text(value):
return ''.join(c for c in str(value) if c in '\n\t' or not unicodedata.category(c).startswith('C'))
def origin(value):
if not isinstance(value, str): raise Failure('Origin must be an HTTPS URL string.')
p = url.urlsplit(value)
if (p.scheme not in ('https', 'http') or not p.hostname or p.username or p.password
or p.path not in ('', '/') or p.query or p.fragment or any(c.isspace() for c in value)):
raise Failure('Use an HTTPS origin without credentials, path, query or fragment.')
if p.scheme == 'http' and p.hostname not in ('127.0.0.1', '::1', 'localhost'):
raise Failure('HTTP is permitted only for loopback development; use HTTPS remotely.')
return url.urlunsplit((p.scheme, p.netloc.lower(), '', '', ''))
def semantic(value, cwd='/'):
value = value.removeprefix('path:')
if '://' in value or '\\' in value or any(unicodedata.category(c).startswith('C') for c in value):
raise Failure('Expected a semantic Path, not a URL or control characters.')
return '/' + posixpath.normpath('/' + posixpath.join(cwd, value)).lstrip('/')
def address(value, cwd='/'):
if value.startswith(('uuid:', 'rev:')):
kind, ident = value.split(':', 1)
if not re.fullmatch(r'[0-9a-fA-F-]{36}', ident):
raise Failure('Invalid UUID address.')
return kind + ':' + ident
if re.match(r'^[a-zA-Z]+:', value) and not value.startswith('path:'):
raise Failure('This client supports path:, uuid: and read-only rev: addresses.')
return 'path:' + semantic(value, cwd)
class NoRedirect(http.HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
return None # Never forward credentials or turn a login HTML page into success.
class Client:
def __init__(self, base, token='', lang='und'):
self.base, self.token, self.lang = origin(base), token, lang
self.opener = http.build_opener(NoRedirect())
def request(self, route, method='GET', data=None, form=None, limit=4 << 20):
if not route.startswith('/') or route.startswith('//'):
raise Failure('Invalid API route.')
headers = {'Accept': 'application/json', 'User-Agent': 'pages-cli/0.1'}
if self.token:
headers['Authorization'] = 'Bearer ' + self.token
body = None
if data is not None:
body = json.dumps(data, ensure_ascii=False).encode()
headers['Content-Type'] = 'application/json'
if form is not None:
body = url.urlencode(form).encode()
headers['Content-Type'] = 'application/x-www-form-urlencoded'
if body is not None and len(body) > 2 << 20:
raise Failure('Request exceeds the 2 MiB client limit.')
req = http.Request(self.base + route, body, headers, method=method)
try:
with self.opener.open(req, timeout=20) as response:
raw = response.read(limit + 1)
if len(raw) > limit:
raise Failure('Response exceeds client limit; use the browser for large files.')
return raw, response.headers
except urllib.error.HTTPError as e:
raw = e.read(65536)
e.close()
try:
payload = json.loads(raw)
except (ValueError, UnicodeError):
payload = {}
if not isinstance(payload, dict):
payload = {}
messages = {303: 'Login redirect refused; log in again.', 401: 'Session expired or authentication required.', 403: 'Access denied.',
404: 'Not found or not readable.', 409: 'Conflict: remote revision changed.'}
raise Failure(messages.get(e.code, 'HTTP request failed (%d); redirects are not followed.' % e.code), e.code, payload) from None
except (urllib.error.URLError, TimeoutError, OSError):
raise Failure('Connection/TLS failure; no automatic write retry was attempted.') from None
def json(self, route, **kwargs):
raw, headers = self.request(route, **kwargs)
if headers.get_content_type() != 'application/json':
raise Failure('Expected JSON, not a login page or other projection.')
try:
value = json.loads(raw)
if not isinstance(value, dict):
raise ValueError()
return value
except (ValueError, UnicodeError):
raise Failure('Invalid JSON response.') from None
def login(self, username, password, otp=getpass.getpass):
try:
result = self.json('/_/auth/login', method='POST', form={'username': username, 'password': password})
except Failure as e:
if e.status != 401 or not e.payload.get('mfa_required') or not e.payload.get('challenge_token'):
raise
result = self.json('/_/auth/totp', method='POST', form={
'challenge_token': e.payload['challenge_token'], 'code': otp('TOTP: ')})
token = result.get('access_token')
if not isinstance(token, str) or not re.fullmatch(r'[A-Za-z0-9_.~-]{16,1024}', token) or result.get('token_type') != 'Bearer':
raise Failure('Server did not issue a Bearer session.')
self.token = token
return time.time() + int(result['expires_in'])
def listing(self, path, prefix=''):
return self.json('/_/api/v1/path-picker?' + url.urlencode({'path': path, 'mode': 'children', 'prefix': prefix}))
def raw(self, addr, method='GET'):
raw, headers = self.request('/v/raw/' + url.quote(addr, safe='/:') + '?' + url.urlencode({'lang': self.lang}), method=method, limit=64 << 20)
digest = headers.get('X-PagesWiki-Document-SHA256')
if method == 'GET' and (not digest or hashlib.sha256(raw).hexdigest() != digest):
raise Failure('Document SHA-256 mismatch.')
return raw, headers
def snapshot(self, addr):
if addr.startswith('rev:'):
raise Failure('Revision addresses are immutable; edit a path: or uuid: address.')
_, headers = self.raw(addr, 'HEAD')
ident = headers.get('X-PagesWiki-Object-UUID')
if not ident:
raise Failure('Missing Object UUID metadata.')
state = self.json('/_/api/v1/editor/state?' + url.urlencode({'uuid': ident, 'lang': self.lang}))
if not state.get('editable') or not state.get('principal_uuid'):
raise Failure('Not an authenticated editable text document; use the browser.')
if not state.get('head_revision_uuid') or not state.get('content_type'):
raise Failure('No saved revision in this exact language; creation/translation is not supported yet.')
state['object_uuid'] = ident
return state
def save(self, state, text, markdown=False):
if len(text.encode()) > 1 << 20:
raise Failure('Text edit exceeds the 1 MiB client limit.')
if markdown:
if not state['content_type'].startswith('text/html'):
raise Failure('Markdown authoring is supported only for HTML Wiki documents.')
text = self.json('/_/api/v1/render/markdown', method='POST', data={
'markdown': text, 'authoring': True, 'projection': False})['html']
if text == state['body']:
return {'status': 'unchanged', 'revision_uuid': state['head_revision_uuid']}
return self.json('/_/api/v1/revisions/save', method='POST', data={
'object_uuid': state['object_uuid'], 'body': text, 'lang': state['lang'],
'content_type': state['content_type'], 'filename': state['filename'],
'base_revision_uuid': state['head_revision_uuid'],
'editor_principal_uuid': state['principal_uuid'], 'message': 'Remote CLI edit'})
def private_file(path):
info = path.lstat()
if not stat.S_ISREG(info.st_mode) or info.st_uid != os.getuid() or info.st_mode & 0o077:
raise Failure('Credential file must be a user-owned regular file with mode 0600.')
class PrivateJSON:
def __init__(self, root, filename):
self.root = Path(root)
self.root.mkdir(mode=0o700, parents=True, exist_ok=True)
info = self.root.lstat()
if not stat.S_ISDIR(info.st_mode) or info.st_uid != os.getuid() or info.st_mode & 0o077:
raise Failure('Profile directory must be owned by you with mode 0700; symlinks are not accepted.')
self.path = self.root / filename
self.data = {}
@contextlib.contextmanager
def locked(self):
fd = os.open(str(self.path) + '.lock', os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW, 0o600)
with os.fdopen(fd, 'w') as lock:
try:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
raise Failure('This profile is already in use by another CLI process.') from None
if self.path.exists() or self.path.is_symlink():
private_file(self.path)
self.data = json.loads(self.path.read_text())
if not isinstance(self.data, dict):
raise Failure('Invalid configuration object.')
yield self
def save(self):
fd, temp = tempfile.mkstemp(dir=self.root, prefix='.profile-')
try:
with os.fdopen(fd, 'w') as out:
json.dump(self.data, out)
out.flush()
os.fsync(out.fileno())
os.replace(temp, self.path)
finally:
if os.path.exists(temp):
os.unlink(temp)
def config_root(root=None):
return Path(root or os.environ.get('XDG_CONFIG_HOME', str(Path.home() / '.config'))) / 'pages-cli'
def profile_name(name):
if not isinstance(name, str) or not re.fullmatch(r'[A-Za-z0-9_-]{1,64}', name):
raise Failure('Profile names use 1–64 letters, digits, underscores or hyphens.')
return name
def language(value):
if not isinstance(value, str) or not re.fullmatch(r'[A-Za-z]{1,8}(?:-[A-Za-z0-9]{1,8})*', value):
raise Failure('Use a language tag such as und, ja or en-US; the server validates the exact tag.')
return value
class Profile(PrivateJSON):
def __init__(self, name, root=None):
self.name = profile_name(name)
# Validate the parent before creating sessions, including on first use.
PrivateJSON(config_root(root), 'config.json')
super().__init__(config_root(root) / 'sessions', profile_name(name) + '.json')
class Configuration(PrivateJSON):
def __init__(self, root=None):
super().__init__(config_root(root), 'config.json')
def validated(self):
if not self.data:
return {'version': 1, 'default_profile': 'default', 'profiles': {}}
if set(self.data) - {'version', 'default_profile', 'profiles'} or self.data.get('version') != 1:
raise Failure('Unsupported configuration format; credentials belong in sessions/, not config.json.')
default = profile_name(self.data.get('default_profile', 'default'))
profiles = self.data.get('profiles', {})
if not isinstance(profiles, dict): raise Failure('Invalid profiles configuration.')
normalized = {}
for name, spec in profiles.items():
profile_name(name)
if not isinstance(spec, dict) or set(spec) - {'origin', 'username', 'lang'}:
raise Failure('Profiles accept only origin, username and lang; never store a password here.')
if not isinstance(spec.get('username', ''), str) or any(unicodedata.category(c).startswith('C') for c in spec.get('username', '')):
raise Failure('Invalid configured username.')
normalized[name] = dict(spec, origin=origin(spec.get('origin', '')), lang=language(spec.get('lang', 'und')))
if profiles and default not in profiles: raise Failure('Default profile does not exist.')
return {'version': 1, 'default_profile': default, 'profiles': normalized}
def configure(words, root=None):
parser = argparse.ArgumentParser(prog='pages configure', description='Manage connections and defaults. No network requests are made.')
sub = parser.add_subparsers(dest='action', required=True)
p = sub.add_parser('set', help='Create or update a connection profile')
p.add_argument('--profile'); p.add_argument('--origin'); p.add_argument('--username'); p.add_argument('--lang')
p = sub.add_parser('use', help='Select the default profile'); p.add_argument('profile')
sub.add_parser('show', help='Show settings, never session tokens')
sub.add_parser('list', help='List configured connections')
if not words: words = ['show']
elif words[0].startswith('--') and words[0] != '--help': words = ['set'] + words
args = parser.parse_args(words)
with Configuration(root).locked() as config:
data = config.validated()
if args.action == 'show':
print(json.dumps(data, ensure_ascii=False, indent=2)); return
if args.action == 'list':
for name, spec in data['profiles'].items():
print(safe_text(('★ ' if name == data['default_profile'] else ' ') + name + ' ' + spec['origin'] + ' ' + spec.get('username', '') + ' [' + spec.get('lang', 'und') + ']'))
if not data['profiles']: print('No connections. Use: pages configure --origin https://wiki.example --username NAME')
return
if args.action == 'use':
name = profile_name(args.profile)
if name not in data['profiles']: raise Failure('Configure this profile before selecting it.')
data['default_profile'] = name
else:
name = profile_name(args.profile or data['default_profile'])
spec = dict(data['profiles'].get(name, {'lang': 'und'}))
if args.origin is not None: spec['origin'] = origin(args.origin)
if args.username is not None: spec['username'] = args.username.strip()
if args.lang is not None: spec['lang'] = language(args.lang)
if not spec.get('origin'): raise Failure('A new connection requires --origin https://HOST.')
# Configuration and session locks use this order. Never retarget an
# active credential to a different Domain or account.
with Profile(name, root).locked() as session:
if session.data.get('token') and (spec['origin'] != session.data.get('origin') or spec.get('username', '') != session.data.get('username')):
raise Failure('Log out this profile before changing its origin or username.')
data['profiles'][name] = spec
if len(data['profiles']) == 1: data['default_profile'] = name
config.data = data
config.validated()
config.save()
print('Configured:', name); return
config.data = data; config.save()
print('Default profile:', data['default_profile'])
def editor(client, addr, mode):
state = client.snapshot(addr)
markdown = mode == 'markdown' or (mode == 'auto' and state['content_type'].startswith('text/html'))
initial = state['markdown'] if markdown else state['body']
directory = Path(tempfile.mkdtemp(prefix='pageswiki-edit-'))
file = directory / ('document.md' if markdown else 'document.txt')
file.touch(mode=0o600)
file.write_text(initial, encoding='utf-8')
success = False
try:
command = shlex.split(os.environ.get('EDITOR', 'vi'))
if not command or subprocess.run(command + [str(file)], check=False).returncode:
raise Failure('Editor exited unsuccessfully.')
text = file.read_text(encoding='utf-8')
if text == initial:
print('Unchanged.')
else:
try:
result = client.save(state, text, markdown)
except Failure as error:
if error.status == 409:
latest = client.snapshot('uuid:' + state['object_uuid'])
remote = latest['markdown'] if markdown else latest['body']
print(safe_text(''.join(difflib.unified_diff(remote.splitlines(True), text.splitlines(True), fromfile='remote', tofile='local'))))
raise
print('Saved revision:', safe_text(result['revision_uuid']))
success = True
finally:
if success:
file.unlink()
directory.rmdir()
else:
metadata = directory / 'recovery.json'
metadata.touch(mode=0o600)
metadata.write_text(json.dumps({'origin': client.base, 'address': 'uuid:' + state['object_uuid'],
'base_revision_uuid': state['head_revision_uuid'], 'lang': state['lang'], 'format': 'markdown' if markdown else 'raw'}))
print('Local edit retained:', safe_text(file), file=sys.stderr)
def execute(client, profile, words):
parser = argparse.ArgumentParser(prog='pages')
sub = parser.add_subparsers(dest='command', required=True)
for command in ('ls', 'cd', 'cat', 'view', 'edit', 'open'):
p = sub.add_parser(command)
p.add_argument('path', nargs='?', default='.')
if command == 'edit':
p.add_argument('--format', choices=['auto', 'raw', 'markdown'], default='auto')
p = sub.add_parser('get'); p.add_argument('path'); p.add_argument('local')
p = sub.add_parser('put'); p.add_argument('local'); p.add_argument('path'); p.add_argument('--base', required=True)
p.add_argument('--format', choices=['raw', 'markdown'], default='raw')
p = sub.add_parser('search'); p.add_argument('query')
sub.add_parser('pwd'); sub.add_parser('logout'); sub.add_parser('status')
args = parser.parse_args(words)
cwd = profile.data.get('cwd', '/')
cmd = args.command
if cmd == 'logout':
client.json('/_/auth/logout', method='POST')
profile.data.pop('token', None); profile.data.pop('expires', None); profile.save()
print('Session revoked.'); return
if cmd == 'pwd':
print(safe_text(cwd)); return
if cmd == 'status':
print(safe_text(client.base + '\n' + profile.data.get('username', '') + '\nPath: ' + cwd))
print('Language:', safe_text(client.lang))
print('Expires:', time.strftime('%Y-%m-%d %H:%M:%S %Z', time.localtime(profile.data.get('expires', 0))))
return
if cmd == 'search':
result = client.json('/_/api/v1/search/suggest?' + url.urlencode({'q': args.query}))
for item in result.get('items', []):
print(safe_text(item['path'] + '\t' + item['label']))
return
addr = address(args.path, cwd)
if cmd in ('ls', 'cd'):
if not addr.startswith('path:'): raise Failure('ls/cd require a Path.')
path = addr[5:]
result = client.listing(path)
if cmd == 'cd':
profile.data['cwd'] = path; profile.save()
else:
for item in result['entries']:
print(safe_text(item['name'] + ('/' if item['kind'] == 'folder' else '')))
if result.get('truncated'): print('Listing truncated; narrow the Path.', file=sys.stderr)
elif cmd == 'open':
route = addr[5:] if addr.startswith('path:') else '/o/' + addr if addr.startswith('uuid:') else '/r/view/' + addr[4:]
webbrowser.open(client.base + url.quote(route, safe='/:'), new=2)
elif cmd == 'edit':
editor(client, addr, args.format)
elif cmd == 'put':
state = client.snapshot(addr)
if state['head_revision_uuid'] != args.base: raise Failure('Conflict: supplied base is not the current language head.', 409)
text = Path(args.local).read_text(encoding='utf-8')
print('Revision:', safe_text(client.save(state, text, args.format == 'markdown')['revision_uuid']))
else:
raw, headers = client.raw(addr)
if cmd == 'get':
fd = os.open(args.local, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(fd, 'wb') as out: out.write(raw)
print('Revision:', safe_text(headers.get('X-PagesWiki-Revision-UUID', 'unknown')))
elif cmd == 'cat' and not sys.stdout.isatty():
sys.stdout.buffer.write(raw)
else:
if not (headers.get_content_type().startswith('text/') or headers.get_content_type() in ('application/json', 'application/xml')):
raise Failure('Binary document: use get or open.')
text = safe_text(raw.decode('utf-8'))
if cmd == 'view' and sys.stdout.isatty():
subprocess.run(shlex.split(os.environ.get('PAGER', 'less')) , input=text.encode(), check=False)
else: print(text, end='')
def shell(client, profile):
import readline
print(safe_text('pages · ' + client.base + ' · ' + profile.data.get('username', '') + ' · ' + client.lang))
# History remains in memory, never on disk; login/password commands are not supported here.
readline.set_history_length(200)
readline.set_completer_delims(' \t\n')
matches = []
def complete(text, index):
if index == 0:
matches.clear()
try:
p = semantic(text, profile.data.get('cwd', '/'))
parent = p if text.endswith('/') else posixpath.dirname(p)
prefix = '' if text.endswith('/') else posixpath.basename(p)
for item in client.listing(parent, prefix)['entries']:
if any(unicodedata.category(c).startswith('C') for c in item['path']):
continue
candidate = item['path'] + ('/' if item['kind'] == 'folder' else '')
if not text.startswith('/'): candidate = posixpath.relpath(candidate, profile.data.get('cwd', '/')) + ('/' if item['kind'] == 'folder' else '')
matches.append(shlex.quote(candidate))
except Failure:
pass
return matches[index] if index < len(matches) else None
readline.set_completer(complete)
readline.parse_and_bind('tab: complete')
while True:
try:
line = input('pages[' + profile.name + ']:' + safe_text(profile.data.get('cwd', '/')) + '> ')
words = shlex.split(line)
if not words: continue
if words[0] in ('exit', 'quit'): break
if words[0] == 'login': raise Failure('Exit shell and use the login command.')
if words[0] != 'logout' and profile.data.get('expires', 0) <= time.time(): raise Failure('Session expired; log in again.')
execute(client, profile, words)
if words[0] == 'logout': break
except (Failure, ValueError, OSError) as e: print(safe_text(str(e)), file=sys.stderr)
except SystemExit: pass
except KeyboardInterrupt: print()
except EOFError: break
def main(argv=None):
parser = argparse.ArgumentParser(prog='pages', description=__doc__, epilog='Commands: configure, login, logout, status, shell, ls, cd, pwd, cat, view, get, put, edit, search, open. Use pages COMMAND --help.')
parser.add_argument('--profile', help='Use another configured connection for this invocation')
parser.add_argument('--lang', help='Override the configured exact language for this invocation')
parser.add_argument('command', nargs='?', default='shell')
parser.add_argument('args', nargs=argparse.REMAINDER)
args = parser.parse_args(argv)
if args.command == 'configure':
if args.profile or args.lang: parser.error('Use pages configure set --profile NAME --lang TAG.')
configure(args.args); return
if args.command == 'help':
parser.print_help(); return
if args.command == 'login':
login = argparse.ArgumentParser(prog='pages login', description='Log into the configured connection. Password and TOTP are prompted privately.')
login.parse_args(args.args)
elif '--help' in args.args:
execute(None, None, [args.command] + args.args); return
with contextlib.ExitStack() as stack:
with Configuration().locked() as config:
settings = config.validated()
name = profile_name(args.profile or settings['default_profile'])
spec = settings['profiles'].get(name)
if spec is None:
raise Failure('No connection configured. Run: pages configure --origin https://HOST --username NAME')
lang = language(args.lang or spec.get('lang', 'und'))
# Hold the profile lock before releasing the configuration lock.
profile = stack.enter_context(Profile(name).locked())
if args.command == 'login':
if not spec.get('username'): raise Failure('Set your account with pages configure --username NAME.')
if profile.data.get('token'): raise Failure('Log out this profile before replacing its session.')
client = Client(spec['origin'], lang=lang)
expires = client.login(spec['username'], getpass.getpass('Password: '))
same_identity = profile.data.get('origin') == client.base and profile.data.get('username') == spec['username']
cwd = profile.data.get('cwd', '/') if same_identity else '/'
profile.data = {'origin': client.base, 'username': spec['username'], 'token': client.token, 'expires': expires, 'cwd': cwd}
try:
profile.save()
except OSError:
# Do not intentionally leave a usable session behind when its
# local credential cannot be persisted. Expiry is the fallback
# if the revocation request itself cannot reach the server.
try: client.json('/_/auth/logout', method='POST')
except Failure: pass
raise Failure('Could not save credentials; attempted to revoke the new session.') from None
print('Logged in. Session saved privately.'); return
if not profile.data.get('token'): raise Failure('Log in first.')
if args.command != 'logout' and (profile.data.get('origin') != spec['origin'] or profile.data.get('username') != spec.get('username')):
raise Failure('Session and configured identity differ. Log out before reconnecting.')
if args.command not in ('logout', 'status') and profile.data.get('expires', 0) <= time.time(): raise Failure('Session expired; log out and log in again.')
client = Client(profile.data['origin'], profile.data['token'], lang)
if args.command == 'shell': shell(client, profile)
else: execute(client, profile, [args.command] + args.args)
if __name__ == '__main__':
try:
main()
except (Failure, OSError, ValueError, KeyError, KeyboardInterrupt) as e:
print(safe_text(str(e)) or 'Interrupted.', file=sys.stderr)
sys.exit(1)
+264
View File
@@ -0,0 +1,264 @@
import contextlib
from email.message import Message
import hashlib
import io
import json
import os
from pathlib import Path
import runpy
import stat
import sys
import tempfile
import threading
import time
import unittest
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from unittest.mock import patch
m = runpy.run_path(str(Path(__file__).with_name('pages')))
Client, Failure, Profile = m['Client'], m['Failure'], m['Profile']
@contextlib.contextmanager
def server(callback):
class Handler(BaseHTTPRequestHandler):
def do_GET(self): callback(self)
def do_POST(self): callback(self)
def log_message(self, *args): pass
instance = ThreadingHTTPServer(('127.0.0.1', 0), Handler)
thread = threading.Thread(target=instance.serve_forever, daemon=True)
thread.start()
try: yield 'http://127.0.0.1:%d' % instance.server_port
finally: instance.shutdown(); instance.server_close(); thread.join()
def reply(h, status, body, mime='application/json'):
raw = json.dumps(body).encode() if isinstance(body, dict) else body
h.send_response(status); h.send_header('Content-Type', mime); h.end_headers(); h.wfile.write(raw)
class RemoteTests(unittest.TestCase):
def configure(self, root, *args):
with contextlib.redirect_stdout(io.StringIO()) as output:
m['configure'](list(args), root)
return output.getvalue()
def test_configuration_defaults_and_switch(self):
with tempfile.TemporaryDirectory() as root:
self.configure(root,'--origin','https://one.example','--username','anya')
self.configure(root,'set','--profile','work','--origin','https://two.example','--username','worker','--lang','ja')
self.configure(root,'use','work')
settings=json.loads(self.configure(root,'show'))
self.assertEqual(settings['default_profile'],'work')
self.assertEqual(settings['profiles']['default']['lang'],'und')
self.assertEqual(settings['profiles']['work']['lang'],'ja')
self.configure(root,'--lang','en-US')
settings=json.loads(self.configure(root,'show'))
self.assertEqual(settings['profiles']['work']['lang'],'en-US')
with m['Configuration'](root).locked() as cfg:
self.assertEqual(stat.S_IMODE(cfg.path.stat().st_mode),0o600)
with self.assertRaises(Failure): self.configure(root,'use','missing')
with self.assertRaises(Failure): self.configure(root,'--lang','ja_JP')
def test_active_credentials_cannot_be_retargeted_or_shown(self):
with tempfile.TemporaryDirectory() as root:
self.configure(root,'--origin','https://one.example','--username','anya')
with Profile('default',root).locked() as profile:
profile.data={'origin':'https://one.example','username':'anya','token':'private-token'}; profile.save()
for args in [('--origin','https://other.example'),('--username','other')]:
with self.assertRaises(Failure): self.configure(root,*args)
self.assertNotIn('private-token',self.configure(root,'show'))
self.assertNotIn('private-token',self.configure(root,'list'))
self.configure(root,'--lang','ja')
with Profile('default',root).locked():
with self.assertRaises(Failure): self.configure(root,'--lang','en')
with Profile('default',root).locked() as profile:
profile.data.pop('token'); profile.save()
self.configure(root,'--origin','https://other.example')
def test_main_uses_saved_defaults_and_rejects_manual_retarget(self):
with tempfile.TemporaryDirectory() as root, patch.dict(os.environ,{'XDG_CONFIG_HOME':root}):
self.configure(root,'set','--profile','work','--origin','https://work.example','--username','worker','--lang','ja')
with Profile('work',root).locked() as profile:
profile.data={'origin':'https://work.example','username':'worker','token':'session-token','expires':time.time()+100,'cwd':'/notes'}; profile.save()
# Patch the globals used by main rather than running a real request.
globals=m['main'].__globals__
with patch.dict(globals,{'execute':lambda client,profile,args: self.assertEqual((client.base,client.lang,args),('https://work.example','ja',['ls']))}):
m['main'](['ls'])
with patch.dict(globals,{'execute':lambda client,profile,args: self.assertEqual(client.lang,'en')}):
m['main'](['--lang','en','ls'])
with m['Configuration'](root).locked() as cfg:
cfg.data['profiles']['work']['origin']='https://evil.example'; cfg.save()
with self.assertRaises(Failure): m['main'](['ls'])
def test_bad_configuration_is_not_printed(self):
with tempfile.TemporaryDirectory() as root:
with m['Configuration'](root).locked() as cfg:
cfg.data={'version':1,'default_profile':'default','profiles':{'default':{'origin':'https://example.test','token':'secret'}}}; cfg.save()
with self.assertRaises(Failure): self.configure(root,'show')
for invalid in [None,42,[],{}]:
with self.assertRaises(Failure): m['profile_name'](invalid)
with self.assertRaises(Failure): m['language'](invalid)
with self.assertRaises(Failure): m['origin'](invalid)
def test_origin_and_paths(self):
for value in ['http://example.com', 'https://user:pass@example.com', 'https://example.com/a', 'https://example.com?token=x']:
with self.assertRaises(Failure): m['origin'](value)
self.assertEqual(m['address']('../日本語 #?%', '/docs/deep'), 'path:/docs/日本語 #?%')
self.assertEqual(m['semantic']('../../..', '/'), '/')
for value in ['https://evil.test', 'asset:x:y', 'rev:bad', '/bad\x1b[31m']:
with self.assertRaises(Failure): m['address'](value)
self.assertNotIn('\x1b', m['safe_text']('\x1b]52;clipboard\x07'))
self.assertNotIn('\u202e', m['safe_text']('abc\u202e'))
def test_redirect_never_receives_bearer(self):
seen = []
with server(lambda h: (seen.append(h.headers.get('Authorization')), reply(h, 200, {}))) as sink:
def redirect(h):
h.send_response(302); h.send_header('Location', sink); h.end_headers()
with server(redirect) as base:
with self.assertRaises(Failure) as error: Client(base, 'secret').json('/test')
self.assertEqual(error.exception.status, 302)
self.assertEqual(seen, [])
def test_protocol_failures_and_limits(self):
for status, body, mime in [(200,b'<html>login</html>','text/html'), (200,b'not-json','application/json'), (403,b'private internal error','text/plain')]:
with server(lambda h: reply(h,status,body,mime)) as base:
with self.assertRaises(Failure) as error: Client(base).json('/test')
self.assertNotIn('private internal', str(error.exception))
with server(lambda h: reply(h,200,b'x'*100)) as base:
with self.assertRaises(Failure): Client(base).request('/large',limit=10)
def test_password_totp_contract(self):
requests = []
def handle(h):
requests.append((h.path, h.rfile.read(int(h.headers['Content-Length'])).decode()))
if h.path.endswith('/login'): reply(h,401,{'mfa_required':True,'challenge_token':'short-lived'})
else: reply(h,200,{'access_token':'a'*32,'token_type':'Bearer','expires_in':3600})
with server(handle) as base:
client = Client(base)
client.login('someone','secret',lambda prompt:'123456')
self.assertEqual(client.token,'a'*32)
self.assertIn('challenge_token=short-lived',requests[1][1])
self.assertNotIn('password',requests[1][1])
def test_private_profile_lock_and_symlinks(self):
with tempfile.TemporaryDirectory() as root:
with Profile('work',root).locked() as profile:
profile.data={'token':'secret'}; profile.save()
self.assertEqual(stat.S_IMODE(profile.path.stat().st_mode),0o600)
self.assertEqual(stat.S_IMODE(profile.root.stat().st_mode),0o700)
with self.assertRaises(Failure):
with Profile('work',root).locked(): pass
with Profile('work',root).locked() as loaded: self.assertEqual(loaded.data['token'],'secret')
profile.path.chmod(0o644)
with self.assertRaises(Failure):
with Profile('work',root).locked(): pass
profile.path.unlink(); profile.path.symlink_to(Path(root)/'missing')
with self.assertRaises(Failure):
with Profile('work',root).locked(): pass
def test_cas_and_authoring_contract(self):
state={'object_uuid':'object','head_revision_uuid':'base','principal_uuid':'person','lang':'ja',
'content_type':'text/html','filename':'a.html','body':'old'}
client=Client('https://example.test')
calls=[]
def api(route, **kwargs):
calls.append((route,kwargs['data']))
return {'html':'<p>new</p>'} if route.endswith('/markdown') else {'revision_uuid':'new'}
with patch.object(client,'json',side_effect=api): client.save(state,'new',True)
self.assertTrue(calls[0][1]['authoring']); self.assertFalse(calls[0][1]['projection'])
self.assertEqual(calls[1][1]['base_revision_uuid'],'base')
self.assertEqual(calls[1][1]['editor_principal_uuid'],'person')
self.assertEqual(calls[1][1]['lang'],'ja')
with patch.object(client,'json',side_effect=Failure('conflict',409)):
with self.assertRaises(Failure): client.save(state,'new')
def test_download_private_and_no_clobber(self):
headers=Message(); headers['X-PagesWiki-Revision-UUID']='rev'
with tempfile.TemporaryDirectory() as root:
profile=Profile('test',root)
client=Client('https://example.test')
file=str(Path(root)/'download')
with patch.object(client,'raw',return_value=(b'data',headers)), contextlib.redirect_stdout(io.StringIO()):
m['execute'](client,profile,['get','/path',file])
self.assertEqual(stat.S_IMODE(Path(file).stat().st_mode),0o600)
with self.assertRaises(FileExistsError): m['execute'](client,profile,['get','/path',file])
def test_editor_conflict_retains_private_recovery(self):
state={'object_uuid':'object','head_revision_uuid':'base','principal_uuid':'person','lang':'und',
'content_type':'text/plain','filename':'x.txt','body':'old','markdown':'old'}
client=Client('https://example.test','secret-token')
with tempfile.TemporaryDirectory() as root:
directory=Path(root)/'working'; directory.mkdir(mode=0o700)
def edit(args, **kwargs):
Path(args[-1]).write_text('my edit')
class Result: returncode=0
return Result()
with patch('tempfile.mkdtemp',return_value=str(directory)), patch('subprocess.run',side_effect=edit), patch.object(client,'snapshot',return_value=state), patch.object(client,'save',side_effect=Failure('conflict',409)), contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
with self.assertRaises(Failure): m['editor'](client,'path:/x','auto')
self.assertEqual((directory/'document.txt').read_text(),'my edit')
recovery=(directory/'recovery.json').read_text()
self.assertNotIn('secret-token',recovery)
self.assertIn('base',recovery)
self.assertEqual(stat.S_IMODE((directory/'recovery.json').stat().st_mode),0o600)
def integration():
fixture=json.load(sys.stdin)
# Exercise the public entry point with persisted defaults, not only Client.
with tempfile.TemporaryDirectory() as root, patch.dict(os.environ,{'XDG_CONFIG_HOME':root}), contextlib.redirect_stdout(io.StringIO()) as output:
m['main'](['configure','--origin',fixture['origin'],'--username','admin'])
with patch('getpass.getpass',return_value='a long test password'): m['main'](['login'])
with Profile('default',root).locked() as session: session_token=session.data['token']
m['main'](['ls'])
m['main'](['status'])
def local_editor(args, **kwargs):
file=Path(args[-1]); file.write_text(file.read_text()+'\nConfigured CLI edit\n')
class Result: returncode=0
return Result()
with patch('subprocess.run',side_effect=local_editor): m['main'](['edit','/editor-fixture'])
check=Client(fixture['origin'],session_token)
assert 'Configured CLI edit' in check.snapshot('path:/editor-fixture')['body']
m['main'](['logout'])
assert session_token not in output.getvalue()
with Profile('default',root).locked() as session: assert 'token' not in session.data
client=Client(fixture['origin'])
client.login('admin','a long test password')
addr='path:/editor-fixture'
state=client.snapshot(addr)
assert state['lang']=='und' and state['principal_uuid']
missing=Client(fixture['origin'], client.token, 'ja')
try: missing.snapshot(addr)
except Failure: pass
else: raise AssertionError('missing language silently fell back')
assert client.listing('/')['entries']
assert client.save(state,'# Remote\n\n{{children}}\n',True)['revision_uuid'] != state['head_revision_uuid']
latest=client.snapshot(addr)
assert '{{children}}' in latest['markdown'] and '<h1' in latest['body']
try: client.save(state,'stale edit')
except Failure as e: assert e.status==409
else: raise AssertionError('stale save accepted')
raw,headers=client.raw(addr)
assert hashlib.sha256(raw).hexdigest()==headers['X-PagesWiki-Document-SHA256']
assert client.raw('rev:'+latest['revision_uuid'])[0]==raw
try: client.snapshot('rev:'+latest['revision_uuid'])
except Failure: pass
else: raise AssertionError('immutable edit accepted')
with tempfile.TemporaryDirectory() as root:
profile=Profile('integration',root)
profile.data={'cwd':'/'}
file=Path(root)/'edit.txt'; file.write_text('local replacement')
with contextlib.redirect_stdout(io.StringIO()):
m['execute'](client,profile,['put',str(file),addr,'--base',latest['head_revision_uuid']])
assert client.snapshot(addr)['body']=='local replacement'
client.json('/_/auth/logout',method='POST')
try: client.snapshot(addr)
except Failure as e: assert e.status in (401,403,303)
else: raise AssertionError('revoked session accepted')
print('Remote login/list/read/authoring/CAS/put/logout integration passed')
if __name__=='__main__':
if sys.argv[1:] == ['--integration']: integration()
else: unittest.main()