Documentation site: mdBook user + contributor guide (2.2)

An mdBook under docs/ rendering eight pages: introduction,
installation, first-run/config, downloading, an anti-bot guide that
captures the hard-won cookies/curl_cffi/POT/player-client knowledge,
troubleshooting (the nine error classes + non-download issues),
architecture (the two-front-ends/one-engine design, settings flow,
layout invariant, persistence), and packaging.

- .forgejo/workflows/docs.yml builds the book and publishes docs/book/
  to the `pages` branch for Codeberg Pages on docs/ changes.
- docs/book/ is gitignored (rendered output).
- README: fix the stale backup.directory comment — every platform nests
  under the one library root, not a channels/ sibling split.
- Add CLAUDE.md (repo guidance for Claude Code).

Builds clean with mdbook 0.5; intra-doc anchor links verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Luna 2026-06-07 02:56:14 -07:00
parent 7906d7d07f
commit e06c2ef826
14 changed files with 833 additions and 1 deletions

108
docs/src/architecture.md Normal file
View file

@ -0,0 +1,108 @@
# Architecture
For contributors. The repo's `CLAUDE.md` is the terse version of this;
read both.
## Two front-ends, one engine
`main.rs` dispatches: `--web``web::run()` (axum, blocks forever);
otherwise `app::App` (eframe/egui desktop GUI). **Both share
`downloader::Downloader`**, the single source of truth for the yt-dlp job
lifecycle.
`Downloader` is **not** async — it spawns an OS thread per yt-dlp process,
streams stdout/stderr back over an `mpsc` channel into each `Job`'s log
buffer, and the caller pumps `Downloader::poll()` regularly (the egui
frame loop and a web background task both do). Anything you add to
`Downloader` is automatically available to both UIs.
`poll()` also drives the cross-cutting job machinery: auto-retry of
transient failures (cooldown + adaptive throttle), the hang watchdog, and
the post-download ffmpeg transcode pass. These work by capturing specs
onto the `Job` at `start()` time (`RetrySpec`, `ConvertSpec`) and acting
on them when the job changes state.
## The settings flow (the easy thing to get wrong)
Almost every configurable feature has the same five-touchpoint shape —
miss one and it silently half-works:
1. `config.rs` — a field/section + its `Default` + the `default_with_dir`
constructor.
2. `download_options.rs` — an `Option<…>` per-channel override (None =
defer to global).
3. `downloader.rs` — a resolver merging global config + per-channel
override into yt-dlp/ffmpeg args, plus a `pub` field on `Downloader`
holding the global default.
4. **Both** UIs render the global setting *and* the per-channel override:
desktop in `app.rs`, web in `web_ui/index.html` **and** `web.rs`'s
`SettingsPayload` (GET reads config, POST writes config + pushes onto
the live `Downloader`).
5. Seed the `Downloader` field at construction **and** on settings-save,
in **both** `app.rs` and `web.rs`.
`subtitle_defaults`, `youtube_player_clients`, and `convert_defaults` are
complete worked examples — grep one end-to-end before adding a setting.
## Filesystem layout
`platform::platform_root(channels_root, platform)` =
`channels_root.join(dir_name)`. **All** platforms (including YouTube,
whose `dir_name` is `channels`) nest under the one configured
`backup.directory`. `.source-url` sidecars in each creator folder let
re-checks recover the exact URL. Library scanning (`library.rs`) is
parallel and consults a `(path, mtime)` SQLite cache to skip re-parsing
unchanged `info.json` sidecars.
## Persistence
`database.rs` wraps an r2d2 SQLite pool. `Database` is cheaply `Clone`
(the pool is an `Arc`), so the parallel scanner takes its own handle.
Schema lives in `init_schema()`; new columns are added via idempotent
`ALTER TABLE … ADD COLUMN` that swallows the duplicate-column error (no
migration framework). The web UI keeps library/notes snapshots in memory;
mutating endpoints mirror DB writes onto those caches and bump a version
counter (the `/api/library` ETag) so reads stay consistent without a
rescan.
The long-lived `WebState` mutexes are accessed via
`util::LockExt::lock_recover()`, which recovers a poisoned lock instead of
cascading one handler's panic into a dead server.
## Web UI is one embedded file
`web_ui/index.html` is the entire SPA (HTML+CSS+JS), `include_str!`-baked
into the binary at compile time — editing it requires a rebuild. Served
`Cache-Control: no-store` so binary upgrades don't strand stale tabs.
Progress streams over `/ws/progress` (WebSocket) with an HTTP-poll
fallback.
## Anti-bot subsystems
`ytdlp_bin.rs` manages the optional self-contained venv at
`~/.local/share/yt-offline/` (nightly `yt-dlp[default]` + `curl_cffi` +
bundled `deno`). `pot_provider.rs` runs `bgutil-pot` for Proof-of-Origin
tokens — its yt-dlp plugin must come from the same release as the server
binary. `error_class.rs` pattern-matches yt-dlp stderr into actionable
classes (order matters in `classify()`: the captcha "Video unavailable"
wall is RateLimited, not NotFound).
## Tests
- Unit tests are inline `#[cfg(test)]` modules (parsers, resolvers, the
error classifier, DB merge logic).
- `tests/api.rs` spawns the **real** `--web` binary against a scratch dir
and drives the HTTP API with curl — genuine end-to-end coverage of the
axum + SQLite + config stack.
`cargo test` runs both. `.forgejo/workflows/test.yml` runs them on every
push.
## Platform support
Tray (`ksni`) and file dialogs (`rfd` xdg-portal) are Linux-only / no-GTK
by design — that's why packaging avoids a GTK dependency. Windows/macOS
aren't first-class yet: the tray needs a per-OS backend before a clean
cross-build. The rest (eframe/wgpu, axum, rusqlite-bundled) already
compiles cross-platform, and `ytdlp_bin` already has `cfg!(windows)`
branches.