All four questions answered: Q1-3 RISKY (credible paths), Q4 PROVEN. WebView-
BotGuard POT (YTDLnis-proven) collapses the JS-runtime problem; HLahwani/yt-
dlp-android solves Q1+Q2+Q3 together. One open question remains: does WebView-
POT recover anti-bot WITHOUT curl_cffi (broken on Android)? The Stage-1
prototype tests exactly that. Fallback: client-to-server app against existing web
API. Update ROADMAP 3.2 with pointer to findings.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Real build-proof run locally: vtt.rs cross-compiles to aarch64 + x86_64 Android
.so (NDK r26d, exported C-ABI symbol). cargo-ndk + uniffi recommended; ~5 days
to reuse 5-6 pure modules. On-device run-proof blocked by sandbox emulator
reaping, downgraded to symbol-export check per plan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The pivotal gate does NOT block: POT is required on mobile, but WebView-based
BotGuard token generation (proven in YTDLnis) is the durable path and collapses
the JS-runtime + POT problems into one, killing the Deno/Node sidecar.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
External JS runtime now a hard yt-dlp dependency (Python jsinterp deprecated
for YouTube 2025.11.12); QuickJS-ng via NDK or HLahwani/yt-dlp-android is the
path. Also fixes a markdown line starting with '#'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Six tasks (Q1–Q4 research + emulator repros, plus scaffold and go/no-go
synthesis) against the approved spec. Research-spike shape: investigate →
reproduce-where-cheap → record evidence → verdict, not TDD.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First sub-project of the Android client: a research spike to decide go/no-go
on a standalone on-device download engine (yt-dlp + JS runtime + POT) before
building any UI. Four research questions, "reproduce where cheap" on the
emulator.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the old search modal with a keyboard-driven command palette: blurred
backdrop, centered input, results grouped by channel with highlighted FTS
snippets (char(2)/char(3) → <mark>), ↑↓ navigation, Enter to open, Esc to
close, recent searches + quick actions (Downloads/Stats/Health/Shortcuts) in
localStorage. Debounced queries to /api/search with a sequence guard so stale
responses don't clobber newer ones. Upload date is looked up client-side from
the loaded library (SearchHit has no date field). Opened by the 🔍 header
button and the `f` hotkey.
Verified: node --check passes, release builds clean, and a headless harness
screenshot confirms grouped results, highlighted matches, and selection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Codeberg repo was renamed yt-offline → catacomb. Update repository/homepage
(Cargo.toml), url + git source (PKGBUILD), and the Windows-zip README Source
line (package.sh) to the new path. The PKGBUILD `$pkgname::` checkout-dir prefix
is no longer needed now that the repo basename matches pkgname.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Prose + commands updated to the new name: brand reads "Catacomb", while
binary/path/package references are the lowercase `catacomb` (commands, deb/rpm
names, ~/.local/share/catacomb, catacomb.db, catacomb.desktop). Codeberg repo
URLs left pointing at the existing repo. Also folds in the pending doc edits
from earlier in the session.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full product rename with a one-time data migration so existing installs keep
their library + bundled toolchain:
- Crate/binary: `yt-offline` → `catacomb` (Cargo.toml, deb/rpm assets, PKGBUILD,
package.sh, the desktop file → catacomb.desktop, launch.json, release CI,
tests/api.rs CARGO_BIN_EXE_catacomb). Codeberg repo URLs left as-is (real
addresses); PKGBUILD pins the checkout dir via `$pkgname::`.
- Data paths: DB `yt-offline.db` → `catacomb.db`, venv `~/.local/share/yt-offline`
→ `~/.local/share/catacomb`. `migrate_legacy_paths()` in main.rs adopts the old
names on first run (renames DB + WAL/SHM sidecars + the venv dir; best-effort,
no-op once migrated). Crash log + restore-temp + IPC socket names follow suit.
- Display: window/app title, tray, web page titles, feed titles, login + SPA
wordmarks → "Catacomb". Env override `YT_OFFLINE_RENDERER` → `CATACOMB_RENDERER`.
Verified: builds clean as target/release/catacomb, 128 unit + 11 integration
tests pass, and a scratch-dir run confirms the DB (+wal) and venv are migrated
with bytes preserved.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The login page was the last surface still on the old flat style. Give it the
same identity as the rest of the UI: serif wordmark with the glowing crimson
"recording" dot, an italic "your private archive" tagline, accent aurora +
film-grain atmosphere on a gradient card, a themed focus ring and accent
button, and a soft entrance. Login JS is unchanged; honours reduced-motion.
Standalone page, so the display font is a system serif stack (no CDN, no
duplicating the SPA's embedded woff2).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the browser's native <video controls> (used for direct /files/
playback) with one custom player for both direct and transcode streams, so
the chrome matches the reskin and gains features it never had. All seeking
still routes through playerSeek/effTime, so the transcode reload-at-offset
path is unchanged.
- Custom scrubber: accent played fill + buffered fill + chapter tick markers
+ a hover time tooltip; click/drag to seek (pointer events, works in both
modes — commits the seek on release so transcode only reloads once).
- Playback speed popover (0.5–2×) and persistent volume + speed + captions
across videos (localStorage).
- Captions (CC) toggle for overlaid <track> subtitles (off by default; the
searchable 📄 transcript pane is unchanged), PiP, fullscreen.
- Auto-hiding controls + cursor after idle while playing; a center play/pause
flash; gradient scrim.
- Expanded keyboard: space/k, j/l ±10, ←/→ ±5, ↑/↓ volume, m, c, p, f,
< > speed, 0–9 seek-to-percent — now active for direct videos too (they
previously relied on native controls).
Verified: node --check passes, release builds clean, live 8081 serves it, and
a headless screenshot of the control chrome (extracted CSS) renders correctly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sessions were an in-memory HashMap seeded empty at startup, so every restart
or upgrade invalidated every browser's cookie (the cause of the recent
"settings save → error" report: a restart out from under a logged-in tab).
Mirror sessions to a new `sessions(token, issued_at)` table:
- store issued-at as a UNIX timestamp (the map switches from Instant→u64 so it
can round-trip the DB; is_authed prunes by wall-clock age vs SESSION_TTL),
- insert on login, delete on logout, clear on password change,
- rehydrate the map at startup via load_sessions(), which also prunes rows
past the TTL. DB writes are best-effort — a failure just means that token
won't survive a restart, not that login breaks.
Verified end-to-end: set a password, log in, restart the server against the
same DB, and the original cookie still authenticates (200) while a cookieless
request is still 401. 128 unit + 11 integration tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sessions are in-memory, so a server restart (or a session TTL expiry) leaves
a still-open SPA tab holding a stale cookie: cached views keep rendering, but
every mutating call 401s and surfaced only as an "authentication required"
toast (e.g. saving Settings) — a confusing dead end.
Make api() treat 401 as "session gone": flash a notice and location.reload(),
which the server answers with the login page. No reload loop (the login page
isn't the SPA), and authed sessions are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reskin the Library-health modal into a systems-diagnostics readout, sibling
to the stats observatory and sharing its sx-* section language. Presentation
only — every id/class/handler the logic depends on (dup-chk, sim-chk, at-chk,
dedup-area, autotag-area, the async polling) is untouched.
- A pulsing health verdict ("Healthy" / "N items to review") whose count and
status colour update live as the async subsystems resolve.
- Four status tiles (Duplicate IDs, Missing assets, Similar content, Unfiled
groups) with health-coloured dots + top rules; sim/at start pending and are
filled by their render functions.
- Sections become instrument panels: duplicate/similar groups as cards with
KEEP/REMOVE pills, missing-asset rows, a gradient dedup progress bar, and
auto-tag groups with confidence dots. Modal widened to 920px.
- Status colour flows through --mx-c off a data-status attribute, scoped to
.mx-body so it can't leak onto other nodes; pulse honours reduced-motion.
Verified: JS passes node --check, release builds clean, live 8081 serves it,
and a headless screenshot of a harness running the extracted CSS+JS against a
mock report confirms the verdict math, tiles, pills and panels all render.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rebuild the Library statistics modal from plain tiles + flat bars + tables
into a real data dashboard, driven by the same /api/stats payload and the
theme variables (so all 10 themes inherit it).
- Metric cards with giant serif numbers that count up from zero on open
(size/runtime spin up through their units), an accent top-rule, and a
glowing hero tile.
- A self-drawing SVG area chart for downloads-per-week: Catmull-Rom-smoothed
line that animates in via stroke-dashoffset, a gradient area fill,
gridlines, a date axis, and hover dots with tooltips.
- A growing-column histogram for videos-by-upload-year with value labels.
- A ranked channel leaderboard with animated meter fills, gold/silver/bronze
ranks, and a Size/Count segmented toggle that re-animates on switch.
- Reveal animations gate behind .sx-go (added post-layout) so they play once
per open, not on re-render; honours prefers-reduced-motion; modal widened
to 1040px.
Verified: JS passes node --check, release builds clean, the live 8081 server
serves it, and a headless screenshot of a harness running the *extracted*
dashboard CSS+JS against a mock payload confirms the chart math, count-up,
histogram and leaderboard all render correctly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A presentational redesign of the web SPA, appended as a design layer so it
overrides by source order while every colour still flows through the existing
7 per-theme CSS variables — so all 10 themes inherit it and no element
id/class the JS relies on changes.
- Typography: embed Instrument Serif (display) + Hanken Grotesk (body) as
base64 woff2 (SIL OFL) — offline-safe, no CDN, no privacy leak. Serif
wordmark/headers/empty-state; clean grotesque for UI.
- Identity: a glowing crimson "recording" dot before the wordmark; a soft
accent aurora + fine SVG film-grain fixed behind content (negative z, so
it never sits under text).
- Depth & motion: translucent blurred masthead with a hairline accent
underglow; cinematic card hover (lift + accent ring + thumbnail zoom +
shadow); one-time staggered shell reveal; themed focus rings; thin themed
scrollbars; restrained button hover (no full crimson flood). Honours
prefers-reduced-motion.
Verified: JS still passes node --check, release builds clean, and a headless
chromium screenshot of the live 8081 server confirms it renders (fonts +
atmosphere + masthead) without breaking the existing layout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a `mac` target that cross-compiles an Apple binary via osxcross
(MAC_ARCH=arm64 default, or x86_64), assembles an unsigned yt-offline.app
(Info.plist + Mach-O + best-effort .icns via png2icns), and zips it. The
crypto stack is ring, which builds against the osxcross SDK cleanly.
- Sets the per-target linker + cc-rs CC/CXX/AR env from the osxcross
wrappers (oa64-clang / o64-clang); discovers the versioned ar triple.
- Gated on the toolchain being present: `all` folds it in only when an
Apple rust target + an osxcross wrapper + zip exist, and a bare `mac`
run skips cleanly (exit 0) otherwise — verified.
- Local-only, not in CI: the macOS SDK can't be hosted in a public image.
- Refresh the stale Windows + macOS sections of docs/PACKAGING.md (they
still described the pre-cfg-gating blocked state) and the CI section;
note a possible future MacPorts port. ROADMAP 3.1 updated.
Untested end-to-end (no osxcross/SDK on this box); the scaffolding + skip
path are verified and `all` on a stock box is unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The tree already links a working x86_64-pc-windows-gnu exe; turn that into
a shipped artifact:
- scripts/package.sh gains a `win` target (and `all` picks it up when the
mingw toolchain is present) that cross-compiles and zips the exe +
LICENSE + a README listing the yt-dlp/ffmpeg/mpv PATH deps.
- Forgejo release workflow installs mingw-w64 + zip + the rust target, so
a tag push produces yt-offline-<ver>-x86_64-windows.zip from the same
Linux container — no Windows runner needed.
- main.rs: attach_windows_console() (cfg(windows), windows-sys
AttachConsole) reattaches a release build to the launching terminal so
--web/CLI output is visible, while a double-click stays windowless
(release sets windows_subsystem = "windows").
- ROADMAP 3.1 updated: Windows ships; macOS deferred (needs a Mac runner).
Native Linux build + 128 unit / 11 integration tests still green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add "download date" sorting (file mtime, distinct from upload date) plus a
broader set of options. Web select is now grouped via <optgroup>; desktop
gains SortMode::DownloadDesc/DownloadAsc/ChannelAsc with matching arms and
toolbar entries.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The startup channel scan used available_parallelism() (every core), which
could briefly peg the whole box on a cold/large library. Cap it at cores-1
so the machine stays responsive; the scan is disk-I/O-bound and runs as a
background task, so one fewer thread costs little. Mirrors the headroom the
fingerprint pass and thumbnail pool already leave.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Patch progress/speed in place on each tick instead of rebuilding the whole
modal body, so open per-job logs no longer collapse and the bar doesn't
flicker while downloading. Full rebuild only on a structural change
(job added/removed, state or retryability changed, queue length changed).
- "↻ Retry all failed" header button that re-issues every retryable failed
job (re-snapshots between retries since indices shift).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Download management UI (web + desktop):
- Cancel a running job (SIGKILL, marked "cancelled" so it isn't auto-retried
and shows a distinct state rather than a misleading error class)
- Cancel a still-queued job before it starts
- Manually retry a failed/cancelled job (fresh auto-retry budget)
- Live speed · ETA · % on running jobs, parsed from the yt-dlp progress line
- Expandable full per-job log fetched on demand
New endpoints: POST /api/jobs/:idx/{cancel,retry}, GET /api/jobs/:idx/log,
DELETE /api/queued/:idx.
Perceptual-dedup off-switch:
- backup.dedup_enabled (default true) hard-disables the "similar content"
scan in both UIs; the web scan endpoint returns {disabled:true} and the
desktop scan no-ops with a note. Wired through all five settings touchpoints.
Docs: README seeking note + ROADMAP recently-shipped updates for the prior
seekable-playback / federation / auto-tagging work.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Transcoded videos stream through a live ffmpeg pipe with no byte ranges, so
the browser couldn't seek/rewind them at all. Fix it end to end:
- get_transcode accepts ?start=<secs> and passes -ss before -i (fast keyframe
seek), rebasing timestamps to 0 from there.
- The player gets a custom control bar for transcode sources: play/pause, a
scrubber driven by the known duration, current/total time, ±10s, mute +
volume, and fullscreen, plus space / arrows / j / l / f keyboard shortcuts.
Seeking re-requests the stream at the new offset; effective time is
offset + element.currentTime. Every seek path (scrubber, chapters,
transcript click, resume, save-position, chapter/transcript highlight) now
routes through one offset-aware playerSeek()/effTime().
- Direct /files/ videos keep native <video controls> (already seekable).
Verified server-side: start=0 vs start=120 yield different streams.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Configure peers as [[remote]] entries (name, url, optional password) and
browse their libraries from this instance, read-only.
- remote.rs: RemoteClient (blocking reqwest + rustls + cookie jar). Fetches
the peer's /api/library, logging in first if the peer has a password and
retrying on a 401. Media is not proxied: the library JSON's media URLs
(/files/, /music-files/, /api/transcode/, /api/sub-vtt/) are rewritten to
absolute peer URLs with the peer's read-only feed token appended, so video
streams straight from the peer to the browser/mpv while only the small JSON
passes through us.
- web.rs: auth_middleware now also accepts the feed token for GET
/api/transcode/ and /api/sub-vtt/ (read-only media, same class as /files/),
so a transcoded peer streams remotely. New GET /api/remotes and
/api/remotes/:id/library (the RemoteClient runs under spawn_blocking).
- web UI: a 🌐 Remotes sidebar switcher; remote mode is read-only (hides the
download bar + per-card mutating actions, keeps Play).
- desktop: a 🌐 Remotes screen that fetches a peer's library off-thread and
plays via mpv on the tokenized URL.
- config.rs: [[remote]] section list; reqwest dep added (rustls/blocking/
cookies, no system OpenSSL).
Verified end-to-end against a second local instance, including a transcoded
stream returning HTTP 200 via the token. ROADMAP 3.5 + CLAUDE.md updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`cargo check --target x86_64-pc-windows-gnu` is now green (only the
upstream egui f32:From<f64> warnings). The Linux-only desktop deps are
target-gated so the rest of the stack — which already cross-compiles —
can build off Linux:
- Cargo.toml: ksni and rfd's xdg-portal backend move to
[target.'cfg(target_os = "linux")'.dependencies]; non-Linux gets rfd
with its native Win32/AppKit backend (default features).
- tray.rs: cfg-split. The ksni/SNI implementation stays Linux-only; other
OSes get a no-op `start() -> None`, i.e. windowed-only (identical to the
Linux no-SNI-host path). TrayEvent/TrayHandle stay cross-platform.
- plex.rs: add a Windows `symlink_file` arm to make_symlink (also fixes an
unused-variable warning on non-unix).
The rest was already portable (disk_space/statvfs, mpv IPC UnixStream,
chmod/PermissionsExt guards all cfg(unix)). macOS is Unix so those apply
there unchanged. Still not a *shipped* binary: needs a real per-OS tray
backend and a linking CI matrix. ROADMAP 3.1 + CLAUDE.md updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New autotag.rs classifies each unfiled channel from already-scanned
metadata — source platform + median video duration + upload cadence —
into a suggested folder group (Music / Shorts / Long-form & Podcasts /
Streams & VODs), with a confidence and a human-readable reason. Mid-length
YouTube is left unsuggested rather than guessed at; channels already in a
folder are skipped. Pure arithmetic over the in-memory library, computed
on demand (no background job).
Surfaced in both Maintenance views:
- web: GET /api/autotag/suggest + POST /api/autotag/apply (create/reuse the
named folder and assign), rendered with per-channel checkboxes and an
"Apply -> group" button that re-analyzes after applying.
- desktop: a section listing each group with a "Move all -> group" button.
Apply mirrors post_assign_folder: DB write + in-memory library update +
ETag bump. Reuses an existing folder of the same name (case-insensitive).
Unit tests cover the classifier; verified the apply/revert round-trip live.
ROADMAP 3.4 marked DONE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Several related UI and startup fixes from one session:
- Web grid: channel and music views rendered in a single column because
their content was wrapped in one nested div that became a single cell of
the outer CSS grid. Span the wrappers (and .empty) across all columns
with grid-column:1/-1.
- Subtitles: route every track through /api/sub-vtt and strip per-cue
position/alignment settings (align:start position:0% etc.) from both SRT
and VTT, so auto-generated captions render centered instead of left-
aligned. New strip_cue_settings/normalize_vtt + sub_vtt_url helpers.
- Comment downloads: add a global backup.fetch_comments toggle (the
per-channel option already existed; it's now a tri-state override that
defers to the global). Full five-touchpoint wiring across config,
download_options, the downloader's apply_comments resolver, both UIs'
global + per-channel controls, and downloader seeding.
- Desktop startup: run the initial library scan + search-index sync on a
background thread (mirrors the web server's deferred bind) so the window
appears immediately instead of blocking on a cold-cache scan; update()
swaps the library in when it lands.
- Perceptual dedup: cap worker threads via fingerprint::default_workers()
(~half the cores, always leaving at least one free) instead of using
every core, so the hashing pass doesn't starve the UI.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
serve() ran a full library scan + search-index rebuild before binding the
HTTP listener. On a large library with a cold cache (e.g. on a slow mount)
that scan is disk-bound for minutes, during which nothing listens on the
port and the server looks dead on the network (notably over Tailscale).
Build WebState with an empty library, bind + print the URL immediately,
then run the scan in a spawn_blocking task that swaps the result in and
bumps the library version (same pattern as post_rescan). The server is now
reachable in ~1s; content appears once the first scan lands and clients
pick it up via the bumped version/ETag.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The desktop forces the wgpu/Vulkan renderer because glow/OpenGL crashes
on NVIDIA + Wayland maximize. But on some systems wgpu creates the window
and never presents a frame, so nothing appears. Keep wgpu the default but
compile glow back in and let it be selected via `--renderer glow` or
`YT_OFFLINE_RENDERER=glow` for those machines.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Subscribe to the whole library or a single channel in any podcast/media
app.
- src/feed.rs: pure RSS 2.0 + iTunes-namespace renderer with absolute
enclosure URLs, MIME-by-extension, HH:MM:SS durations, and a
chrono-free RFC-2822 date (civil-date algorithm). Unit-tested
(mime / escape / duration / date / render).
- web.rs: GET /feed.xml (whole library, newest first, capped at 300) and
GET /feed/:platform/:handle (one channel); /api/feed-info returns the
token for the UI. Enclosure + thumbnail URLs are absolute (derived from
Host + X-Forwarded-Proto) and carry the feed token.
- Auth: a stable read-only `feed_token` (persisted setting) lets a
tokenized `?token=` GET reach /feed*, /files, and /music-files even when
a password is set — podcast clients can't do the cookie login. Scoped to
reads of feeds + media; never /api mutations.
- Web UI: a "📡 Podcast feed" section in Settings (library URL + Copy) and
a "📡 Feed URL" button in each channel's options dialog.
Integration tests cover the served RSS (enclosure path, MIME, pubDate),
the channel feed + 404, and the full token gate (401 without token once a
password is set, 200 with it, /files reachable with it). 127 tests pass.
Desktop URL display follows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the guide up to date with this session's work: the hang watchdog in
poll(); the three (path,mtime)-keyed caches (info_cache, search index,
fingerprints); the FTS5 drop+recreate migration exception to the
ALTER-only rule; a Search & perceptual-dedup subsystem section
(fingerprint.rs, vtt.rs, the shared rebuild_and_group pipeline, the
background-job pattern); tests/api.rs integration tests + the opt-in
real-ffmpeg test; the node --check tip for the embedded SPA; and
sponsorblock_mode / fingerprint / vtt in the worked-example + module
lists.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document the work since the last README pass: full-text library search
(titles/descriptions/transcripts via FTS5), searchable transcript pane +
desktop transcript window, threaded comment viewer, configurable
SponsorBlock, per-channel overrides, post-download format conversion, the
anti-bot/POT stack, auto-retry + adaptive throttle, perceptual
'similar content' dedup, a Reliability section (hang watchdog, disk-full
preflight, error classifier, crash log, backup/restore), folders/notes/
bulk-tagging, desktop UI zoom, and a link to the docs site. Config example
gains sponsorblock_mode / POT / player-clients / ui_scale.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 🔍 library search now matches spoken words, not just titles /
channels / descriptions.
- database.rs: add a `transcript` column to the `video_search` FTS5 index.
Fresh DBs get it directly; existing ones are migrated (FTS5 has no ADD
COLUMN, so the table is recreated and search_meta cleared to force a
one-time reindex). `sync_search_index` reads each video's first subtitle
(mtime-gated like the description), flattens it via the shared `vtt`
parser (`transcript_text`, capped at 256 KB), and indexes it. Search
snippets now use column -1 so the excerpt comes from whichever column
matched (description or transcript).
- library.rs: `build_search_entries` passes the first subtitle path.
- Both UIs: search copy updated to mention transcripts.
Tests: a unit test (a word only in the .vtt is found), a migration test
(seed an old 5-column index + stale meta, open, confirm the recreated
index indexes transcript text), and the search integration test now seeds
a subtitle and asserts a spoken-only word hits. Also bumped the test
server's startup budget so the ffmpeg-heavy dedup test can't starve a
sibling server into a flaky timeout. 120 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extracts the dedup pipeline (mtime-gate -> parallel fingerprint -> prune
-> group) into `fingerprint::rebuild_and_group`, returning groups of file
paths. Both front-ends now call it and map paths to their own review
rows — web's run_dedup is refactored onto it (no behaviour change; the
integration test still passes).
Desktop: the 🩺 Maintenance screen gains a "Similar content (perceptual)"
section — a Scan button that runs the fingerprint job on a background
thread (live "N / total" progress via shared atomics + an mpsc result
channel), grouped results with a recommended-keep marker, and a per-group
"Remove non-recommended copies" that deletes via the existing
remove_files path and re-scans. Closes the desktop/web parity gap for
content-aware dedup.
118 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wires the fingerprint engine into the web Maintenance view.
- web.rs: a DedupState job (one at a time) on its own thread. POST
/api/maintenance/dedup/scan snapshots the library, mtime-gates against
stored fingerprints, fingerprints the new/changed videos in parallel
(progress counter), prunes vanished entries, groups by visual
similarity, and builds review rows (title/channel/size/files + a
recommended-keep = largest copy). GET /api/maintenance/dedup/status
polls progress + results. Deletion reuses /api/maintenance/remove.
- index.html: a "Similar content (perceptual)" section in the Maintenance
modal — Scan button, live progress bar, grouped results with
checkboxes (recommended-keep pre-unchecked), and a re-scan that drops
deleted copies. Poller self-cancels when the modal closes.
Integration test (ffmpeg-gated): generates orig + CRF-38 downscaled
re-encode + an unrelated clip, runs the real scan end-to-end, and asserts
the first two group while the third stays out. 118 tests pass.
Desktop Maintenance UI next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The core of content-aware duplicate detection — finding the same video
under a different ID (reupload / mirror / re-encode), which the ID-based
maintenance scan can't.
- src/fingerprint.rs: samples 6 frames per video via ffmpeg keyframe seek
(fast — never full-decodes), downscales to a 9x8 grayscale grid, and
dHashes each to a 64-bit frame hash. `group_similar` clusters videos
whose frame hashes match within a Hamming threshold, comparing only
within a duration-tolerance window (sorted sliding window + union-find)
so it stays near-linear instead of O(n²). Parallel `compute_batch` over
a hand-rolled scoped thread pool (no rayon), with a progress counter.
- src/database.rs: `video_fingerprint` table keyed by (path, mtime) like
info_cache — hash once, reuse forever; new downloads hashed as they
arrive. Store / load / mtime-gate / prune helpers.
Unit tests cover dHash, Hamming, duration-bucketed + transitive grouping,
and DB round-trip/replace/prune. An opt-in `real_ffmpeg_groups_reencodes`
test (run with --ignored) generates a video + a downscaled CRF-38
re-encode + an unrelated clip and asserts the first two group and the
third doesn't — measured ~0.5s/video serial (≈65ms/video across 8 cores).
UI wiring (background build + review screen in both front-ends) follows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Surface the downloaded .vtt subtitles as a searchable, click-to-seek
transcript — a thing Tartube doesn't do at all.
Web (pure frontend): a 📄 button in the player opens a transcript pane
beside the video. It fetches the already-served .vtt, parses cues
client-side, and offers full-text search (highlighted), click-to-seek,
and a live highlight + auto-scroll of the current line as the video
plays. Reuses the chapters-pane layout; hidden on narrow screens.
Desktop (egui): a 📄 Transcript button on the detail panel opens a
floating window that reads the .vtt off disk (new `vtt` parser module),
shows the searchable cue list, and seeks the running mpv via its
JSON-IPC socket when you click a line.
New `src/vtt.rs`: a small, tolerant WebVTT/SRT cue parser (start time +
tag-stripped text, consecutive-duplicate collapse) with unit tests.
Closes the desktop/web parity gap for transcripts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A new FTS5 index (`video_search`) over every video's title, channel, and
description — searchable across the whole library, not just the loaded
grid. `search_meta` tracks each video's mtime so a routine rescan only
re-reads a description sidecar when the video actually changed; vanished
videos are evicted. The index is refreshed after every scan in both
front-ends via the shared `library::build_search_entries`.
- database.rs: schema + `sync_search_index` (mtime-gated, one txn) +
`search_videos` (ranked, with a highlighted snippet) + a safe prefix
MATCH builder. Unit-tested (index/search/prefix/AND/evict/garbage).
- web.rs: `GET /api/search?q=&limit=`; index synced after the initial
scan, rescan, and maintenance-remove.
- Web UI: a 🔍 header button + `f` shortcut open a debounced search modal
with ranked results and highlighted description snippets; clicking a
result jumps to it.
- Desktop (egui): a 🔎 Search button opens a floating results window
querying the same index; clicking a result plays the video. Closes the
desktop/web parity gap for search.
Integration test seeds a real video + description, rescans, and asserts
title / prefix / description-only matches hit and unrelated misses.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment viewer (web UI): the flat tree gains search (highlight + keep
ancestor context), sort (threaded / top / newest / oldest), per-thread
collapse/expand with collapse-all/expand-all, an OP badge for the
uploader, and a "new since last visit" highlight + count (localStorage
per-video timestamp). API now returns each comment's `timestamp` and
`author_is_uploader`.
SponsorBlock: was a hardcoded `--sponsorblock-mark all`. Now a real
setting (off / mark / remove) with a per-channel override, threaded
through the full five touchpoints — config `[backup].sponsorblock_mode`
(default "mark", preserving prior behavior), DownloadOptions override,
a downloader resolver (apply_sponsorblock), and both UIs (egui Settings
+ channel dialog; web Settings modal + channel dialog + SettingsPayload).
Integration tests extended: settings round-trip asserts the sponsorblock
default + persistence; channel-options round-trip asserts the override.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The .forgejo/workflows/* definitions don't execute on Codeberg (no
runner). Docs publish via scripts/publish-docs.sh; tests/packages run
locally. Fixes claims introduced earlier today.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codeberg doesn't execute Forgejo Actions (no runners), so the docs.yml
workflow never ran. Replace it with scripts/publish-docs.sh: build the
mdBook and force-push docs/book/ to the `pages` branch that Codeberg
Pages serves. Installs mdbook on demand, derives the Pages URL from the
origin remote, and honors CODEBERG_TOKEN for non-interactive auth.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An mdBook under docs/ rendering eight pages: introduction,
installation, first-run/config, downloading, an anti-bot guide that
captures the hard-won cookies/curl_cffi/POT/player-client knowledge,
troubleshooting (the nine error classes + non-download issues),
architecture (the two-front-ends/one-engine design, settings flow,
layout invariant, persistence), and packaging.
- .forgejo/workflows/docs.yml builds the book and publishes docs/book/
to the `pages` branch for Codeberg Pages on docs/ changes.
- docs/book/ is gitignored (rendered output).
- README: fix the stale backup.directory comment — every platform nests
under the one library root, not a channels/ sibling split.
- Add CLAUDE.md (repo guidance for Claude Code).
Builds clean with mdbook 0.5; intra-doc anchor links verified.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Unit tests cover parsers/resolvers but not whether the axum handlers,
SQLite layer, and config persistence actually wire together. New
tests/api.rs spawns the compiled binary in --web mode against a scratch
library dir and exercises the real HTTP API the way a browser does —
no network / yt-dlp needed (every endpoint here is local state).
Coverage: index + /api/library served; ETag conditional GET → 304;
settings round-trip (POST → GET reflects → config.toml on disk updated);
folder CRUD + the self-parent cycle guard (→ 400); notes upsert/delete;
channel-options store + is_empty() clear; backup/db returns a real
SQLite file.
HTTP via curl (transparently handles the server's gzip + chunked
encoding; already a CI dependency) with a graceful skip if curl is
absent. Each test gets its own server, port (bind :0 to grab a free
one), and tempdir, so they run in parallel and clean up on drop.
Added .forgejo/workflows/test.yml so `cargo test` (unit + integration)
runs on every push/PR — the release workflow only fired on tags. 105
tests pass (98 unit + 7 integration).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
