Bring the guide up to date with this session's work: the hang watchdog in
poll(); the three (path,mtime)-keyed caches (info_cache, search index,
fingerprints); the FTS5 drop+recreate migration exception to the
ALTER-only rule; a Search & perceptual-dedup subsystem section
(fingerprint.rs, vtt.rs, the shared rebuild_and_group pipeline, the
background-job pattern); tests/api.rs integration tests + the opt-in
real-ffmpeg test; the node --check tip for the embedded SPA; and
sponsorblock_mode / fingerprint / vtt in the worked-example + module
lists.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document the work since the last README pass: full-text library search
(titles/descriptions/transcripts via FTS5), searchable transcript pane +
desktop transcript window, threaded comment viewer, configurable
SponsorBlock, per-channel overrides, post-download format conversion, the
anti-bot/POT stack, auto-retry + adaptive throttle, perceptual
'similar content' dedup, a Reliability section (hang watchdog, disk-full
preflight, error classifier, crash log, backup/restore), folders/notes/
bulk-tagging, desktop UI zoom, and a link to the docs site. Config example
gains sponsorblock_mode / POT / player-clients / ui_scale.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 🔍 library search now matches spoken words, not just titles /
channels / descriptions.
- database.rs: add a `transcript` column to the `video_search` FTS5 index.
Fresh DBs get it directly; existing ones are migrated (FTS5 has no ADD
COLUMN, so the table is recreated and search_meta cleared to force a
one-time reindex). `sync_search_index` reads each video's first subtitle
(mtime-gated like the description), flattens it via the shared `vtt`
parser (`transcript_text`, capped at 256 KB), and indexes it. Search
snippets now use column -1 so the excerpt comes from whichever column
matched (description or transcript).
- library.rs: `build_search_entries` passes the first subtitle path.
- Both UIs: search copy updated to mention transcripts.
Tests: a unit test (a word only in the .vtt is found), a migration test
(seed an old 5-column index + stale meta, open, confirm the recreated
index indexes transcript text), and the search integration test now seeds
a subtitle and asserts a spoken-only word hits. Also bumped the test
server's startup budget so the ffmpeg-heavy dedup test can't starve a
sibling server into a flaky timeout. 120 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extracts the dedup pipeline (mtime-gate -> parallel fingerprint -> prune
-> group) into `fingerprint::rebuild_and_group`, returning groups of file
paths. Both front-ends now call it and map paths to their own review
rows — web's run_dedup is refactored onto it (no behaviour change; the
integration test still passes).
Desktop: the 🩺 Maintenance screen gains a "Similar content (perceptual)"
section — a Scan button that runs the fingerprint job on a background
thread (live "N / total" progress via shared atomics + an mpsc result
channel), grouped results with a recommended-keep marker, and a per-group
"Remove non-recommended copies" that deletes via the existing
remove_files path and re-scans. Closes the desktop/web parity gap for
content-aware dedup.
118 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wires the fingerprint engine into the web Maintenance view.
- web.rs: a DedupState job (one at a time) on its own thread. POST
/api/maintenance/dedup/scan snapshots the library, mtime-gates against
stored fingerprints, fingerprints the new/changed videos in parallel
(progress counter), prunes vanished entries, groups by visual
similarity, and builds review rows (title/channel/size/files + a
recommended-keep = largest copy). GET /api/maintenance/dedup/status
polls progress + results. Deletion reuses /api/maintenance/remove.
- index.html: a "Similar content (perceptual)" section in the Maintenance
modal — Scan button, live progress bar, grouped results with
checkboxes (recommended-keep pre-unchecked), and a re-scan that drops
deleted copies. Poller self-cancels when the modal closes.
Integration test (ffmpeg-gated): generates orig + CRF-38 downscaled
re-encode + an unrelated clip, runs the real scan end-to-end, and asserts
the first two group while the third stays out. 118 tests pass.
Desktop Maintenance UI next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The core of content-aware duplicate detection — finding the same video
under a different ID (reupload / mirror / re-encode), which the ID-based
maintenance scan can't.
- src/fingerprint.rs: samples 6 frames per video via ffmpeg keyframe seek
(fast — never full-decodes), downscales to a 9x8 grayscale grid, and
dHashes each to a 64-bit frame hash. `group_similar` clusters videos
whose frame hashes match within a Hamming threshold, comparing only
within a duration-tolerance window (sorted sliding window + union-find)
so it stays near-linear instead of O(n²). Parallel `compute_batch` over
a hand-rolled scoped thread pool (no rayon), with a progress counter.
- src/database.rs: `video_fingerprint` table keyed by (path, mtime) like
info_cache — hash once, reuse forever; new downloads hashed as they
arrive. Store / load / mtime-gate / prune helpers.
Unit tests cover dHash, Hamming, duration-bucketed + transitive grouping,
and DB round-trip/replace/prune. An opt-in `real_ffmpeg_groups_reencodes`
test (run with --ignored) generates a video + a downscaled CRF-38
re-encode + an unrelated clip and asserts the first two group and the
third doesn't — measured ~0.5s/video serial (≈65ms/video across 8 cores).
UI wiring (background build + review screen in both front-ends) follows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Surface the downloaded .vtt subtitles as a searchable, click-to-seek
transcript — a thing Tartube doesn't do at all.
Web (pure frontend): a 📄 button in the player opens a transcript pane
beside the video. It fetches the already-served .vtt, parses cues
client-side, and offers full-text search (highlighted), click-to-seek,
and a live highlight + auto-scroll of the current line as the video
plays. Reuses the chapters-pane layout; hidden on narrow screens.
Desktop (egui): a 📄 Transcript button on the detail panel opens a
floating window that reads the .vtt off disk (new `vtt` parser module),
shows the searchable cue list, and seeks the running mpv via its
JSON-IPC socket when you click a line.
New `src/vtt.rs`: a small, tolerant WebVTT/SRT cue parser (start time +
tag-stripped text, consecutive-duplicate collapse) with unit tests.
Closes the desktop/web parity gap for transcripts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A new FTS5 index (`video_search`) over every video's title, channel, and
description — searchable across the whole library, not just the loaded
grid. `search_meta` tracks each video's mtime so a routine rescan only
re-reads a description sidecar when the video actually changed; vanished
videos are evicted. The index is refreshed after every scan in both
front-ends via the shared `library::build_search_entries`.
- database.rs: schema + `sync_search_index` (mtime-gated, one txn) +
`search_videos` (ranked, with a highlighted snippet) + a safe prefix
MATCH builder. Unit-tested (index/search/prefix/AND/evict/garbage).
- web.rs: `GET /api/search?q=&limit=`; index synced after the initial
scan, rescan, and maintenance-remove.
- Web UI: a 🔍 header button + `f` shortcut open a debounced search modal
with ranked results and highlighted description snippets; clicking a
result jumps to it.
- Desktop (egui): a 🔎 Search button opens a floating results window
querying the same index; clicking a result plays the video. Closes the
desktop/web parity gap for search.
Integration test seeds a real video + description, rescans, and asserts
title / prefix / description-only matches hit and unrelated misses.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment viewer (web UI): the flat tree gains search (highlight + keep
ancestor context), sort (threaded / top / newest / oldest), per-thread
collapse/expand with collapse-all/expand-all, an OP badge for the
uploader, and a "new since last visit" highlight + count (localStorage
per-video timestamp). API now returns each comment's `timestamp` and
`author_is_uploader`.
SponsorBlock: was a hardcoded `--sponsorblock-mark all`. Now a real
setting (off / mark / remove) with a per-channel override, threaded
through the full five touchpoints — config `[backup].sponsorblock_mode`
(default "mark", preserving prior behavior), DownloadOptions override,
a downloader resolver (apply_sponsorblock), and both UIs (egui Settings
+ channel dialog; web Settings modal + channel dialog + SettingsPayload).
Integration tests extended: settings round-trip asserts the sponsorblock
default + persistence; channel-options round-trip asserts the override.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The .forgejo/workflows/* definitions don't execute on Codeberg (no
runner). Docs publish via scripts/publish-docs.sh; tests/packages run
locally. Fixes claims introduced earlier today.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codeberg doesn't execute Forgejo Actions (no runners), so the docs.yml
workflow never ran. Replace it with scripts/publish-docs.sh: build the
mdBook and force-push docs/book/ to the `pages` branch that Codeberg
Pages serves. Installs mdbook on demand, derives the Pages URL from the
origin remote, and honors CODEBERG_TOKEN for non-interactive auth.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An mdBook under docs/ rendering eight pages: introduction,
installation, first-run/config, downloading, an anti-bot guide that
captures the hard-won cookies/curl_cffi/POT/player-client knowledge,
troubleshooting (the nine error classes + non-download issues),
architecture (the two-front-ends/one-engine design, settings flow,
layout invariant, persistence), and packaging.
- .forgejo/workflows/docs.yml builds the book and publishes docs/book/
to the `pages` branch for Codeberg Pages on docs/ changes.
- docs/book/ is gitignored (rendered output).
- README: fix the stale backup.directory comment — every platform nests
under the one library root, not a channels/ sibling split.
- Add CLAUDE.md (repo guidance for Claude Code).
Builds clean with mdbook 0.5; intra-doc anchor links verified.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Unit tests cover parsers/resolvers but not whether the axum handlers,
SQLite layer, and config persistence actually wire together. New
tests/api.rs spawns the compiled binary in --web mode against a scratch
library dir and exercises the real HTTP API the way a browser does —
no network / yt-dlp needed (every endpoint here is local state).
Coverage: index + /api/library served; ETag conditional GET → 304;
settings round-trip (POST → GET reflects → config.toml on disk updated);
folder CRUD + the self-parent cycle guard (→ 400); notes upsert/delete;
channel-options store + is_empty() clear; backup/db returns a real
SQLite file.
HTTP via curl (transparently handles the server's gzip + chunked
encoding; already a CI dependency) with a graceful skip if curl is
absent. Each test gets its own server, port (bind :0 to grab a free
one), and tempdir, so they run in parallel and clean up on drop.
Added .forgejo/workflows/test.yml so `cargo test` (unit + integration)
runs on every push/PR — the release workflow only fired on tags. 105
tests pass (98 unit + 7 integration).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reconcile with what shipped since 2026-06-01: format conversion (1.7),
filter presets (1.3+), hang watchdog + poison-recover locks (2.5),
desktop UI scale.
- State table: format conversion + filter UI → Tied; stability row
expanded (crash log + disk-full + watchdog + lock recovery); added
desktop UI scale row. Score 14 ahead / 1 behind / 13 tied — the lone
"behind" is edge-case maturity, not a feature.
- Phase 1 marked COMPLETE; the two last parity items recorded.
- Phase 2: 2.5 done; only 2.1 (integration tests) + 2.2 (docs) remain.
- Phase 3: unblocked 3.1 (packaging shipped) and reframed it around the
real blocker (Linux-only tray/file-dialog); expanded 3.7 dedup with
the perceptual-hash design.
- Refreshed intro + footer to "at parity, now surpassing."
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A yt-dlp process can stall with the OS process alive but no progress — a
wedged TLS handshake, an unresponsive server, a dead socket mid-fragment.
Before, that job hung forever, holding a concurrency slot until the user
noticed and cancelled.
Now each Job tracks last_activity (bumped on any stdout/stderr line or
progress message). poll() runs check_watchdog(): a Running job silent for
HANG_TIMEOUT (5 min) gets SIGKILLed by pid via libc::kill. The threshold
sits well above any legitimate quiet gap — yt-dlp prints a line per
download chunk, polls every 30s under --wait-for-video, and our longest
sleep is ~30s — so 5 min of total silence is a real stall.
To capture the pid, spawn_job + enqueue_transcode now spawn the child
*before* the reader thread (the thread still owns it for the blocking
wait()). A watchdog kill sets watchdog_killed, and drain() forces the
resulting failure to NetworkError (retryable) instead of the Other that
classify() would return for an output-less kill — so the existing
auto-retry path re-queues it (a hang is transient).
Non-Unix builds keep the flagging but can't force-kill (no portable
pid-kill; Windows isn't first-class). JobState gains Debug for tests.
2 tests: watchdog kill → retryable class; normal failures still
classify from the log. 98 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The web server holds several long-lived Mutexes in WebState (library,
downloader, watched, flags, positions, sessions, …). Every access used
.lock().unwrap(), so a single panic while any one was held would poison
it and turn EVERY subsequent request into a panic — one handler's bug
killing the whole server until restart.
New util::LockExt::lock_recover() acquires the guard but recovers the
inner data via PoisonError::into_inner() on poison. Our critical
sections are short and don't leave half-updated invariants, so the data
stays usable and a stuck request beats a stuck process. Applied to all
70 lock sites in web.rs.
Test poisons a mutex from a panicking thread and confirms lock_recover
still returns usable data. 96 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The chip filters (watch/date/size/subs/chapters) were session-only.
Add saveable named presets, finishing the last small Phase-1 line.
- ★ Save preset button (shown when any filter is active) → names the
current chip set via prompt and stores it.
- Saved presets render as chips in the filter row; click to apply,
the inline ✕ deletes. The preset matching the current filters is
highlighted.
- Persisted under their own `filter-presets` localStorage key so they
survive clearFilters() and aren't entangled with view state.
Handlers are index-based (not name-based) so preset names containing
quotes/apostrophes can't break the inline onclick — the list re-renders
after every change so indices always track the DOM. Same-name save
overwrites rather than duplicating.
Web-only: the desktop UI has no chip filters. Verified the save/apply/
delete/match/overwrite logic in isolation; JS syntax-checked; builds.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The card-density slider only resized the channel-grid cards — the
sidebar, settings, detail panel, and all text stayed fixed. Add a real
global scale that grows/shrinks the *entire* UI crisply.
egui already handles Ctrl + / Ctrl - / Ctrl 0 via zoom_factor (it
re-rasterizes fonts rather than bitmap-stretching), but it wasn't
persisted and had no visible control. Now:
- New `[ui] ui_scale` config (default 1.0), applied with
set_zoom_factor at startup.
- update() mirrors ctx.zoom_factor() back into config each frame and
saves on a 500ms debounce, so both the slider and the built-in
keyboard zoom persist across restarts without writing config.toml
every frame of a drag.
- A "UI scale" slider (0.5–3.0×) + Reset button in the Settings screen,
driving set_zoom_factor live.
The card-density "Size:" slider stays as a separate grid-density
preference (it composes with the global zoom). 95 tests pass; GUI
verified running with the per-frame zoom-sync path.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Post-download ffmpeg pass with three modes, global + per-channel.
Closes the last real Tartube parity gap.
## Modes
- remux-mp4: fast container change mkv→mp4 (stream-copy video, aac audio)
- h264-mp4: re-encode to H.264/AAC at a configurable CRF + x264 preset
- audio: extract to mp3 / m4a / opus / flac
## Design (separate ffmpeg pass)
The main download adds `--print after_move:CONVERT_PATH:%(filepath)s`
when conversion is active. poll() scans freshly-Done download jobs for
those lines and enqueues a distinct ffmpeg transcode Job per finished
file — so the UI shows transcoding as its own phase. On success the
source is renamed to <name>.original.<ext> (keep_original) or deleted;
the converted file lands next to it. Transcodes don't auto-retry and
don't themselves convert (no infinite loop).
## Config
New [convert] section (mode / crf / preset / audio_format /
keep_original) + per-channel DownloadOptions.convert_mode override
(None = defer to global, "off" = force-disable for that channel).
ConvertSpec::resolve merges them; CRF 0→23, empty preset→medium, empty
audio→mp3 defaults.
## UI
Desktop + web: a "Format conversion (global defaults)" Settings section
with mode-dependent rows (CRF/preset for h264, format for audio, keep-
original toggle), and a per-channel Convert dropdown in the channel-
options dialog.
Verified: all three ffmpeg modes produce valid output on a real library
file (remux instant, h264 CRF23 ~12s for a 3.4min clip, mp3 extraction);
settings round-trip (global → config.toml [convert] → echo; per-channel
override saves + reads back). 4 resolver tests; 95 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stale or anonymous cookies make YouTube's bot-detection WORSE than no
cookies — so warn the user before downloads start captcha-failing.
cookies_freshness() parses the Netscape jar and reports:
- expires_at / expired / days_left: earliest expiry among the auth
cookies that actually gate login (SID, SAPISID, __Secure-*PSID,
LOGIN_INFO, …). Non-auth cookies (PREF, consent) are ignored so their
expiry doesn't trigger false warnings.
- no_auth_cookies: the jar has cookies but NONE of the login ones — i.e.
it's an anonymous/visitor session, not signed in. This is the worst
case for bot-detection and a common foot-gun (exporting cookies from a
browser that wasn't logged in to YouTube).
Surfaced in GET /api/cookies and both Settings UIs (web + desktop) with
graded severity: anonymous (red) > expired (red) > expiring-soon
(yellow) > fine.
5 tests cover expired, fresh, session/non-auth ignored, and the
anonymous-jar case. 91 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
YouTube's bot-detection cracks down on different player clients over
time — the `web` client gets captcha-walled, then later `tv`/`mweb` are
the safe ones, etc. Let the user route around whichever client is
currently being targeted instead of being stuck with yt-dlp's pick.
- New backup.youtube_player_clients config (comma-separated, blank =
yt-dlp default). Maps to --extractor-args youtube:player_client=…
- Per-channel DownloadOptions.youtube_player_clients override (None =
defer to global). Resolved in the downloader's apply_player_client,
which omits the flag entirely when empty so yt-dlp keeps its default
client set (the recommended baseline).
- Threaded into the live downloader at construction + settings save
(app + web).
UI: a "YouTube player clients" field in both global Settings (desktop +
web) and the per-channel options dialog, with hints pointing at
"tv,mweb" as the current least-bot-checked combo for captcha-prone
channels.
Verified the settings round-trip (global persists to config + echoes;
per-channel override saves + reads back). 87 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two measures to keep a long channel scan going when YouTube's
bot-detection trips mid-batch.
## Auto-retry on transient failures
When a download fails with a retryable class (RateLimited — which now
includes captcha walls — or NetworkError), the downloader re-queues it
automatically after a cooldown, up to MAX_AUTO_RETRIES (2). Linear
backoff: 1st retry waits 90 s, 2nd waits 180 s. Permanent classes
(NotFound, MembersOnly, Geoblocked, CodecMissing, DiskFull, BadCookies)
are never retried — re-running changes nothing.
Implementation: start() captures a RetrySpec (url/info/flags/quality/
opts) onto the Job; poll() scans freshly-failed jobs, schedules due
retries into a cooldown queue, and fires them by rebuilding the command
through start(). Live recordings and synthetic preflight failures opt
out. A log line announces each scheduled retry so it's visible in the
Downloads panel.
## Adaptive throttle
After any rate-limit hit, rate_limited_backoff engages and the sleep
flags roughly triple for the rest of the batch (--sleep-requests 1→3,
--sleep-interval 2-6 s → 8-20 s), easing off an already-suspicious
endpoint instead of hammering it. Clears once downloads go idle so the
next fresh batch starts at normal speed.
2 new tests (retryable-class matrix, backoff sleep values). 87 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The captcha/bot-challenge error reads:
Video unavailable. YouTube is requiring a captcha challenge before playback
Because it contains "Video unavailable", the NotFound rule grabbed it
and showed the misleading "removed by the uploader — nothing to
download" hint. But the upload isn't gone; it's a bot-detection wall.
Add captcha / "not a bot" fingerprints to the RateLimited rule, which
runs before NotFound so it wins the "Video unavailable" overlap. Also
broadened the RateLimited hint to mention captchas + the POT provider
as remedies alongside cookies + waiting.
New test pins the captcha line → RateLimited; the genuine
"removed by the uploader" line still maps to NotFound. 85 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Completes the subtitle feature started in 0814ba0 (backend + desktop).
## Global settings (web Settings modal)
- New "Subtitles (global defaults)" section: enable toggle that reveals
auto-captions / embed / languages / convert-format controls.
- SettingsPayload gains the 5 subtitle fields; get_settings reads them
from [subtitles] config, post_settings writes them back, saves config,
and pushes the live value onto downloader.subtitle_defaults.
## Per-channel overrides (channel-options dialog)
- Tri-state selects (Default / On / Off) for download-subtitles,
auto-captions, embed; a convert-format text field. "Default" defers
to the global setting.
- triSelect() / triValue() helpers map Option<bool> ⇄ select value.
readChannelOptionsForm includes the four new fields; they serialize
straight onto DownloadOptions via serde (post_channel_options already
takes the whole struct).
## Verified end-to-end (live server)
- Global: GET defaults → POST (auto off, embed on, langs=en, format=srt)
→ echoed back → written to config.toml [subtitles] → persists across
restart.
- Per-channel: POST overrides → read back exactly; all-default body
hits the is_empty() delete path and clears the row.
84 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds full subtitle control (backend + desktop UI; web UI follows).
## Model
- New [subtitles] config section: enabled / auto_generated / embed /
format / langs. Sensible defaults (subs on, auto on, no embed/convert,
all langs).
- DownloadOptions gains per-channel Option overrides: subtitles_enabled,
subtitles_auto, subtitles_embed, subtitle_format (subtitle_langs
already existed). None = defer to global.
## Resolver
- Downloader gains subtitle_defaults (the global section) + a single
apply_subtitle_flags() that merges global + per-channel and emits the
yt-dlp flags (--write-subs / --write-auto-subs / --sub-langs /
--convert-subs / --embed-subs). Disabled = emit nothing.
- Replaces the hardcoded --write-subs --write-auto-subs at the main
download + repair builders. Moved the --sub-langs emission out of
DownloadOptions::apply into the resolver so global + override merge in
one place.
## Desktop UI
- Settings screen: "Subtitles (global defaults)" section.
- Channel-options dialog: tri-state (Default/On/Off) combos for each
override + a convert-format field.
## Wiring
- subtitle_defaults seeded at Downloader construction (app + web) and
refreshed on settings save (desktop).
4 resolver tests cover global defaults, disabled, global format/embed/
langs, and per-channel override-wins. 84 pass.
Web UI rows are the remaining piece (next commit).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Symptom: mid-batch (e.g. item 34 of 40) channel scans failed with
ERROR: [youtube] <id>: Video unavailable. YouTube is requiring a
captcha challenge before playback
Root cause: we hardcoded `--extractor-args youtube:player_client=web`
on every download. That flag dates from when `web` was the
throttle-avoidance client, but YouTube has since made `web` the MOST
aggressively bot-checked client — it's exactly the one that demands a
captcha. Verified directly: forcing player_client=web throws the
captcha error on a specific video that resolves fine without it.
Fix:
- Remove player_client=web from all three download builders (main,
repair, music). Modern yt-dlp auto-selects better clients
(android_vr, tv, web_safari) that aren't captcha-walled, and our POT
provider already serves tokens for them (web_safari in the logs).
- Add a randomized 2-6 s --sleep-interval between videos. The jitter
(vs a fixed cadence) is what keeps a long channel scan from looking
robotic and tripping the wall around request ~30.
Verified: both video IDs from the bug report (4JJ434QQoE4, Gr4bTC7DZWA)
that captcha'd under player_client=web now resolve cleanly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
yt-dlp prints a scary-looking ERROR when it can't confirm a channel
tab loaded authenticated:
ERROR: [youtube:tab] @Channel: Playlists that require authentication
may not extract correctly without a successful webpage download…
pass "--extractor-args youtubetab:skip=authcheck" to skip this check
For PUBLIC channels that's just noise — there's no auth-gated content
to miss, and the flag silences it without changing which videos are
found. But for members-only/private channels archived with cookies,
the warning is a real "your cookies may not be working, you might be
getting an incomplete list" signal worth keeping.
So make it a per-channel toggle rather than a blanket setting:
- DownloadOptions gains `skip_auth_check: bool`; apply() emits
`--extractor-args youtubetab:skip=authcheck` when set. (Its own flag;
the youtubetab: namespace is distinct from the POT provider's
youtubepot-bgutilhttp: one, so they coexist.)
- Desktop + web channel-options dialogs gain a checkbox with a tooltip
spelling out the public-vs-private trade-off.
1 new test; 81 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Maximizing the window crashed the GUI immediately with:
Error: Glutin(Error { raw_code: Some(12291), kind: OutOfMemory })
12291 is EGL_BAD_ALLOC. The default eframe `glow` (OpenGL) renderer
fails to reallocate its surface when the window resizes to maximized
dimensions on NVIDIA + Wayland — a well-known fragility of NVIDIA's
GL-on-Wayland path. It surfaces as a clean Err from run_native (not a
panic), which is why nothing landed in yt-offline.crash.log.
Switch to the wgpu (Vulkan) renderer, which reconfigures the swapchain
cleanly on resize:
- Cargo.toml: drop eframe's default glow feature, enable wgpu, keep
accesskit / default_fonts / wayland / x11.
- main.rs: NativeOptions.renderer = Renderer::Wgpu.
Verified on the NVIDIA GTX 1650 SUPER + Wayland session that triggered
the crash: app launches and maximizes cleanly, no EGL error.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Symptom: with the bundled yt-dlp, every --impersonate target showed
"(unavailable)" and impersonation silently did nothing.
Root cause: it's a yt-dlp ⇄ curl_cffi version-gate mismatch, not a
curl_cffi problem. curl_cffi 0.15.0 was installed and works on its own
(a direct impersonate request returns 200), but stable yt-dlp 2026.3.17
caps support at curl_cffi <0.15:
ImportError: Only curl_cffi versions 0.5.10 and 0.10.x through 0.14.x
are supported
…so yt-dlp refuses to load the impersonate backend and disables every
target. yt-dlp master already raised the gate to <0.16, but that's only
in nightly, not stable.
Fix: install the nightly yt-dlp via `pip install --pre`. Nightly is
yt-dlp's recommended track for keeping up with YouTube's anti-bot
changes anyway (newly relevant now that we ship POT support), and it
accepts current curl_cffi so we don't have to pin a fragile version.
Verified on the bundled venv: nightly 2026.05.25 + curl_cffi 0.15.1b1
lists all impersonate targets, and `--impersonate chrome` resolves a
real video end-to-end. The install script's existing "grep chrome in
--list-impersonate-targets" verification now passes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The original integration pip-installed `bgutil-ytdlp-pot-provider` from
PyPI (Brainicism's upstream, currently 1.3.1). But the Rust server
(jim60105's port) is at 0.8.1, and yt-dlp REFUSES to use a token when
the plugin and server major versions mismatch:
WARNING: Plugin and HTTP server major versions are mismatched.
(plugin: 1.3.1, HTTP server: 0.8.1)
…so the feature silently produced no POT tokens.
jim60105 ships a version-matched plugin zip
(bgutil-ytdlp-pot-provider-rs.zip) in each release. install_command now:
- downloads that zip from the SAME release as the server binary
- uninstalls any stale PyPI plugin + removes old getpot_bgutil*.py
- resolves the venv site-packages via `python -c sysconfig` (version
independent) and unpacks the plugin's yt_dlp_plugins/ tree there
Verified end to end on a clean install: server mints a real BotGuard
token, yt-dlp loads bgutil:http-0.8.1 (matched), no mismatch warning,
and a 4K format URL resolves through the POT-gated path:
[pot:bgutil:http] Generating a gvs PO Token … via bgutil HTTP server
Rick Astley … | 401+251 | 3840x2160
Adds an `unzip` check to the install script (already required for the
bundled deno install).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Folders were flat — one level under each platform. Tartube supports
arbitrary nesting; now we do too.
## Schema
Add `folders.parent_id INTEGER REFERENCES folders(id) ON DELETE CASCADE`
via an idempotent migration (ALTER TABLE, swallow the duplicate-column
error on already-migrated DBs). NULL = top-level. Cascade means deleting
a parent removes its subtree's folder rows; member channels revert to
Unfiled via the existing channel_assignments cascade.
## DB
- FolderRecord gains parent_id (flows to both UIs via serde).
- set_folder_parent(id, parent) reparents, with a cycle guard: walks the
ancestor chain of the proposed parent and refuses if `id` appears
(you can't nest a folder inside its own descendant). Self-parent also
rejected. Returns a friendly SQLITE_CONSTRAINT error the web layer
maps to 400.
## API
POST /api/folders/:id/parent { parent_id } — reparent or null for top.
## Web UI
- Sidebar renders the folder tree recursively: each folder indents one
step under its parent, member channels indent under their folder, and
the count shown is the whole subtree. A seen-set guards against any
malformed cycle in the data.
- Folder manager gains a per-row "parent" dropdown; choices that would
cycle (self + descendants) are excluded.
## Desktop UI
- Channel-panel sidebar builds the same tree with an iterative DFS
(depth-indented folder headers, subtree counts).
- Folder manager gains a parent ComboBox per row with the same
cycle-exclusion.
Verified the migration runs against an existing real DB (parent_id added,
no data loss). 1 new test covers nesting + both cycle cases; 80 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Ships beyond the Arch PKGBUILD so non-Arch users can install without a
Rust toolchain.
## Package metadata (Cargo.toml)
- [package.metadata.deb] for cargo-deb: declares the runtime subprocess
deps (yt-dlp, ffmpeg, mpv, xdg-utils) that auto-detection can't find,
plus libxcb1/libc6 link deps and libnotify4 as recommended.
- [package.metadata.generate-rpm] for cargo-generate-rpm with the
RPM-distro dep names.
- Filled in license / repository / authors / readme package fields.
## scripts/package.sh
One entry point: `scripts/package.sh [deb|rpm|appimage|all]`. Builds the
release binary once and reuses it. Installs cargo-deb / cargo-generate-rpm
on demand; downloads appimagetool to dist/tools/ on first AppImage build.
Per-format failures are isolated and summarized at the end rather than
aborting the run. Output to dist/ (gitignored).
AppImage is hand-rolled (AppDir + AppRun + root desktop/icon) and bundles
only the GUI binary's shared-lib closure — yt-dlp/ffmpeg/mpv stay host
PATH deps, same as the package declarations.
## CI
.forgejo/workflows/release.yml builds all formats on a v* tag push and
attaches them to the Codeberg release via actions/forgejo-release. Manual
workflow_dispatch builds without uploading for smoke testing. Sets
APPIMAGE_EXTRACT_AND_RUN since CI containers lack FUSE.
## docs/PACKAGING.md
Per-format build + install instructions, the Arch PKGBUILD pointer, and
an honest Windows/macOS section: both are blocked on the Linux-only tray
(ksni) + file-picker (rfd xdg-portal) deps needing per-OS abstraction.
The rest of the stack already cross-compiles.
Verified .deb (dpkg control + payload correct) and .rpm (payload correct)
build cleanly through the script on this host.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Free-text annotations on any channel or video, searchable from the
global filter box.
## Storage
New `notes` table keyed by (target_kind, target_id):
- target_kind: "channel" or "video"
- target_id: "platform/handle" for channels, video ID for videos
- empty body deletes the row, so the table only holds real notes
DB methods: get_note, set_note (upsert-or-delete), get_all_notes.
## API
- GET /api/notes → { "video:<id>": body, "channel:<plat>/<handle>": body }
fetched once on page load, held in memory for indicators + search
- POST /api/notes/:kind/:id { body } → upsert/delete
## Web UI
- 📝 button on every video card; turns accent-colored when a note
exists. Opens an edit-in-place modal.
- 📝 Add/Edit note… sub-action under each channel in the sidebar.
- Global search now also matches note bodies, so you can find a clip
by what you wrote about it.
## Desktop UI
- Note field in the video detail panel (bottom dock). Loads lazily
from the DB when the selection changes; persists on focus-loss.
## Restore
Backup import merges the notes table too (keep-later-timestamp,
same as watched/positions). Guarded with a table-exists check so
pre-notes backups still import cleanly. RestoreSummary gains
notes_added; both UIs show it.
4 new tests (2 notes DB behavior, restore merge already covered the
shape). 79 pass total.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
YouTube increasingly requires a per-video Proof-of-Origin token bound
to each video ID before handing back format URLs. Without one, yt-dlp
sees empty formats and downloads fail. The upstream solution is the
Brainicism bgutil-ytdlp-pot-provider Python plugin paired with a
long-running server that mints tokens via BotGuard.
We integrate the Rust port (jim60105/bgutil-ytdlp-pot-provider-rs) so
the server is a single static binary, no Node.js required.
## Architecture
- bgutil-pot binary lives alongside the bundled deno + yt-dlp at
~/.local/share/yt-offline/bin/bgutil-pot.
- The matching Python plugin is pip-installed into the bundled venv:
python -m pip install bgutil-ytdlp-pot-provider.
- Downloader spawns the server lazily on first job (port 4416,
loopback only) and kills it on Drop.
- Each yt-dlp invocation gets
--extractor-args "youtubepot-bgutilhttp:base_url=http://127.0.0.1:4416"
appended in spawn_job when use_pot_provider is on and the server
child is alive.
## UX
- Settings → "POT token provider" row with toggle + Install/Update
button (mirrors the bundled yt-dlp row).
- Disabled unless use_bundled_ytdlp is also on (plugin lives in that
venv).
- Refuses install if the bundled yt-dlp venv isn't there yet, with a
helpful 428 error.
## Lifecycle
- Lazy spawn: ensure_pot_server() runs at the top of start() so the
server is up before yt-dlp queries the plugin.
- Drop: kill_server() on Downloader drop so we don't leave an orphaned
process bound to 4416.
- start_pot_provider_update() kills the running child first so the
install can overwrite the binary in place (no ETXTBSY).
3 new unit tests cover URL/extractor-args formatting; 77 total pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previously platform_root resolved YouTube to channels_root verbatim and
non-YouTube to siblings under channels_root.parent(). That assumed the
user's library was split across two levels:
/library/yt-offline/ ← config.backup.directory (YouTube)
/library/bandcamp/ ← sibling
/library/tiktok/ ← sibling
In practice users (myself included) keep everything under one umbrella:
/library/yt-offline/
channels/ ← YouTube
bandcamp/
tiktok/
music/
…
Make that the canonical layout. platform_root now returns
channels_root.join(dir_name) for every platform including YouTube
(which keeps its `channels` dir_name for back-compat with existing
on-disk trees that already have the channels/ subdir populated).
library_root is now == channels_root in both app.rs and web.rs — the
two-level concept is gone. The field stays as a distinct name so
file_url() and the /files/ static-file mount keep working without
churn at every call site. music_root and music dir also moved from
channels_root.with_file_name("music") to channels_root.join("music").
Tests in platform.rs updated to the new layout. 74 pass.
Note: this is a breaking change for installs that had the sibling
layout. The fix is to mv the platform dirs into the channels_root.
The next commit walks the user through that for my install.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The page-bottom footer holding the URL input + quality picker + flags
was a fixed 50-ish-pixel strip eating screen space on every screen,
even though most of the time users aren't starting a new download.
Move all of it into the ⬇ Downloads modal as a "new download" section
above the active jobs list. Same controls (URL, quality, Fast/Live/
Clips), same element IDs so existing helpers (previewDownload,
fullScan, dlLive, refreshTwitchClipsVisibility, updateDlMode) keep
working unchanged. URL input auto-focuses on modal open so the
paste-and-Enter flow is one motion.
Footer shrinks to just the AGPL §13 source notice (required for
deployment compliance, ~12px tall now).
updateDlMode is hardened with null-checks since the labels only exist
while the modal is open.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four changes that compound — most users will notice these on first
launch of a large library.
## Release profile
- opt-level: 2 → 3 (more inlining + vectorization, ~5-15% on hot paths)
- lto = "thin" (cross-crate inlining; thin avoids the rust-lld + bundled
sqlite link breakage that full LTO previously caused, which is what
the !lto note was about)
- codegen-units = 1 (whole-crate inlining, ~5% runtime, +30s build)
- strip = "debuginfo" (drops ~30 MB from the binary)
## Thumbnail decode pool
The single decoder thread was the bottleneck on grid fill for libraries
with hundreds of channels. Replace with a worker pool sized at
clamp(2, cores/2, 8). Workers share the request channel behind a Mutex;
lock contention is negligible because each worker spends ~all its time
in image::open + resize.
## info.json parse cache
Library scans re-parse every video's info.json sidecar on every rescan —
serde_json::from_str is the dominant cost on a warm-filesystem scan
(file reads themselves are OS-cached). Add a `info_cache` SQLite table
keyed by (path, mtime); enrich_with_cache hits it first and only parses
on miss. mtime-keyed invalidation means yt-dlp re-writing an info.json
auto-invalidates without explicit eviction.
Each scan worker checks out its own r2d2 connection so lookups don't
serialize. Database is now Clone (the inner Pool is Arc, so cloning is
cheap) so we can hand handles to parallel workers.
## /api/library body cache
The ETag short-circuit catches clients with a matching cached version;
this catches clients without one (curl, freshly-opened tabs, mobile
WebViews that don't store ETags reliably). When the version matches
the cached entry's, we hand back the previously-serialized bytes
without re-walking the channel tree or re-running serde_json::to_string.
A post-build version check prevents poisoning the cache with stale
bytes when the library changes during serialization.
74 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two independent improvements from Phase 2.5.
## Panic hook → crash.log
GUI users launched from a .desktop file have no stderr — today panics
vanish without trace. Install a panic::set_hook early in main() that
appends a structured entry to <library_root>/yt-offline.crash.log for
every panic from every thread (UI, axum workers, downloader spawns,
tray bg thread, thumbnail decode workers).
Each entry: ISO-8601 timestamp, thread name, file:line, panic message,
and a backtrace when RUST_BACKTRACE is set. The default panic hook
still runs after ours, so dev workflows keep getting the stderr trace.
Bounded: when the log passes 256 KB the next write rotates it to .1
(one previous generation kept). No external date library — small
civil-time formula in-tree.
## Disk-full preflight
Before yt-dlp ever launches, statvfs the target filesystem. If free
space is under 500 MB, push a synthetic Failed job classified as
DiskFull instead of starting yt-dlp. That way:
- Users see the actual problem ("only 230 MB free on /mnt/…") in the
Downloads modal with the same "free up space and retry" hint the
error-classifier emits for mid-download ENOSPC.
- No half-written file lands on disk.
- No 30-second yt-dlp warmup wasted on a doomed run.
statvfs returning None (non-Unix, missing path) skips the check —
better to let yt-dlp run than to refuse on no data.
5 new tests: 3 in crash (civil-time, log shape, real-panic capture),
2 in disk_space (statvfs plumbing + missing-path None). 74 pass total.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Backup direction has shipped for a while; this adds the import side
so users can actually round-trip their snapshot.
Database::restore_from_backup attaches the uploaded file and merges
each table with per-table semantics:
- watched / positions / channel_options: keep the later timestamp
- video_flags: bitwise-OR each flag (favourite + bookmark across two
devices both stick)
- folders: insert names that don't exist yet
- channel_assignments: insert when (platform, handle) is unassigned;
folder_id is re-resolved by name so backup/live ID drift is fine
- settings: deliberately skipped (password hash + source_url are
per-deployment state, not user data)
Re-importing the same snapshot is a no-op. Schema validation rejects
non-yt-offline SQLite files up front with a 400.
POST /api/restore/db takes raw .db bytes (100 MB cap, magic-header
check) and returns a RestoreSummary with per-table added counts.
After success, the in-memory watched/positions/flags caches are
refreshed and the library version is bumped so cached /api/library
responses revalidate.
Desktop: 📂 Import library backup… button in Settings, mirrors the
existing 💾 save button. Triggers rescan after the merge so channel
folder assignments take effect.
Web UI: 📂 Import library snapshot… button next to the download
button. Hidden file input + confirm dialog + result summary.
Includes 4 new unit tests in src/database.rs covering the watched +
positions merge, flag OR-ing, idempotency on re-run, and schema
rejection.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When a job fails, today the user sees a raw stderr line — often
something like "ERROR: [youtube] dQw4w9WgXcQ: Sign in to confirm
you're not a bot" that doesn't tell a non-expert what to do.
New src/error_class.rs scans the failed job's log buffer for the
nine well-known yt-dlp fingerprints and returns:
- an ErrorClass enum (rate-limited, members-only, geo-blocked,
not-found, codec-missing, disk-full, network-error, bad-cookies,
other)
- a one-line human-readable suggested action paired with each class
Auto-populated in Job::drain when state transitions to Failed.
Exposed on JobSnapshot as error_class + error_hint. Rendered in
the web UI Downloads modal (red badge + hint box) and the desktop
downloads panel (colored label + hint text).
Other (no fingerprint matched) is filtered out of the badge so the
existing raw last_line keeps doing the talking for unknown failures.
Includes 10 unit tests covering each pattern + the walks-from-end
priority case.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Score moved from 8-ahead/8-behind to 11-ahead/4-behind. The four
remaining Tartube-leads items are now Phase 1's entire scope:
- 1.5 per-channel/video notes
- 1.7 format conversion pipeline
- 1.8 per-distro packaging (.deb, .rpm, .exe, .dmg, CI matrix)
- maturity / edge cases (continuous, not a discrete item)
Phase 1 also picks up two extension items for things that shipped at
"v1" depth but where Tartube has more polish:
- 1.3+ filter presets (chips ship, naming/saving doesn't)
- 1.2+ N-level folder nesting (one level ships, deeper doesn't)
Phase 2 2.4 narrowed to just the restore direction since backup ships.
Added a "Recently shipped" section so the doc records what closed
out each line rather than silently dropping it from the table.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Make ui.minimize_to_tray default-off so users on desktops without a
working StatusNotifier host (GNOME sans AppIndicator extension being
the common case) don't get stuck — without an opt-in, clicking X
would hide an invisible window with no obvious way back.
The tray menu's Show/Hide/Quit and Ctrl+Q still work regardless of
the flag; the toggle only affects what happens when the user clicks
the window's close button.
Added a settings-screen checkbox for the flag with a tooltip explaining
the SNI dependency.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Uses ksni (StatusNotifierItem via zbus) so we keep our pure-Rust /
no-GTK stack — zbus is already pulled in by notify-rust and rfd.
Behavior:
- Tray icon shows the bundled icon.png; left-click brings the window
back to focus
- Right-click menu: Show / Hide / Quit
- Clicking the window's X button minimizes to tray instead of exiting
- Ctrl+Q (or the tray's Quit item) actually exits
Failure modes are silent and non-blocking: no DBus → app behaves as if
tray flag was off. No StatusNotifier host (e.g. GNOME without the
AppIndicator extension) → icon invisible but app still runs normally,
and the X-button close cancellation only fires when the tray spawn
returned a handle.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Tartube parity 1.3 — the grid now sits under a row of filter chips that
narrow what's shown without changing the sort or sidebar view.
Dimensions:
- Watch: All / Unwatched / In progress / Watched
- Date: All / Today / Week / Month / Year / Older (against upload_date)
- Size: All / < 100 MB / 100 MB – 1 GB / > 1 GB
- 💬 Subs toggle: only videos with subtitle tracks
- 🔖 Chapters toggle: only videos with chapter markers
Filters AND together and persist to localStorage alongside view/sort.
They apply across every grid mode (channel, continue, recent, smart
folders) so a single chip selection follows you between views.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The fixed bottom #jobs bar ate up to 40vh of screen real estate even
when only one job was running, and on mobile it competed with the URL
input. Replace it with a ⬇ button in the header that:
- shows a badge with active+queued count (turns accent-colored when >0)
- opens a modal listing all jobs when clicked (or with the 'd' shortcut)
- repaints live from the same WS/polling snapshot that drove the old bar
The footer URL input + quality picker stay where they are — that's the
entry point, not the status display.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The three big dialogs were floating egui::Windows that crammed everything
into a fixed-height popover and hid the library underneath. They're now
full CentralPanel views with a "← Library" back button and a vertical
ScrollArea, so long settings panels and big stats reports scroll cleanly.
Top-bar nav uses a Screen enum instead of a trio of show_* booleans;
clicking the active screen returns to Library. Floating dialogs that are
genuinely modal (channel options, folder manager, move-to-folder) stay
as windows since they're actions, not screens.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
