Prose + commands updated to the new name: brand reads "Catacomb", while
binary/path/package references are the lowercase `catacomb` (commands, deb/rpm
names, ~/.local/share/catacomb, catacomb.db, catacomb.desktop). Codeberg repo
URLs left pointing at the existing repo. Also folds in the pending doc edits
from earlier in the session.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Download management UI (web + desktop):
- Cancel a running job (SIGKILL, marked "cancelled" so it isn't auto-retried
and shows a distinct state rather than a misleading error class)
- Cancel a still-queued job before it starts
- Manually retry a failed/cancelled job (fresh auto-retry budget)
- Live speed · ETA · % on running jobs, parsed from the yt-dlp progress line
- Expandable full per-job log fetched on demand
New endpoints: POST /api/jobs/:idx/{cancel,retry}, GET /api/jobs/:idx/log,
DELETE /api/queued/:idx.
Perceptual-dedup off-switch:
- backup.dedup_enabled (default true) hard-disables the "similar content"
scan in both UIs; the web scan endpoint returns {disabled:true} and the
desktop scan no-ops with a note. Wired through all five settings touchpoints.
Docs: README seeking note + ROADMAP recently-shipped updates for the prior
seekable-playback / federation / auto-tagging work.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Several related UI and startup fixes from one session:
- Web grid: channel and music views rendered in a single column because
their content was wrapped in one nested div that became a single cell of
the outer CSS grid. Span the wrappers (and .empty) across all columns
with grid-column:1/-1.
- Subtitles: route every track through /api/sub-vtt and strip per-cue
position/alignment settings (align:start position:0% etc.) from both SRT
and VTT, so auto-generated captions render centered instead of left-
aligned. New strip_cue_settings/normalize_vtt + sub_vtt_url helpers.
- Comment downloads: add a global backup.fetch_comments toggle (the
per-channel option already existed; it's now a tri-state override that
defers to the global). Full five-touchpoint wiring across config,
download_options, the downloader's apply_comments resolver, both UIs'
global + per-channel controls, and downloader seeding.
- Desktop startup: run the initial library scan + search-index sync on a
background thread (mirrors the web server's deferred bind) so the window
appears immediately instead of blocking on a cold-cache scan; update()
swaps the library in when it lands.
- Perceptual dedup: cap worker threads via fingerprint::default_workers()
(~half the cores, always leaving at least one free) instead of using
every core, so the hashing pass doesn't starve the UI.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment viewer (web UI): the flat tree gains search (highlight + keep
ancestor context), sort (threaded / top / newest / oldest), per-thread
collapse/expand with collapse-all/expand-all, an OP badge for the
uploader, and a "new since last visit" highlight + count (localStorage
per-video timestamp). API now returns each comment's `timestamp` and
`author_is_uploader`.
SponsorBlock: was a hardcoded `--sponsorblock-mark all`. Now a real
setting (off / mark / remove) with a per-channel override, threaded
through the full five touchpoints — config `[backup].sponsorblock_mode`
(default "mark", preserving prior behavior), DownloadOptions override,
a downloader resolver (apply_sponsorblock), and both UIs (egui Settings
+ channel dialog; web Settings modal + channel dialog + SettingsPayload).
Integration tests extended: settings round-trip asserts the sponsorblock
default + persistence; channel-options round-trip asserts the override.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A yt-dlp process can stall with the OS process alive but no progress — a
wedged TLS handshake, an unresponsive server, a dead socket mid-fragment.
Before, that job hung forever, holding a concurrency slot until the user
noticed and cancelled.
Now each Job tracks last_activity (bumped on any stdout/stderr line or
progress message). poll() runs check_watchdog(): a Running job silent for
HANG_TIMEOUT (5 min) gets SIGKILLed by pid via libc::kill. The threshold
sits well above any legitimate quiet gap — yt-dlp prints a line per
download chunk, polls every 30s under --wait-for-video, and our longest
sleep is ~30s — so 5 min of total silence is a real stall.
To capture the pid, spawn_job + enqueue_transcode now spawn the child
*before* the reader thread (the thread still owns it for the blocking
wait()). A watchdog kill sets watchdog_killed, and drain() forces the
resulting failure to NetworkError (retryable) instead of the Other that
classify() would return for an output-less kill — so the existing
auto-retry path re-queues it (a hang is transient).
Non-Unix builds keep the flagging but can't force-kill (no portable
pid-kill; Windows isn't first-class). JobState gains Debug for tests.
2 tests: watchdog kill → retryable class; normal failures still
classify from the log. 98 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Post-download ffmpeg pass with three modes, global + per-channel.
Closes the last real Tartube parity gap.
## Modes
- remux-mp4: fast container change mkv→mp4 (stream-copy video, aac audio)
- h264-mp4: re-encode to H.264/AAC at a configurable CRF + x264 preset
- audio: extract to mp3 / m4a / opus / flac
## Design (separate ffmpeg pass)
The main download adds `--print after_move:CONVERT_PATH:%(filepath)s`
when conversion is active. poll() scans freshly-Done download jobs for
those lines and enqueues a distinct ffmpeg transcode Job per finished
file — so the UI shows transcoding as its own phase. On success the
source is renamed to <name>.original.<ext> (keep_original) or deleted;
the converted file lands next to it. Transcodes don't auto-retry and
don't themselves convert (no infinite loop).
## Config
New [convert] section (mode / crf / preset / audio_format /
keep_original) + per-channel DownloadOptions.convert_mode override
(None = defer to global, "off" = force-disable for that channel).
ConvertSpec::resolve merges them; CRF 0→23, empty preset→medium, empty
audio→mp3 defaults.
## UI
Desktop + web: a "Format conversion (global defaults)" Settings section
with mode-dependent rows (CRF/preset for h264, format for audio, keep-
original toggle), and a per-channel Convert dropdown in the channel-
options dialog.
Verified: all three ffmpeg modes produce valid output on a real library
file (remux instant, h264 CRF23 ~12s for a 3.4min clip, mp3 extraction);
settings round-trip (global → config.toml [convert] → echo; per-channel
override saves + reads back). 4 resolver tests; 95 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
YouTube's bot-detection cracks down on different player clients over
time — the `web` client gets captcha-walled, then later `tv`/`mweb` are
the safe ones, etc. Let the user route around whichever client is
currently being targeted instead of being stuck with yt-dlp's pick.
- New backup.youtube_player_clients config (comma-separated, blank =
yt-dlp default). Maps to --extractor-args youtube:player_client=…
- Per-channel DownloadOptions.youtube_player_clients override (None =
defer to global). Resolved in the downloader's apply_player_client,
which omits the flag entirely when empty so yt-dlp keeps its default
client set (the recommended baseline).
- Threaded into the live downloader at construction + settings save
(app + web).
UI: a "YouTube player clients" field in both global Settings (desktop +
web) and the per-channel options dialog, with hints pointing at
"tv,mweb" as the current least-bot-checked combo for captcha-prone
channels.
Verified the settings round-trip (global persists to config + echoes;
per-channel override saves + reads back). 87 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two measures to keep a long channel scan going when YouTube's
bot-detection trips mid-batch.
## Auto-retry on transient failures
When a download fails with a retryable class (RateLimited — which now
includes captcha walls — or NetworkError), the downloader re-queues it
automatically after a cooldown, up to MAX_AUTO_RETRIES (2). Linear
backoff: 1st retry waits 90 s, 2nd waits 180 s. Permanent classes
(NotFound, MembersOnly, Geoblocked, CodecMissing, DiskFull, BadCookies)
are never retried — re-running changes nothing.
Implementation: start() captures a RetrySpec (url/info/flags/quality/
opts) onto the Job; poll() scans freshly-failed jobs, schedules due
retries into a cooldown queue, and fires them by rebuilding the command
through start(). Live recordings and synthetic preflight failures opt
out. A log line announces each scheduled retry so it's visible in the
Downloads panel.
## Adaptive throttle
After any rate-limit hit, rate_limited_backoff engages and the sleep
flags roughly triple for the rest of the batch (--sleep-requests 1→3,
--sleep-interval 2-6 s → 8-20 s), easing off an already-suspicious
endpoint instead of hammering it. Clears once downloads go idle so the
next fresh batch starts at normal speed.
2 new tests (retryable-class matrix, backoff sleep values). 87 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds full subtitle control (backend + desktop UI; web UI follows).
## Model
- New [subtitles] config section: enabled / auto_generated / embed /
format / langs. Sensible defaults (subs on, auto on, no embed/convert,
all langs).
- DownloadOptions gains per-channel Option overrides: subtitles_enabled,
subtitles_auto, subtitles_embed, subtitle_format (subtitle_langs
already existed). None = defer to global.
## Resolver
- Downloader gains subtitle_defaults (the global section) + a single
apply_subtitle_flags() that merges global + per-channel and emits the
yt-dlp flags (--write-subs / --write-auto-subs / --sub-langs /
--convert-subs / --embed-subs). Disabled = emit nothing.
- Replaces the hardcoded --write-subs --write-auto-subs at the main
download + repair builders. Moved the --sub-langs emission out of
DownloadOptions::apply into the resolver so global + override merge in
one place.
## Desktop UI
- Settings screen: "Subtitles (global defaults)" section.
- Channel-options dialog: tri-state (Default/On/Off) combos for each
override + a convert-format field.
## Wiring
- subtitle_defaults seeded at Downloader construction (app + web) and
refreshed on settings save (desktop).
4 resolver tests cover global defaults, disabled, global format/embed/
langs, and per-channel override-wins. 84 pass.
Web UI rows are the remaining piece (next commit).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Symptom: mid-batch (e.g. item 34 of 40) channel scans failed with
ERROR: [youtube] <id>: Video unavailable. YouTube is requiring a
captcha challenge before playback
Root cause: we hardcoded `--extractor-args youtube:player_client=web`
on every download. That flag dates from when `web` was the
throttle-avoidance client, but YouTube has since made `web` the MOST
aggressively bot-checked client — it's exactly the one that demands a
captcha. Verified directly: forcing player_client=web throws the
captcha error on a specific video that resolves fine without it.
Fix:
- Remove player_client=web from all three download builders (main,
repair, music). Modern yt-dlp auto-selects better clients
(android_vr, tv, web_safari) that aren't captcha-walled, and our POT
provider already serves tokens for them (web_safari in the logs).
- Add a randomized 2-6 s --sleep-interval between videos. The jitter
(vs a fixed cadence) is what keeps a long channel scan from looking
robotic and tripping the wall around request ~30.
Verified: both video IDs from the bug report (4JJ434QQoE4, Gr4bTC7DZWA)
that captcha'd under player_client=web now resolve cleanly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
YouTube increasingly requires a per-video Proof-of-Origin token bound
to each video ID before handing back format URLs. Without one, yt-dlp
sees empty formats and downloads fail. The upstream solution is the
Brainicism bgutil-ytdlp-pot-provider Python plugin paired with a
long-running server that mints tokens via BotGuard.
We integrate the Rust port (jim60105/bgutil-ytdlp-pot-provider-rs) so
the server is a single static binary, no Node.js required.
## Architecture
- bgutil-pot binary lives alongside the bundled deno + yt-dlp at
~/.local/share/yt-offline/bin/bgutil-pot.
- The matching Python plugin is pip-installed into the bundled venv:
python -m pip install bgutil-ytdlp-pot-provider.
- Downloader spawns the server lazily on first job (port 4416,
loopback only) and kills it on Drop.
- Each yt-dlp invocation gets
--extractor-args "youtubepot-bgutilhttp:base_url=http://127.0.0.1:4416"
appended in spawn_job when use_pot_provider is on and the server
child is alive.
## UX
- Settings → "POT token provider" row with toggle + Install/Update
button (mirrors the bundled yt-dlp row).
- Disabled unless use_bundled_ytdlp is also on (plugin lives in that
venv).
- Refuses install if the bundled yt-dlp venv isn't there yet, with a
helpful 428 error.
## Lifecycle
- Lazy spawn: ensure_pot_server() runs at the top of start() so the
server is up before yt-dlp queries the plugin.
- Drop: kill_server() on Downloader drop so we don't leave an orphaned
process bound to 4416.
- start_pot_provider_update() kills the running child first so the
install can overwrite the binary in place (no ETXTBSY).
3 new unit tests cover URL/extractor-args formatting; 77 total pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previously platform_root resolved YouTube to channels_root verbatim and
non-YouTube to siblings under channels_root.parent(). That assumed the
user's library was split across two levels:
/library/yt-offline/ ← config.backup.directory (YouTube)
/library/bandcamp/ ← sibling
/library/tiktok/ ← sibling
In practice users (myself included) keep everything under one umbrella:
/library/yt-offline/
channels/ ← YouTube
bandcamp/
tiktok/
music/
…
Make that the canonical layout. platform_root now returns
channels_root.join(dir_name) for every platform including YouTube
(which keeps its `channels` dir_name for back-compat with existing
on-disk trees that already have the channels/ subdir populated).
library_root is now == channels_root in both app.rs and web.rs — the
two-level concept is gone. The field stays as a distinct name so
file_url() and the /files/ static-file mount keep working without
churn at every call site. music_root and music dir also moved from
channels_root.with_file_name("music") to channels_root.join("music").
Tests in platform.rs updated to the new layout. 74 pass.
Note: this is a breaking change for installs that had the sibling
layout. The fix is to mv the platform dirs into the channels_root.
The next commit walks the user through that for my install.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two independent improvements from Phase 2.5.
## Panic hook → crash.log
GUI users launched from a .desktop file have no stderr — today panics
vanish without trace. Install a panic::set_hook early in main() that
appends a structured entry to <library_root>/yt-offline.crash.log for
every panic from every thread (UI, axum workers, downloader spawns,
tray bg thread, thumbnail decode workers).
Each entry: ISO-8601 timestamp, thread name, file:line, panic message,
and a backtrace when RUST_BACKTRACE is set. The default panic hook
still runs after ours, so dev workflows keep getting the stderr trace.
Bounded: when the log passes 256 KB the next write rotates it to .1
(one previous generation kept). No external date library — small
civil-time formula in-tree.
## Disk-full preflight
Before yt-dlp ever launches, statvfs the target filesystem. If free
space is under 500 MB, push a synthetic Failed job classified as
DiskFull instead of starting yt-dlp. That way:
- Users see the actual problem ("only 230 MB free on /mnt/…") in the
Downloads modal with the same "free up space and retry" hint the
error-classifier emits for mid-download ENOSPC.
- No half-written file lands on disk.
- No 30-second yt-dlp warmup wasted on a doomed run.
statvfs returning None (non-Unix, missing path) skips the check —
better to let yt-dlp run than to refuse on no data.
5 new tests: 3 in crash (civil-time, log shape, real-panic capture),
2 in disk_space (statvfs plumbing + missing-path None). 74 pass total.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When a job fails, today the user sees a raw stderr line — often
something like "ERROR: [youtube] dQw4w9WgXcQ: Sign in to confirm
you're not a bot" that doesn't tell a non-expert what to do.
New src/error_class.rs scans the failed job's log buffer for the
nine well-known yt-dlp fingerprints and returns:
- an ErrorClass enum (rate-limited, members-only, geo-blocked,
not-found, codec-missing, disk-full, network-error, bad-cookies,
other)
- a one-line human-readable suggested action paired with each class
Auto-populated in Job::drain when state transitions to Failed.
Exposed on JobSnapshot as error_class + error_hint. Rendered in
the web UI Downloads modal (red badge + hint box) and the desktop
downloads panel (colored label + hint text).
Other (no fingerprint matched) is filtered out of the badge so the
existing raw last_line keeps doing the talking for unknown failures.
Includes 10 unit tests covering each pattern + the walks-from-end
priority case.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the single largest gap with Tartube documented in docs/tartube-spec.md.
Users can now attach a small set of yt-dlp overrides to any individual
channel; the overrides apply automatically during scheduled re-checks
and the right-click "Check for new videos" action.
Data layer
- New `channel_options` SQLite table keyed on (platform, handle), with
`options_json TEXT` blob + updated_at. PK ensures one row per channel.
- Database gains get_channel_options / set_channel_options /
delete_channel_options / get_all_channel_options. The bulk getter is
used by the library scanner so each rescan hydrates options without
per-channel SQL round-trips.
- `DownloadQuality` derives Serialize/Deserialize so it can roundtrip
through the options blob.
New module: src/download_options.rs
- `DownloadOptions` struct with 9 fields:
quality (Option<DownloadQuality>) — per-channel quality cap
audio_only (bool) — force --extract-audio chain
limit_rate_kb (Option<u32>) — --limit-rate
min_filesize_mb / max_filesize_mb — yt-dlp filesize filters
date_after (Option<String>) — --dateafter YYYYMMDD
match_filter (Option<String>) — yt-dlp --match-filter passthrough
subtitle_langs (Vec<String>) — --sub-langs CSV
extra_args (Vec<String>) — raw yt-dlp passthrough (Tartube's
`extra_cmd_string`)
- `apply(&mut Command)` appends the right flags conditionally.
- `is_empty()` lets the API layer collapse no-op writes into DELETEs so
we don't keep useless rows around.
- `from_json()` falls back to defaults on malformed rows so a schema
drift doesn't take a channel offline.
- 8 new unit tests cover apply-emits-correct-flags, JSON roundtrip,
corrupt-fallback, and is_empty.
Library
- `Channel` gains `download_options: DownloadOptions` (default empty).
- New `library::apply_channel_options(channels, map)` helper that the
caller invokes after `scan_channels` to hydrate from the bulk DB load.
- All five `scan_channels` call sites updated.
Downloader
- `Downloader::start` gains a new `channel_options: Option<&DownloadOptions>`
parameter. Applied last so per-channel overrides win over global
defaults. The explicit "submit from URL bar" flow passes None
(channel unknown until yt-dlp resolves it). Scheduled re-checks +
right-click checks pass the channel's stored options. Channel-options'
`quality` field overrides the hard-coded DownloadQuality::Best for
re-checks.
Web API
- New routes on /api/channels/:platform/:handle/options:
GET — fetch (returns defaults when no row exists)
POST — upsert (empty body collapses to DELETE)
DELETE — clear overrides
- All three update the in-memory library snapshot immediately so the
next re-check sees the change without waiting for a rescan, and bump
the library_version ETag so polling clients pick up the new state.
Web UI
- Sidebar each channel's expanded section gains a "⚙ Channel options…"
entry next to "Check for new videos".
- openChannelOptions() fetches the current options + renders a modal
with one input per field. Save / Clear / Cancel buttons. The Clear
button confirms via window.confirm() because it's destructive.
- All form fields validate locally before POSTing; empty numeric
fields map to null.
Desktop UI
- New `channel_options_window` egui::Window opened from the existing
channel right-click context menu (new "⚙ Channel options…" item).
- Form mirrors the web side: ComboBox for quality, checkbox for
audio-only, TextEdit (singleline + multiline) for the other fields.
- Save / Clear / Cancel buttons; saves go straight to SQLite and the
live `App::library` so the next scheduled-check or right-click runs
with the new overrides.
Documentation
- `docs/tartube-spec.md` checklist: per-target download options ticked
with a note describing the v1 scope.
55 unit tests pass (47 + 8 new).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four items knocked off the unscoped list:
Auto-chmod 0600 cookies.txt
- web::write_cookies now sets mode 0600 on the file after writing, on Unix.
Mirrors the same guard we put on yt-offline.db. cookies.txt carries
live session credentials; no reason for it to stay umask-default.
Redact cookies path from job log
- Downloader::spawn_job now runs each stdout/stderr line through a small
redactor that replaces the absolute cookies.txt path with the bare
filename before forwarding to the Job log. yt-dlp can echo the path
in error strings, which leaks $HOME into the UI and /api/progress
responses.
- 2 new tests cover the strip + pass-through cases.
Bandcamp full-discography mode
- Bandcamp at the bare artist URL is already a Channel in our classifier
and yt-dlp's BandcampUserIE returns the full discography. The output
template for Bandcamp's channel case now organizes tracks into
per-album subfolders via `%(album|Unknown)s` so a discography pull
stays coherent instead of one flat directory.
- New `Downloader::apply_platform_extras` adds `--embed-thumbnail` for
audio-first platforms (Bandcamp, SoundCloud) so music players see the
cover art via embedded tags rather than scanning for a sidecar JPEG.
Library ETag + gzip
- WebState gains `library_version: AtomicU64`. `bump_library_version()`
is called from post_rescan, post_watched, post_maintenance_remove, and
post_resume (only when crossing the "Continue watching" >3.0 boundary
so playback-time updates don't constantly invalidate the cache).
- get_library is now Response-returning; it consults `If-None-Match` and
short-circuits with 304 Not Modified when the client's ETag matches.
Otherwise it sends `ETag: "<n>"` alongside the JSON.
- JS `loadLibrary()` caches the ETag and sends it on subsequent calls.
Returns early on 304 keeping the existing in-memory library array.
- Adds `tower_http::compression::CompressionLayer` (gzip). Already-
compressed media served from ServeDir is auto-skipped by the layer;
JSON responses get ~10× smaller. New `compression-gzip` feature on
the tower-http dep.
46 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three roadmap items together since they touch overlapping code:
Live-stream recording
- New `live: bool` argument on Downloader::start. When true, attaches
`--live-from-start --wait-for-video 30`, suppresses
`--break-on-existing` (every recording is a unique file), and
prepends a UTC timestamp suffix to the output filename so
re-recordings of the same broadcaster don't collide.
- The job label gains a "🔴 LIVE" marker so a long-running record is
obviously different from a VOD pull.
- Both UIs gain a "🔴 Live stream" checkbox in the download panel,
hidden when Music mode is selected. Works for Twitch *and* YouTube
Live since yt-dlp recognizes both from the URL.
- Web `POST /api/download` accepts a `live: bool` body field; all
other callers (scheduled re-check, right-click "Check for new
videos") pass false.
Recent-additions feed
- `library::Video` gains `mtime_unix: Option<u64>` — populated from
the metadata we already read for `file_size`, so no extra fs hit.
- New `SidebarView::Recent` (desktop) + `showRecent` (web) sort the
whole library by mtime descending and cap at 100 entries.
- Sidebar entry "🕒 Recent additions" appears only when the library
has any dated content, so a fresh install doesn't show an empty view.
Per-platform impersonation
- New `Platform::impersonate_target()`:
- Twitch → None (skip impersonation; their OAuth/Helix dislikes
mismatched TLS fingerprints).
- TikTok → "Chrome-Android-131" (mobile profile matches their
first-party app surface).
- Everything else → "Chrome-146:Macos-26".
- `Downloader::apply_impersonation` now takes the platform and routes
through the override; `repair()` (always YouTube-implicit) and
`start_music()` (uses classify_url on the URL) wire through too.
- `/api/preview` does the same classification before dispatching to
yt-dlp.
- 3 new tests confirm Twitch → None, TikTok → mobile, YouTube → desktop.
44 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous standalone PyInstaller binary couldn't run `--impersonate`
because curl_cffi isn't packaged into it. Bundled mode lost the
bot-detection bypass as a result.
Install layout is now:
~/.local/share/yt-offline/
bin/deno ← unchanged: prepended to PATH for JS deciphering
venv/bin/yt-dlp ← new: pip-installed yt-dlp[default] + curl_cffi
The install script:
- Errors clearly if python3 or the `venv` module isn't available
(the latter is split out on Debian as `python3-venv`).
- Creates the venv if missing, otherwise reuses it for upgrades.
- pip-installs `yt-dlp[default]` and `curl_cffi`, both with PyPI's
built-in SHA-256 verification.
- Cleans up the legacy `bin/yt-dlp` PyInstaller binary if a prior
install left one behind.
- Probes `--list-impersonate-targets` at the end and logs whether
curl_cffi loaded successfully.
Downloader::apply_impersonation now passes --impersonate
unconditionally — both bundled (via the venv) and system yt-dlp (via
pip-installed curl_cffi) honor it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The bundled PyInstaller binary doesn't include curl_cffi, so passing
--impersonate Chrome-146:Macos-26 makes it error out with
"Impersonate target ... is not available". System yt-dlp installed via
pip can have curl_cffi alongside it and works fine.
Centralize the decision in `Downloader::apply_impersonation` and apply
the same gate to the `/api/preview` handler.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
yt-dlp already speaks ~1,800 sites; the changes here teach the rest of
the app to route them through platform-specific output folders and
remember where each creator was downloaded from.
New `platform` module
- `Platform` enum: YouTube, TikTok, Twitch, Vimeo, Bandcamp, SoundCloud,
Odysee, Other.
- `classify_url` returns both the source platform and a `UrlKind`
(Channel / Playlist / Video / Unknown) — per-platform path parsing
for each supported host.
- `platform_root(channels_root, platform)` returns the on-disk folder
per platform. YouTube keeps the legacy `channels/` for backward
compat; the others live as siblings (`tiktok/`, `twitch/`, …).
- `write_source_url` / `read_source_url` persist the originating URL
in a `.source-url` sidecar at each creator folder so re-checks no
longer rely on the YouTube-only "UC + 22 chars" heuristic.
Downloader
- `Downloader::start` now takes `&UrlInfo` instead of `&UrlKind` and
routes the yt-dlp `-o` template into the right platform folder.
- For channel downloads it also writes `.source-url` so future
re-checks recover the exact URL.
- Output template uses `%(uploader,channel,creator|Unknown)s` so
non-YouTube sites that don't expose `channel` still get a sensible
creator folder.
- Per-platform `archive.txt` keeps cross-platform IDs (TikTok numeric
vs YouTube base64) from colliding.
- New `recheck_url(&Channel)` prefers `source_url` and falls back to
the legacy YouTube heuristic for libraries created before this
change.
Library scanner
- `scan_channels` now walks every platform's folder and tags each
`Channel` with its `platform` + `source_url`. Sort key puts the
platform first, then name, so the sidebar groups cleanly.
- `Channel` gains `platform: Platform` and `source_url: Option<String>`
fields, populated from the on-disk sidecar.
Web UI
- `WebState` gains `library_root` (= parent of channels_root). The
`/files/` ServeDir is mounted at library_root so non-YouTube videos
resolve at `/files/<platform>/<creator>/<file>`. Existing YouTube
URLs still work — they're now `/files/channels/...` instead of
`/files/...` and the server-rendered JSON updates accordingly.
- Maintenance scan + remove use `library_root` so non-YouTube content
is included.
- `ChannelInfo` JSON exposes `platform`, `platform_label`,
`platform_icon`, and `source_url`.
- Sidebar groups channels by platform with each platform's icon.
- "Check for new videos" now uses the stored `source_url` (with the
YouTube heuristic as a last-resort fallback).
- Download dialog preview shows the detected source + destination
folder.
Desktop UI
- App gains a `library_root` field mirroring the web side; maintenance
ops scan/remove against it.
- Sidebar channel labels are prefixed with the platform icon for
non-YouTube channels; tooltip carries the platform name.
- Download URL field surfaces both source platform and folder path
preview as you type.
Plex
- Non-YouTube creators get their show folder prefixed with the
platform name (e.g., `TikTok - cooluser`) so a YouTube channel and
a TikTok account with the same name don't collide.
Backward compatibility
- Existing YouTube libraries (`channels/<handle>/...`) are untouched.
New platform folders are created on first run as siblings.
- Channels without a `.source-url` (i.e., everything that pre-dates
this commit) fall through to the YouTube heuristic, preserving the
existing re-check behavior.
Tests
- 14 new `platform` tests cover URL classification per platform,
channel handle extraction, and `platform_root` layout. The old
`detect_url_kind` / `extract_after` tests move into platform.rs
since the canonical implementation lives there now.
- 41 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bug fixes
- Desktop auto-rescan: snapshot was_running before check_notifications
updates prev_job_states, so the post-download rescan actually fires.
- web::run shutdown: replace `let _ = tx;` (which dropped the sender
immediately and short-circuited the shutdown select!) with
std::mem::forget(tx); add an idle fallback so the `-> !` is honored.
- /api/preview now resolves the bundled yt-dlp path and extends PATH
for deno discovery, matching the rest of the downloader.
- Scheduler interval clamped to >=1h on read so a manually-edited
config.toml with 0 hours can't trigger every tick.
- maintenance::is_within canonicalizes the parent + filename when the
target is missing; remove_files treats NotFound as success.
Security
- Session tokens stored with issued-at Instant and pruned past 30 day
TTL on every touch (was: unbounded HashSet).
- Login rate-limited per source IP: 5 failures → 60s lockout (429).
- Secure cookie flag added when X-Forwarded-Proto: https is present.
- 4 MiB body size cap via DefaultBodyLimit.
- Security headers middleware: CSP, X-Content-Type-Options,
X-Frame-Options, Referrer-Policy.
- JS safeUrl() defangs javascript:/vbscript:/data:text URLs before
interpolation into src= attributes.
- Bundled yt-dlp install verifies SHA-256 against the release's
SHA2-256SUMS; deno hash printed for visual inspection. Install also
reports byte counts every 3s so users see live progress.
- yt-offline.db chmod 0600 at open time on Unix.
New features
- Bundled yt-dlp + deno mode: settings toggle (System/Bundled) plus
one-click Install/Update from settings. Ensures executable bit on
every launch in case the install chmod was missed.
- Download queue with configurable max_concurrent (1–10); pending
jobs surfaced in both UIs.
- Music download mode: --extract-audio into music/<artist>/, separate
sidebar entry, /api/music + /music-files/ endpoints, inline <audio>.
- Download quality selector: Best / 1080p / 720p / 480p / 360p.
- Sort videos by Newest / Oldest (upload_date from info.json with
release_date fallback).
- Plex metadata: per-episode .nfo (title, season/episode, aired,
runtime, plot from .description), <stem>-thumb.jpg symlink, and
show-level tvshow.nfo.
- yt-dlp retry/throttle defaults: --retries 30 --fragment-retries 30
--retry-sleep linear=1:30:2 --sleep-requests 1 to ride out the
"Connection reset by peer" failures.
Performance
- password_required cached in AtomicBool; refreshed only on password
change. Eliminates a SQLite query per static-file fetch.
- Job::log → VecDeque for O(1) front-pop instead of O(n) drain.
- Library scan parallelized across cores via stdlib parallel_map.
- Web UI poll: 600ms while active, 5s when idle, 2s after errors.
- Music scan now recursive (Artist/Album/Track.opus).
- percent_encode_segment uses write! to avoid per-byte String allocs.
Code quality
- 27 unit tests covering parse_progress, parse_stem, extract_after,
detect_url_kind, srt_to_vtt, count_cookies, percent_encode_segment,
bind_mode_of, hash_password, lang_label, is_within, sanitize,
year_of, aired_date, xml_escape.
- Unified library::find_video + Channel::all_videos() iterator,
replacing three duplicated linear searches.
- Downloader::browser wired to --cookies-from-browser as a fallback
when no cookies.txt is present.
- mpv detection now matches the binary basename, not substring.
- Settings modal scroll fix; web settings expose all new fields.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Full-scan download toggle: omits --break-on-existing so all unarchived
videos are fetched regardless of order, filling gaps in partial archives
- Filter "Aborting remaining downloads" from job logs (break-on-existing noise)
- Password hash moved from config.toml to SQLite settings table; survives
git checkouts and rebuilds. config.toml and yt-offline.db added to .gitignore;
config auto-generated on first run if absent
- Channel re-check URLs derived from folder name (/@handle or /channel/UCxxx)
instead of info.json's canonical channel_url, preventing duplicate UCxxx folders
- SRT subtitles detected by library scanner and served via /api/sub-vtt/* endpoint
that converts to WebVTT on the fly; browser <track> element can now play them
- Cookie clear button in both desktop and web UI (DELETE /api/cookies)
- Per-job X close button on finished download jobs in desktop GUI
- Fix widget ID clash when multiple download jobs are shown (push_id per job)
- Watched videos excluded from Continue Watching list and badge count
- settings table added to SQLite schema (key/value store)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Update cookies.txt by pasting Netscape-format contents in web UI settings and
the desktop GUI settings window (shared write/validation in web.rs)
- Treat yt-dlp exit code 101 (--break-on-existing reached an archived video) as
a successful "up to date" outcome instead of a failed job
- Add --no-write-playlist-metafiles so channel avatars/info aren't written as
phantom "Title [CHANNEL_ID]" files
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Features:
- Port all 7 desktop themes (Dracula, Trans, Emo variants, etc.) to the web UI
- Configurable bind interface (localhost/Tailscale/LAN/all) with detected IPs,
settable from both the web UI and desktop GUI
- Optional password that gates the whole UI and all API access via session
cookies (Argon2), with a login page, logout, and middleware; settable from
both UIs
- Library maintenance: scan for duplicate video IDs and missing assets
(thumbnail/info.json/description), remove duplicates (scan-and-confirm, with
out-of-library path guard), and re-fetch missing sidecars via yt-dlp.
Surfaced in both the web UI and a desktop GUI window.
Optimizations / cleanup:
- Cache has_chapters at scan time (library.rs) instead of re-reading and
parsing every info.json on each /api/library request
- Use the cached channel size in get_library
- Pool a single SQLite connection in WebState instead of opening one per request
- Remove a 1.2 GB stray nested src/yt-offline duplicate tree
Fixes:
- Rescan no longer clears all thumbnail textures (caused thumbnails to unload);
prune only removed entries and bump library_generation so the card grid
refreshes correctly
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- SQLite DB for watched/position persistence (database.rs)
- Mark watched toggle on cards and detail panel
- mpv IPC socket integration for resume position tracking (unix)
- Sort by title, duration, file size
- Thumbnail density slider in top bar
- Channel metadata banner (subscriber count, uploader, channel URL)
- Cookie browser selector dropdown in settings
- Auto-rescan library when a download job finishes
- Fix yt-dlp --no-progress-bar flag (does not exist; removed)
- Browser field wired through config → downloader
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- library.rs: scans channels/<name>/ into channels + videos by filename stem
- downloader.rs: runs yt-dlp in a background thread, streams progress to the UI
- app.rs / main.rs: channel sidebar, searchable thumbnail list, detail/description
panel, downloads panel; plays videos via mpv (falls back to xdg-open)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
