YouTube's bot-detection cracks down on different player clients over
time — the `web` client gets captcha-walled, then later `tv`/`mweb` are
the safe ones, etc. Let the user route around whichever client is
currently being targeted instead of being stuck with yt-dlp's pick.
- New backup.youtube_player_clients config (comma-separated, blank =
yt-dlp default). Maps to --extractor-args youtube:player_client=…
- Per-channel DownloadOptions.youtube_player_clients override (None =
defer to global). Resolved in the downloader's apply_player_client,
which omits the flag entirely when empty so yt-dlp keeps its default
client set (the recommended baseline).
- Threaded into the live downloader at construction + settings save
(app + web).
UI: a "YouTube player clients" field in both global Settings (desktop +
web) and the per-channel options dialog, with hints pointing at
"tv,mweb" as the current least-bot-checked combo for captcha-prone
channels.
Verified the settings round-trip (global persists to config + echoes;
per-channel override saves + reads back). 87 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Completes the subtitle feature started in 0814ba0 (backend + desktop).
## Global settings (web Settings modal)
- New "Subtitles (global defaults)" section: enable toggle that reveals
auto-captions / embed / languages / convert-format controls.
- SettingsPayload gains the 5 subtitle fields; get_settings reads them
from [subtitles] config, post_settings writes them back, saves config,
and pushes the live value onto downloader.subtitle_defaults.
## Per-channel overrides (channel-options dialog)
- Tri-state selects (Default / On / Off) for download-subtitles,
auto-captions, embed; a convert-format text field. "Default" defers
to the global setting.
- triSelect() / triValue() helpers map Option<bool> ⇄ select value.
readChannelOptionsForm includes the four new fields; they serialize
straight onto DownloadOptions via serde (post_channel_options already
takes the whole struct).
## Verified end-to-end (live server)
- Global: GET defaults → POST (auto off, embed on, langs=en, format=srt)
→ echoed back → written to config.toml [subtitles] → persists across
restart.
- Per-channel: POST overrides → read back exactly; all-default body
hits the is_empty() delete path and clears the row.
84 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds full subtitle control (backend + desktop UI; web UI follows).
## Model
- New [subtitles] config section: enabled / auto_generated / embed /
format / langs. Sensible defaults (subs on, auto on, no embed/convert,
all langs).
- DownloadOptions gains per-channel Option overrides: subtitles_enabled,
subtitles_auto, subtitles_embed, subtitle_format (subtitle_langs
already existed). None = defer to global.
## Resolver
- Downloader gains subtitle_defaults (the global section) + a single
apply_subtitle_flags() that merges global + per-channel and emits the
yt-dlp flags (--write-subs / --write-auto-subs / --sub-langs /
--convert-subs / --embed-subs). Disabled = emit nothing.
- Replaces the hardcoded --write-subs --write-auto-subs at the main
download + repair builders. Moved the --sub-langs emission out of
DownloadOptions::apply into the resolver so global + override merge in
one place.
## Desktop UI
- Settings screen: "Subtitles (global defaults)" section.
- Channel-options dialog: tri-state (Default/On/Off) combos for each
override + a convert-format field.
## Wiring
- subtitle_defaults seeded at Downloader construction (app + web) and
refreshed on settings save (desktop).
4 resolver tests cover global defaults, disabled, global format/embed/
langs, and per-channel override-wins. 84 pass.
Web UI rows are the remaining piece (next commit).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Folders were flat — one level under each platform. Tartube supports
arbitrary nesting; now we do too.
## Schema
Add `folders.parent_id INTEGER REFERENCES folders(id) ON DELETE CASCADE`
via an idempotent migration (ALTER TABLE, swallow the duplicate-column
error on already-migrated DBs). NULL = top-level. Cascade means deleting
a parent removes its subtree's folder rows; member channels revert to
Unfiled via the existing channel_assignments cascade.
## DB
- FolderRecord gains parent_id (flows to both UIs via serde).
- set_folder_parent(id, parent) reparents, with a cycle guard: walks the
ancestor chain of the proposed parent and refuses if `id` appears
(you can't nest a folder inside its own descendant). Self-parent also
rejected. Returns a friendly SQLITE_CONSTRAINT error the web layer
maps to 400.
## API
POST /api/folders/:id/parent { parent_id } — reparent or null for top.
## Web UI
- Sidebar renders the folder tree recursively: each folder indents one
step under its parent, member channels indent under their folder, and
the count shown is the whole subtree. A seen-set guards against any
malformed cycle in the data.
- Folder manager gains a per-row "parent" dropdown; choices that would
cycle (self + descendants) are excluded.
## Desktop UI
- Channel-panel sidebar builds the same tree with an iterative DFS
(depth-indented folder headers, subtree counts).
- Folder manager gains a parent ComboBox per row with the same
cycle-exclusion.
Verified the migration runs against an existing real DB (parent_id added,
no data loss). 1 new test covers nesting + both cycle cases; 80 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Free-text annotations on any channel or video, searchable from the
global filter box.
## Storage
New `notes` table keyed by (target_kind, target_id):
- target_kind: "channel" or "video"
- target_id: "platform/handle" for channels, video ID for videos
- empty body deletes the row, so the table only holds real notes
DB methods: get_note, set_note (upsert-or-delete), get_all_notes.
## API
- GET /api/notes → { "video:<id>": body, "channel:<plat>/<handle>": body }
fetched once on page load, held in memory for indicators + search
- POST /api/notes/:kind/:id { body } → upsert/delete
## Web UI
- 📝 button on every video card; turns accent-colored when a note
exists. Opens an edit-in-place modal.
- 📝 Add/Edit note… sub-action under each channel in the sidebar.
- Global search now also matches note bodies, so you can find a clip
by what you wrote about it.
## Desktop UI
- Note field in the video detail panel (bottom dock). Loads lazily
from the DB when the selection changes; persists on focus-loss.
## Restore
Backup import merges the notes table too (keep-later-timestamp,
same as watched/positions). Guarded with a table-exists check so
pre-notes backups still import cleanly. RestoreSummary gains
notes_added; both UIs show it.
4 new tests (2 notes DB behavior, restore merge already covered the
shape). 79 pass total.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
YouTube increasingly requires a per-video Proof-of-Origin token bound
to each video ID before handing back format URLs. Without one, yt-dlp
sees empty formats and downloads fail. The upstream solution is the
Brainicism bgutil-ytdlp-pot-provider Python plugin paired with a
long-running server that mints tokens via BotGuard.
We integrate the Rust port (jim60105/bgutil-ytdlp-pot-provider-rs) so
the server is a single static binary, no Node.js required.
## Architecture
- bgutil-pot binary lives alongside the bundled deno + yt-dlp at
~/.local/share/yt-offline/bin/bgutil-pot.
- The matching Python plugin is pip-installed into the bundled venv:
python -m pip install bgutil-ytdlp-pot-provider.
- Downloader spawns the server lazily on first job (port 4416,
loopback only) and kills it on Drop.
- Each yt-dlp invocation gets
--extractor-args "youtubepot-bgutilhttp:base_url=http://127.0.0.1:4416"
appended in spawn_job when use_pot_provider is on and the server
child is alive.
## UX
- Settings → "POT token provider" row with toggle + Install/Update
button (mirrors the bundled yt-dlp row).
- Disabled unless use_bundled_ytdlp is also on (plugin lives in that
venv).
- Refuses install if the bundled yt-dlp venv isn't there yet, with a
helpful 428 error.
## Lifecycle
- Lazy spawn: ensure_pot_server() runs at the top of start() so the
server is up before yt-dlp queries the plugin.
- Drop: kill_server() on Downloader drop so we don't leave an orphaned
process bound to 4416.
- start_pot_provider_update() kills the running child first so the
install can overwrite the binary in place (no ETXTBSY).
3 new unit tests cover URL/extractor-args formatting; 77 total pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previously platform_root resolved YouTube to channels_root verbatim and
non-YouTube to siblings under channels_root.parent(). That assumed the
user's library was split across two levels:
/library/yt-offline/ ← config.backup.directory (YouTube)
/library/bandcamp/ ← sibling
/library/tiktok/ ← sibling
In practice users (myself included) keep everything under one umbrella:
/library/yt-offline/
channels/ ← YouTube
bandcamp/
tiktok/
music/
…
Make that the canonical layout. platform_root now returns
channels_root.join(dir_name) for every platform including YouTube
(which keeps its `channels` dir_name for back-compat with existing
on-disk trees that already have the channels/ subdir populated).
library_root is now == channels_root in both app.rs and web.rs — the
two-level concept is gone. The field stays as a distinct name so
file_url() and the /files/ static-file mount keep working without
churn at every call site. music_root and music dir also moved from
channels_root.with_file_name("music") to channels_root.join("music").
Tests in platform.rs updated to the new layout. 74 pass.
Note: this is a breaking change for installs that had the sibling
layout. The fix is to mv the platform dirs into the channels_root.
The next commit walks the user through that for my install.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four changes that compound — most users will notice these on first
launch of a large library.
## Release profile
- opt-level: 2 → 3 (more inlining + vectorization, ~5-15% on hot paths)
- lto = "thin" (cross-crate inlining; thin avoids the rust-lld + bundled
sqlite link breakage that full LTO previously caused, which is what
the !lto note was about)
- codegen-units = 1 (whole-crate inlining, ~5% runtime, +30s build)
- strip = "debuginfo" (drops ~30 MB from the binary)
## Thumbnail decode pool
The single decoder thread was the bottleneck on grid fill for libraries
with hundreds of channels. Replace with a worker pool sized at
clamp(2, cores/2, 8). Workers share the request channel behind a Mutex;
lock contention is negligible because each worker spends ~all its time
in image::open + resize.
## info.json parse cache
Library scans re-parse every video's info.json sidecar on every rescan —
serde_json::from_str is the dominant cost on a warm-filesystem scan
(file reads themselves are OS-cached). Add a `info_cache` SQLite table
keyed by (path, mtime); enrich_with_cache hits it first and only parses
on miss. mtime-keyed invalidation means yt-dlp re-writing an info.json
auto-invalidates without explicit eviction.
Each scan worker checks out its own r2d2 connection so lookups don't
serialize. Database is now Clone (the inner Pool is Arc, so cloning is
cheap) so we can hand handles to parallel workers.
## /api/library body cache
The ETag short-circuit catches clients with a matching cached version;
this catches clients without one (curl, freshly-opened tabs, mobile
WebViews that don't store ETags reliably). When the version matches
the cached entry's, we hand back the previously-serialized bytes
without re-walking the channel tree or re-running serde_json::to_string.
A post-build version check prevents poisoning the cache with stale
bytes when the library changes during serialization.
74 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Backup direction has shipped for a while; this adds the import side
so users can actually round-trip their snapshot.
Database::restore_from_backup attaches the uploaded file and merges
each table with per-table semantics:
- watched / positions / channel_options: keep the later timestamp
- video_flags: bitwise-OR each flag (favourite + bookmark across two
devices both stick)
- folders: insert names that don't exist yet
- channel_assignments: insert when (platform, handle) is unassigned;
folder_id is re-resolved by name so backup/live ID drift is fine
- settings: deliberately skipped (password hash + source_url are
per-deployment state, not user data)
Re-importing the same snapshot is a no-op. Schema validation rejects
non-yt-offline SQLite files up front with a 400.
POST /api/restore/db takes raw .db bytes (100 MB cap, magic-header
check) and returns a RestoreSummary with per-table added counts.
After success, the in-memory watched/positions/flags caches are
refreshed and the library version is bumped so cached /api/library
responses revalidate.
Desktop: 📂 Import library backup… button in Settings, mirrors the
existing 💾 save button. Triggers rescan after the merge so channel
folder assignments take effect.
Web UI: 📂 Import library snapshot… button next to the download
button. Hidden file input + confirm dialog + result summary.
Includes 4 new unit tests in src/database.rs covering the watched +
positions merge, flag OR-ing, idempotency on re-run, and schema
rejection.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When a job fails, today the user sees a raw stderr line — often
something like "ERROR: [youtube] dQw4w9WgXcQ: Sign in to confirm
you're not a bot" that doesn't tell a non-expert what to do.
New src/error_class.rs scans the failed job's log buffer for the
nine well-known yt-dlp fingerprints and returns:
- an ErrorClass enum (rate-limited, members-only, geo-blocked,
not-found, codec-missing, disk-full, network-error, bad-cookies,
other)
- a one-line human-readable suggested action paired with each class
Auto-populated in Job::drain when state transitions to Failed.
Exposed on JobSnapshot as error_class + error_hint. Rendered in
the web UI Downloads modal (red badge + hint box) and the desktop
downloads panel (colored label + hint text).
Other (no fingerprint matched) is filtered out of the badge so the
existing raw last_line keeps doing the talking for unknown failures.
Includes 10 unit tests covering each pattern + the walks-from-end
priority case.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
WebSocket progress
- New `/ws/progress` route using axum 0.7's `ws` feature. Upgrades to
a WebSocket that streams the same `ProgressResponse` payload as the
HTTP polling endpoint, pushed instantly whenever a job moves.
- `WebState.progress_tx` is a `tokio::sync::broadcast::Sender<String>`
(capacity 16, lossy — Lagged subscribers resync on the next tick).
- A background tokio task ticks every 500 ms while any download is
running or queued, every 5 s when idle. Skips the JSON encode when
no subscribers are listening, so a single-user install pays zero
CPU for the broadcast when the browser tab is closed.
- JS replaces the per-interval HTTP poll with a WebSocket connection.
The polling path stays as fallback for mobile carriers / reverse
proxies that strip WS — when the socket closes or fails to open,
schedulePoll() resumes immediately and a reconnect attempt fires
after 5 s.
- Initial snapshot is sent right after subscribe so the UI is live
before the next tick.
Mobile-responsive web UI
- Existing `@media(max-width:640px)` block tightened:
- Hide the H1, AGPL footer notice, sort dropdown, and library-size
counter — they don't fit and aren't actionable.
- Header buttons get min 34×34 hit targets (Apple HIG / Material).
- `.card-foot` wraps and grows to 34×34 buttons so the five action
icons (play / watched / favourite / bookmark / waiting) stay
tappable instead of cramming into thumb-unfriendly 22 px chips.
- Modals go full-screen (no 10 px page-margin) so the iPhone safe
area doesn't crop content.
- Footer (URL bar + download button) wraps onto two rows so neither
squeezes the other off-screen.
- Bulk-actions toolbar wraps and gets bigger buttons.
- Per-job stdout preview hides on small screens.
- New `@media(max-width:380px)` collapses the grid to single column
for very narrow phones.
55 unit tests still pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two small follow-ups to the folder hierarchy that round out the
folder-as-a-profile workflow and give the user a one-click way to keep
an offsite copy of their library state.
Per-folder check
- New `POST /api/folders/:id/check` mirrors the existing
/api/scheduler/run endpoint but scoped to one folder's members.
Each channel's stored DownloadOptions (quality / rate limit /
match-filter / extra args) applies just like a scheduled re-check.
Returns 409 when downloads are already running.
- Web folder-manager modal: each non-empty folder gets a "⬇ Check"
button next to Rename / Delete. Status line reports the count of
channels queued.
- Desktop folder-manager window: same — a small ⬇ icon-button per row,
visible only for folders with at least one member. Same options-
honouring re-check path as the right-click "Check for new videos"
action.
- This effectively gives us "Profiles" lite: organize channels into a
folder, then trigger a batch re-check whenever you want.
DB backup
- New `GET /api/backup/db` reads `<channels_root>/yt-offline.db` and
streams it back with `Content-Disposition: attachment` so the
browser downloads instead of rendering. Filename includes the unix
timestamp so downloads don't clobber. Returns 404 when running in
in-memory mode (no DB file to dump).
- We deliberately don't bundle config.toml + cookies.txt here:
config is short to recreate and cookies.txt carries live session
credentials that shouldn't fly over the wire unprompted.
- Web settings modal gains a "⬇ Download library snapshot" button in
a new Backup section, with a hint explaining what's covered (watched
/ favourites / bookmarks / waiting / channel-options / folders).
55 unit tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Users can now organize channels into named folders independent of the
underlying platform groupings. v1 ships single-level (no nesting) since
that's where ~90% of the organizational value lives — nested folders +
drag-to-reparent + per-folder cascading options are deferred to v2 once
we have real-world feedback on what the workflow needs.
Data layer
- New `folders` SQLite table: id, name (UNIQUE), position, created_at.
- New `channel_assignments` table: (platform, handle) → folder_id with
ON DELETE CASCADE so deleting a folder cleanly unfiles its channels.
- Database methods: create_folder, rename_folder, delete_folder (with
PRAGMA foreign_keys ON for the cascade), list_folders,
set_channel_folder, get_all_channel_assignments.
- New `FolderRecord` struct (id/name/position) is `Serialize` so it
rides on the /api/library response.
Library
- `Channel` gains `folder_id: Option<i64>` (None = Unfiled).
- New `library::apply_channel_folders` mirror of `apply_channel_options`
populates it from the bulk DB map after every scan / rescan.
Web API
- `POST /api/folders` create with `{name}` → `{id}`.
- `POST /api/folders/:id/rename` rename with `{name}`.
- `DELETE /api/folders/:id` cascade-deletes assignments.
- `POST /api/channels/:platform/:handle/folder` upsert/clear via
`{folder_id: <i64|null>}`. All endpoints bump the library ETag.
- `/api/library` response now carries a `folders: [...]` array so the
client can render the sidebar grouping without a second round trip.
Web UI
- Sidebar gains a "📁 Folders" section above the platform groupings.
Each folder lists its member channels (with platform icon prefix);
channels with `folder_id` are pulled out of their platform section
to avoid duplication. Unfiled channels keep platform grouping.
- "manage" / "new" button next to the Folders heading opens a modal
with the full folder list (member counts inline) plus rename /
delete buttons and a Create input.
- New per-channel sub-action "📁 Move to folder…" opens a small
picker modal with one row per folder + "— Unfiled —". Current
assignment highlighted.
Desktop UI
- Sidebar refactored to build a `Vec<SidebarItem>` ordered list
(FolderHeader / FolderManageBtn / PlatformHeader / Channel) and
iterate it once, instead of the previous flat
`for i in 0..library.len()` loop. Folder section renders above
the platform sections; channels with `folder_id` are skipped from
the platform-grouped loop so they appear in exactly one place.
- New `folder_manager_window` + `move_to_folder_window` egui windows
cover create / rename (via the create-buffer field) / delete and
per-channel folder picking. Right-click context menu on a channel
gains "📁 Move to folder…".
- App struct gains: `folders`, `show_folder_manager`,
`folder_create_buffer`, `show_move_to_folder`, `move_to_folder_target`.
Tartube spec checklist
- Folder hierarchy box ticked (with the v2 deferrals noted in the
inline annotation).
55 unit tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three Tartube-parity items shipped together since they touch the same
schema and rendering paths.
Per-video state flags
- New `video_flags` SQLite table with `bookmark`/`favourite`/`waiting`/
`archive` columns (one row per video that has at least one flag set).
- `Database::set_video_flag` validates the flag name against an
allow-list before interpolating into the SQL — column names can't be
parameterised in SQLite, but the allow-list keeps it injection-free.
- `Database::get_video_flags` bulk-loads into a new
`VideoFlagsBundle { bookmark, favourite, waiting, archive: HashSet<String> }`
so subsequent reads are O(1) in-memory hits.
- `WebState.flags` (Mutex<VideoFlagsBundle>) + `App.flags`
(VideoFlagsBundle, GUI is single-threaded) populated at startup.
- `POST /api/videos/:id/flags/:flag` toggles a flag; the in-memory
bundle is mirrored immediately so the next /api/library reflects the
change without a rescan. Library ETag is bumped.
- `VideoInfo` JSON gains `bookmark` / `favourite` / `waiting` /
`archive` boolean fields.
- `archive` is wired through everywhere but no UI gate yet — it's
reserved for the future auto-delete feature.
Smart folders
- Both UIs gain "★ Favourites", "🔖 Bookmarks", "⏳ Waiting" sidebar
entries between Continue Watching and Channels. Each is hidden when
its set is empty, so a fresh install isn't polluted with zero-count
rows.
- Desktop: new `SidebarView::Favourites / Bookmarks / Waiting`
variants. `compute_cards` filters the library by the matching
`self.flags.<set>` HashSet and returns the resulting cards using the
same rendering path as the other views.
- Web: `showFavourites/Bookmarks/Waiting` flags + smart-folder branch
in `currentVideos()`. Refactored the view-mode selectors through a
new `resetViewSelectors()` helper so adding three more booleans
didn't quadruple every `setX` function.
Per-card flag toggles
- Web: each video card grows three new action buttons (☆/★ favourite,
🔖 bookmark, ⏳ waiting) next to the existing watched toggle.
Optimistic UI — flip in-memory immediately, undo on server error.
- Desktop: same three buttons next to Watched in the card row, deferred
through a `toggle_flag_card: Option<&'static str>` to keep the
borrow checker happy. New `App::toggle_video_flag` helper persists
the flip to SQLite and invalidates the card cache.
Comments capture
- `DownloadOptions` gains `fetch_comments: bool`. When set, the
downloader emits `--write-comments` so yt-dlp embeds the full
comment tree into info.json.
- New `GET /api/comments/:id` reads the array out of info.json and
returns a slimmed-down version (id, author, text, likes, parent,
time) — strips the multi-MB extras yt-dlp can otherwise inline.
- Web player modal grows a 💬 button that opens a separate modal
with a threaded viewer: replies indented by parent id, "N comments"
header, helpful message when nothing's captured pointing at the
channel-options toggle.
- Channel-options dialog in both UIs gains a "Fetch comments"
checkbox + hint about the slowdown.
Tartube spec checklist
- Three more boxes ticked: per-video state flags, smart folders,
comments capture. Score now: 5 ahead, 5 behind, 1 tied + matching
on 4 parity items. (Half done.)
55 unit tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the single largest gap with Tartube documented in docs/tartube-spec.md.
Users can now attach a small set of yt-dlp overrides to any individual
channel; the overrides apply automatically during scheduled re-checks
and the right-click "Check for new videos" action.
Data layer
- New `channel_options` SQLite table keyed on (platform, handle), with
`options_json TEXT` blob + updated_at. PK ensures one row per channel.
- Database gains get_channel_options / set_channel_options /
delete_channel_options / get_all_channel_options. The bulk getter is
used by the library scanner so each rescan hydrates options without
per-channel SQL round-trips.
- `DownloadQuality` derives Serialize/Deserialize so it can roundtrip
through the options blob.
New module: src/download_options.rs
- `DownloadOptions` struct with 9 fields:
quality (Option<DownloadQuality>) — per-channel quality cap
audio_only (bool) — force --extract-audio chain
limit_rate_kb (Option<u32>) — --limit-rate
min_filesize_mb / max_filesize_mb — yt-dlp filesize filters
date_after (Option<String>) — --dateafter YYYYMMDD
match_filter (Option<String>) — yt-dlp --match-filter passthrough
subtitle_langs (Vec<String>) — --sub-langs CSV
extra_args (Vec<String>) — raw yt-dlp passthrough (Tartube's
`extra_cmd_string`)
- `apply(&mut Command)` appends the right flags conditionally.
- `is_empty()` lets the API layer collapse no-op writes into DELETEs so
we don't keep useless rows around.
- `from_json()` falls back to defaults on malformed rows so a schema
drift doesn't take a channel offline.
- 8 new unit tests cover apply-emits-correct-flags, JSON roundtrip,
corrupt-fallback, and is_empty.
Library
- `Channel` gains `download_options: DownloadOptions` (default empty).
- New `library::apply_channel_options(channels, map)` helper that the
caller invokes after `scan_channels` to hydrate from the bulk DB load.
- All five `scan_channels` call sites updated.
Downloader
- `Downloader::start` gains a new `channel_options: Option<&DownloadOptions>`
parameter. Applied last so per-channel overrides win over global
defaults. The explicit "submit from URL bar" flow passes None
(channel unknown until yt-dlp resolves it). Scheduled re-checks +
right-click checks pass the channel's stored options. Channel-options'
`quality` field overrides the hard-coded DownloadQuality::Best for
re-checks.
Web API
- New routes on /api/channels/:platform/:handle/options:
GET — fetch (returns defaults when no row exists)
POST — upsert (empty body collapses to DELETE)
DELETE — clear overrides
- All three update the in-memory library snapshot immediately so the
next re-check sees the change without waiting for a rescan, and bump
the library_version ETag so polling clients pick up the new state.
Web UI
- Sidebar each channel's expanded section gains a "⚙ Channel options…"
entry next to "Check for new videos".
- openChannelOptions() fetches the current options + renders a modal
with one input per field. Save / Clear / Cancel buttons. The Clear
button confirms via window.confirm() because it's destructive.
- All form fields validate locally before POSTing; empty numeric
fields map to null.
Desktop UI
- New `channel_options_window` egui::Window opened from the existing
channel right-click context menu (new "⚙ Channel options…" item).
- Form mirrors the web side: ComboBox for quality, checkbox for
audio-only, TextEdit (singleline + multiline) for the other fields.
- Save / Clear / Cancel buttons; saves go straight to SQLite and the
live `App::library` so the next scheduled-check or right-click runs
with the new overrides.
Documentation
- `docs/tartube-spec.md` checklist: per-target download options ticked
with a note describing the v1 scope.
55 unit tests pass (47 + 8 new).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four nice-to-have items, plus surfacing the upload date in both UIs
since the user explicitly asked.
Upload date display
- Desktop: card meta row now includes a `YYYY-MM-DD` cell between the
ID and the duration; detail panel header shows it as `📅 YYYY-MM-DD`.
- Web: video player modal title gains a date+duration subline so the
viewer has context without opening the metadata pane (card meta
already shows the date).
- New `format_upload_date()` helper on the desktop side; the web side
reuses the existing `fmtDate()` JS.
Extract HTML_UI to include_str!
- `LOGIN_HTML` and `HTML_UI` moved out of `src/web.rs` into
`src/web_ui/login.html` and `src/web_ui/index.html`. The Rust file
drops from 2915 lines to 1806; editors now syntax-highlight the
HTML/CSS/JS, and UI diffs no longer overwhelm the Rust diff view.
- `include_str!` bakes them into the binary at compile time, so there
is no runtime fs lookup and no extra deploy artifacts.
Twitch clips-only mode
- `classify_twitch` now recognises `twitch.tv/<user>/clips` as a
Channel URL (was Unknown), so yt-dlp's TwitchClips extractor is
reachable from our normal download pipeline.
- Both UIs gain a "Clips only" toggle visible only when the URL is
a Twitch channel (not a specific VOD/clip). On submit, the URL is
rewritten to `twitch.tv/<user>/clips` before dispatch.
- 1 new platform test covers the clips-listing classification.
DB connection pool (r2d2_sqlite)
- `Database` now wraps an `r2d2`-managed SQLite pool. File-backed
databases get up to 4 connections; the in-memory fallback is capped
at 1 (since each in-memory connection is a fresh empty DB by default
in SQLite).
- `WebState.db` drops its outer `Mutex` — concurrent handlers each
check out their own connection from the pool. Static-file fetches
through `/files/` no longer serialize on the watched/positions/
password lookups that the auth middleware performs.
- Pool init failures are converted to `rusqlite::Error` so the
existing `Result<T>` return types still work.
47 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Four items knocked off the unscoped list:
Auto-chmod 0600 cookies.txt
- web::write_cookies now sets mode 0600 on the file after writing, on Unix.
Mirrors the same guard we put on yt-offline.db. cookies.txt carries
live session credentials; no reason for it to stay umask-default.
Redact cookies path from job log
- Downloader::spawn_job now runs each stdout/stderr line through a small
redactor that replaces the absolute cookies.txt path with the bare
filename before forwarding to the Job log. yt-dlp can echo the path
in error strings, which leaks $HOME into the UI and /api/progress
responses.
- 2 new tests cover the strip + pass-through cases.
Bandcamp full-discography mode
- Bandcamp at the bare artist URL is already a Channel in our classifier
and yt-dlp's BandcampUserIE returns the full discography. The output
template for Bandcamp's channel case now organizes tracks into
per-album subfolders via `%(album|Unknown)s` so a discography pull
stays coherent instead of one flat directory.
- New `Downloader::apply_platform_extras` adds `--embed-thumbnail` for
audio-first platforms (Bandcamp, SoundCloud) so music players see the
cover art via embedded tags rather than scanning for a sidecar JPEG.
Library ETag + gzip
- WebState gains `library_version: AtomicU64`. `bump_library_version()`
is called from post_rescan, post_watched, post_maintenance_remove, and
post_resume (only when crossing the "Continue watching" >3.0 boundary
so playback-time updates don't constantly invalidate the cache).
- get_library is now Response-returning; it consults `If-None-Match` and
short-circuits with 304 Not Modified when the client's ETag matches.
Otherwise it sends `ETag: "<n>"` alongside the JSON.
- JS `loadLibrary()` caches the ETag and sends it on subsequent calls.
Returns early on 304 keeping the existing in-memory library array.
- Adds `tower_http::compression::CompressionLayer` (gzip). Already-
compressed media served from ServeDir is auto-skipped by the layer;
JSON responses get ~10× smaller. New `compression-gzip` feature on
the tower-http dep.
46 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three roadmap items together since they touch overlapping code:
Live-stream recording
- New `live: bool` argument on Downloader::start. When true, attaches
`--live-from-start --wait-for-video 30`, suppresses
`--break-on-existing` (every recording is a unique file), and
prepends a UTC timestamp suffix to the output filename so
re-recordings of the same broadcaster don't collide.
- The job label gains a "🔴 LIVE" marker so a long-running record is
obviously different from a VOD pull.
- Both UIs gain a "🔴 Live stream" checkbox in the download panel,
hidden when Music mode is selected. Works for Twitch *and* YouTube
Live since yt-dlp recognizes both from the URL.
- Web `POST /api/download` accepts a `live: bool` body field; all
other callers (scheduled re-check, right-click "Check for new
videos") pass false.
Recent-additions feed
- `library::Video` gains `mtime_unix: Option<u64>` — populated from
the metadata we already read for `file_size`, so no extra fs hit.
- New `SidebarView::Recent` (desktop) + `showRecent` (web) sort the
whole library by mtime descending and cap at 100 entries.
- Sidebar entry "🕒 Recent additions" appears only when the library
has any dated content, so a fresh install doesn't show an empty view.
Per-platform impersonation
- New `Platform::impersonate_target()`:
- Twitch → None (skip impersonation; their OAuth/Helix dislikes
mismatched TLS fingerprints).
- TikTok → "Chrome-Android-131" (mobile profile matches their
first-party app surface).
- Everything else → "Chrome-146:Macos-26".
- `Downloader::apply_impersonation` now takes the platform and routes
through the override; `repair()` (always YouTube-implicit) and
`start_music()` (uses classify_url on the URL) wire through too.
- `/api/preview` does the same classification before dispatching to
yt-dlp.
- 3 new tests confirm Twitch → None, TikTok → mobile, YouTube → desktop.
44 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous standalone PyInstaller binary couldn't run `--impersonate`
because curl_cffi isn't packaged into it. Bundled mode lost the
bot-detection bypass as a result.
Install layout is now:
~/.local/share/yt-offline/
bin/deno ← unchanged: prepended to PATH for JS deciphering
venv/bin/yt-dlp ← new: pip-installed yt-dlp[default] + curl_cffi
The install script:
- Errors clearly if python3 or the `venv` module isn't available
(the latter is split out on Debian as `python3-venv`).
- Creates the venv if missing, otherwise reuses it for upgrades.
- pip-installs `yt-dlp[default]` and `curl_cffi`, both with PyPI's
built-in SHA-256 verification.
- Cleans up the legacy `bin/yt-dlp` PyInstaller binary if a prior
install left one behind.
- Probes `--list-impersonate-targets` at the end and logs whether
curl_cffi loaded successfully.
Downloader::apply_impersonation now passes --impersonate
unconditionally — both bundled (via the venv) and system yt-dlp (via
pip-installed curl_cffi) honor it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The bundled PyInstaller binary doesn't include curl_cffi, so passing
--impersonate Chrome-146:Macos-26 makes it error out with
"Impersonate target ... is not available". System yt-dlp installed via
pip can have curl_cffi alongside it and works fine.
Centralize the decision in `Downloader::apply_impersonation` and apply
the same gate to the `/api/preview` handler.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
yt-dlp already speaks ~1,800 sites; the changes here teach the rest of
the app to route them through platform-specific output folders and
remember where each creator was downloaded from.
New `platform` module
- `Platform` enum: YouTube, TikTok, Twitch, Vimeo, Bandcamp, SoundCloud,
Odysee, Other.
- `classify_url` returns both the source platform and a `UrlKind`
(Channel / Playlist / Video / Unknown) — per-platform path parsing
for each supported host.
- `platform_root(channels_root, platform)` returns the on-disk folder
per platform. YouTube keeps the legacy `channels/` for backward
compat; the others live as siblings (`tiktok/`, `twitch/`, …).
- `write_source_url` / `read_source_url` persist the originating URL
in a `.source-url` sidecar at each creator folder so re-checks no
longer rely on the YouTube-only "UC + 22 chars" heuristic.
Downloader
- `Downloader::start` now takes `&UrlInfo` instead of `&UrlKind` and
routes the yt-dlp `-o` template into the right platform folder.
- For channel downloads it also writes `.source-url` so future
re-checks recover the exact URL.
- Output template uses `%(uploader,channel,creator|Unknown)s` so
non-YouTube sites that don't expose `channel` still get a sensible
creator folder.
- Per-platform `archive.txt` keeps cross-platform IDs (TikTok numeric
vs YouTube base64) from colliding.
- New `recheck_url(&Channel)` prefers `source_url` and falls back to
the legacy YouTube heuristic for libraries created before this
change.
Library scanner
- `scan_channels` now walks every platform's folder and tags each
`Channel` with its `platform` + `source_url`. Sort key puts the
platform first, then name, so the sidebar groups cleanly.
- `Channel` gains `platform: Platform` and `source_url: Option<String>`
fields, populated from the on-disk sidecar.
Web UI
- `WebState` gains `library_root` (= parent of channels_root). The
`/files/` ServeDir is mounted at library_root so non-YouTube videos
resolve at `/files/<platform>/<creator>/<file>`. Existing YouTube
URLs still work — they're now `/files/channels/...` instead of
`/files/...` and the server-rendered JSON updates accordingly.
- Maintenance scan + remove use `library_root` so non-YouTube content
is included.
- `ChannelInfo` JSON exposes `platform`, `platform_label`,
`platform_icon`, and `source_url`.
- Sidebar groups channels by platform with each platform's icon.
- "Check for new videos" now uses the stored `source_url` (with the
YouTube heuristic as a last-resort fallback).
- Download dialog preview shows the detected source + destination
folder.
Desktop UI
- App gains a `library_root` field mirroring the web side; maintenance
ops scan/remove against it.
- Sidebar channel labels are prefixed with the platform icon for
non-YouTube channels; tooltip carries the platform name.
- Download URL field surfaces both source platform and folder path
preview as you type.
Plex
- Non-YouTube creators get their show folder prefixed with the
platform name (e.g., `TikTok - cooluser`) so a YouTube channel and
a TikTok account with the same name don't collide.
Backward compatibility
- Existing YouTube libraries (`channels/<handle>/...`) are untouched.
New platform folders are created on first run as siblings.
- Channels without a `.source-url` (i.e., everything that pre-dates
this commit) fall through to the YouTube heuristic, preserving the
existing re-check behavior.
Tests
- 14 new `platform` tests cover URL classification per platform,
channel handle extraction, and `platform_root` layout. The old
`detect_url_kind` / `extract_after` tests move into platform.rs
since the canonical implementation lives there now.
- 41 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Statistics
- New `stats` module computes totals, top-N channels, per-year upload
histogram, and per-week download activity from the in-memory library
+ watched/positions data.
- GET /api/stats endpoint returns the full report as JSON.
- Web UI: 📊 button in the header opens a stats modal with summary
tiles, two CSS bar charts (weeks + years), and side-by-side top-N
tables.
- Desktop GUI: 📊 Stats toggle in the top bar opens a window with
the same data, rendered via egui rects (stdlib calendar math so
there's no chrono dep).
- Bar chart helper `draw_bars` is reusable across stats sections.
Source URL (AGPL §13)
- `web.source_url` is now editable in both Settings UIs — was
previously documented-only and required hand-editing config.toml.
- Settings POST persists it and the footer link updates immediately.
- Default config.toml ships with the Codeberg URL set.
SECURITY_AUDIT.md
- Re-audited the codebase as of commit 5999673; rewrote the file to
document the threat model and which adversary each defense addresses.
- Resolved-from-2026-05-17 column shows what landed: session expiry,
Secure cookie, rate-limit, body cap, CSP, X-Frame-Options,
X-Content-Type-Options, safeUrl(), SHA-256 verify, chmod 0600 db.
- Documented accepted risks (plain-HTTP LAN, cookies.txt umask,
job log stderr verbatim).
- Future-hardening checklist preserved for posterity.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bug fixes
- Desktop auto-rescan: snapshot was_running before check_notifications
updates prev_job_states, so the post-download rescan actually fires.
- web::run shutdown: replace `let _ = tx;` (which dropped the sender
immediately and short-circuited the shutdown select!) with
std::mem::forget(tx); add an idle fallback so the `-> !` is honored.
- /api/preview now resolves the bundled yt-dlp path and extends PATH
for deno discovery, matching the rest of the downloader.
- Scheduler interval clamped to >=1h on read so a manually-edited
config.toml with 0 hours can't trigger every tick.
- maintenance::is_within canonicalizes the parent + filename when the
target is missing; remove_files treats NotFound as success.
Security
- Session tokens stored with issued-at Instant and pruned past 30 day
TTL on every touch (was: unbounded HashSet).
- Login rate-limited per source IP: 5 failures → 60s lockout (429).
- Secure cookie flag added when X-Forwarded-Proto: https is present.
- 4 MiB body size cap via DefaultBodyLimit.
- Security headers middleware: CSP, X-Content-Type-Options,
X-Frame-Options, Referrer-Policy.
- JS safeUrl() defangs javascript:/vbscript:/data:text URLs before
interpolation into src= attributes.
- Bundled yt-dlp install verifies SHA-256 against the release's
SHA2-256SUMS; deno hash printed for visual inspection. Install also
reports byte counts every 3s so users see live progress.
- yt-offline.db chmod 0600 at open time on Unix.
New features
- Bundled yt-dlp + deno mode: settings toggle (System/Bundled) plus
one-click Install/Update from settings. Ensures executable bit on
every launch in case the install chmod was missed.
- Download queue with configurable max_concurrent (1–10); pending
jobs surfaced in both UIs.
- Music download mode: --extract-audio into music/<artist>/, separate
sidebar entry, /api/music + /music-files/ endpoints, inline <audio>.
- Download quality selector: Best / 1080p / 720p / 480p / 360p.
- Sort videos by Newest / Oldest (upload_date from info.json with
release_date fallback).
- Plex metadata: per-episode .nfo (title, season/episode, aired,
runtime, plot from .description), <stem>-thumb.jpg symlink, and
show-level tvshow.nfo.
- yt-dlp retry/throttle defaults: --retries 30 --fragment-retries 30
--retry-sleep linear=1:30:2 --sleep-requests 1 to ride out the
"Connection reset by peer" failures.
Performance
- password_required cached in AtomicBool; refreshed only on password
change. Eliminates a SQLite query per static-file fetch.
- Job::log → VecDeque for O(1) front-pop instead of O(n) drain.
- Library scan parallelized across cores via stdlib parallel_map.
- Web UI poll: 600ms while active, 5s when idle, 2s after errors.
- Music scan now recursive (Artist/Album/Track.opus).
- percent_encode_segment uses write! to avoid per-byte String allocs.
Code quality
- 27 unit tests covering parse_progress, parse_stem, extract_after,
detect_url_kind, srt_to_vtt, count_cookies, percent_encode_segment,
bind_mode_of, hash_password, lang_label, is_within, sanitize,
year_of, aired_date, xml_escape.
- Unified library::find_video + Channel::all_videos() iterator,
replacing three duplicated linear searches.
- Downloader::browser wired to --cookies-from-browser as a fallback
when no cookies.txt is present.
- mpv detection now matches the binary basename, not substring.
- Settings modal scroll fix; web settings expose all new fields.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Full-scan download toggle: omits --break-on-existing so all unarchived
videos are fetched regardless of order, filling gaps in partial archives
- Filter "Aborting remaining downloads" from job logs (break-on-existing noise)
- Password hash moved from config.toml to SQLite settings table; survives
git checkouts and rebuilds. config.toml and yt-offline.db added to .gitignore;
config auto-generated on first run if absent
- Channel re-check URLs derived from folder name (/@handle or /channel/UCxxx)
instead of info.json's canonical channel_url, preventing duplicate UCxxx folders
- SRT subtitles detected by library scanner and served via /api/sub-vtt/* endpoint
that converts to WebVTT on the fly; browser <track> element can now play them
- Cookie clear button in both desktop and web UI (DELETE /api/cookies)
- Per-job X close button on finished download jobs in desktop GUI
- Fix widget ID clash when multiple download jobs are shown (push_id per job)
- Watched videos excluded from Continue Watching list and badge count
- settings table added to SQLite schema (key/value store)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Update cookies.txt by pasting Netscape-format contents in web UI settings and
the desktop GUI settings window (shared write/validation in web.rs)
- Treat yt-dlp exit code 101 (--break-on-existing reached an archived video) as
a successful "up to date" outcome instead of a failed job
- Add --no-write-playlist-metafiles so channel avatars/info aren't written as
phantom "Title [CHANNEL_ID]" files
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Features:
- Port all 7 desktop themes (Dracula, Trans, Emo variants, etc.) to the web UI
- Configurable bind interface (localhost/Tailscale/LAN/all) with detected IPs,
settable from both the web UI and desktop GUI
- Optional password that gates the whole UI and all API access via session
cookies (Argon2), with a login page, logout, and middleware; settable from
both UIs
- Library maintenance: scan for duplicate video IDs and missing assets
(thumbnail/info.json/description), remove duplicates (scan-and-confirm, with
out-of-library path guard), and re-fetch missing sidecars via yt-dlp.
Surfaced in both the web UI and a desktop GUI window.
Optimizations / cleanup:
- Cache has_chapters at scan time (library.rs) instead of re-reading and
parsing every info.json on each /api/library request
- Use the cached channel size in get_library
- Pool a single SQLite connection in WebState instead of opening one per request
- Remove a 1.2 GB stray nested src/yt-offline duplicate tree
Fixes:
- Rescan no longer clears all thumbnail textures (caused thumbnails to unload);
prune only removed entries and bump library_generation so the card grid
refreshes correctly
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Web UI:
- Replace SSE with simple 600ms polling (GET /api/progress) — SSE added
complexity with no real benefit for a local tool and was the source of
connection failures
- Poll task moved into request handler (dl.poll() on each /api/progress call)
so no background goroutine needed; simplifies state greatly
- Remove tokio-stream dependency (no longer needed)
- Fix: create channels dir before opening DB in web serve()
- Rescan also refreshes watched set from DB
- Bulk watched, sort, channel download button all working in web UI
Sort labels:
- Rename "Dur ↑" / "Dur ↓" → "Shortest" / "Longest"
- Rename "Size ↑" / "Size ↓" → "Smallest" / "Largest"
Channel right-click context menu:
- Right-click any channel in sidebar → "Check for new videos" starts a
yt-dlp job on the stored channel_url from its info.json
- Also shows "Open folder" shortcut
- Grayed out with tooltip if no URL is stored yet (channel has no info.json)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
License:
- GPL3 → AGPL3
New features:
- Web interface (--web [port]): axum/tokio server with SSE progress, library browser,
download trigger, watched toggle; embedded single-page HTML/JS UI
- Desktop notifications via notify-rust when downloads complete or fail
- Continue Watching sidebar entry showing videos with saved resume positions
- Bulk select mode: select multiple videos, mark all watched/unwatched at once
- Storage stats: channel disk usage shown in sidebar next to video count
- Scheduled channel checks: configurable interval (hours), auto re-downloads all
tracked channels using channel_url from info.json; enabled in Settings
Theme fixes:
- Scene Queen: remove override_text_color so active button text uses fg_stroke
(dark navy on neon green) instead of near-white — fixes unreadable contrast
- Trans: override_text_color → hot pink #cc0066 for pink-tinted text throughout
Settings additions:
- Auto-check channels toggle + interval_hours DragValue
- Web UI port setting
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
