Pull the parity checklist in line with what actually shipped this session.
- Profiles: marked [x] partial — folders double as profiles via the
per-folder "Check all" action wired in c86e9b9.
- Manual chapter / SponsorBlock editing: marked [x] deferred — comments
capture covered the more useful sibling feature; manual chapter
editing remains a low-priority ask.
- extra_args passthrough: marked [x] — shipped as part of Phase 1.1
DownloadOptions.
- Tidy + Refresh: clarified the remaining work (rolling phantom + disk
detection into the existing maintenance UI).
Added a "Beyond Tartube" section listing the surpass-toehold UX
shipped this session that Tartube has no equivalent for:
- DB-snapshot download.
- Bulk tagging (Favourite / Bookmark / Waiting).
- Channel-name search.
- Keyboard shortcuts + help dialog.
- Persisted UI state across reloads.
- 🎲 Shuffle — random unwatched pick.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
🎲 button in the header picks a random video from
{unwatched + downloaded + has video_url} and opens it directly in the
player. Falls through to "any downloaded video" when everything is
already watched, with a status message that lets the user know.
Tartube has nothing like it — small bit of extra "surpass" beyond
plain parity.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reloading the tab no longer blows away the user's context. The active
view (channels / continue / recent / favourites / bookmarks / waiting /
specific channel / specific playlist), the current filter input, and
the chosen sort all survive a hard refresh via localStorage.
- New `saveUiState()` serializes `{view, search, sort}` after every
interaction that could change them: each setX() view switch, search
oninput, sort onchange.
- `currentViewToken()` collapses the seven `showX` booleans + the two
index integers into a single string. Channel-view tokens reference
the channel by name (`channel:<name>|<playlistIdx>`) instead of by
array index, so a library re-order doesn't strand the user on the
wrong channel.
- `restoreUiState()` runs once after the first library load completes —
the channel resolution needs the `library` array populated, so the
init block was wrapped in an async IIFE to sequence the two steps.
Channels deleted between sessions fall back to "All" cleanly because
`findIndex` returns -1 and we just leave the show* booleans at their
default.
55 unit tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four shortcuts: `/` focuses the filter input, `r` triggers a library
rescan, `Esc` closes the topmost modal, `?` opens a help dialog
listing the bindings. All disabled while the focus is inside a text
input / textarea / contentEditable element so typing a video title
doesn't accidentally fire actions — except Esc, which always closes
modals (even from an input).
A new `?` button next to the existing rescan / maintenance / settings
icons makes the shortcuts discoverable; the help dialog is plain HTML
table with monospace key chips.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bulk tagging
- Desktop bulk-mode toolbar grows three new buttons next to ✓ Watched
and ○ Unwatched: ★ Favourite, 🔖 Bookmark, ⏳ Waiting. Each applies
the corresponding flag to every video in the current bulk selection.
- New `App::bulk_set_flag` mirrors the existing `bulk_mark_watched`:
iterates selection, persists each flag via `Database::set_video_flag`,
mirrors into the in-memory `flags` bundle, busts the cards-cache so
the smart-folder counts refresh on the next render.
- Web bulk-actions row gets the same three buttons. New `bulkFlag(flag)`
fires the POSTs in parallel via Promise.all and reloads the library
once they complete.
- Tagging in bulk only ever sets a flag — no toggle. Removing flags in
bulk is rarely what users want; the per-card buttons handle un-flag.
Channel-name search
- Both UIs already filter by video title + id; extend to match the
channel name too. Searching "Linus" now surfaces every video from a
channel called "Linus Tech Tips" without typing the title.
- Description matching deliberately skipped — would require reading the
.description sidecar per-video per-keystroke. Mentioned in the code
comment as a future "load descriptions into the search index on
rescan" pass if real users want it.
- New JS helper `matchesSearch(v, chName, q)` centralises the four
call sites in `currentVideos()` (continue / recent / smart-folder /
default).
55 unit tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two small follow-ups to the folder hierarchy that round out the
folder-as-a-profile workflow and give the user a one-click way to keep
an offsite copy of their library state.
Per-folder check
- New `POST /api/folders/:id/check` mirrors the existing
/api/scheduler/run endpoint but scoped to one folder's members.
Each channel's stored DownloadOptions (quality / rate limit /
match-filter / extra args) applies just like a scheduled re-check.
Returns 409 when downloads are already running.
- Web folder-manager modal: each non-empty folder gets a "⬇ Check"
button next to Rename / Delete. Status line reports the count of
channels queued.
- Desktop folder-manager window: same — a small ⬇ icon-button per row,
visible only for folders with at least one member. Same options-
honouring re-check path as the right-click "Check for new videos"
action.
- This effectively gives us "Profiles" lite: organize channels into a
folder, then trigger a batch re-check whenever you want.
DB backup
- New `GET /api/backup/db` reads `<channels_root>/yt-offline.db` and
streams it back with `Content-Disposition: attachment` so the
browser downloads instead of rendering. Filename includes the unix
timestamp so downloads don't clobber. Returns 404 when running in
in-memory mode (no DB file to dump).
- We deliberately don't bundle config.toml + cookies.txt here:
config is short to recreate and cookies.txt carries live session
credentials that shouldn't fly over the wire unprompted.
- Web settings modal gains a "⬇ Download library snapshot" button in
a new Backup section, with a hint explaining what's covered (watched
/ favourites / bookmarks / waiting / channel-options / folders).
55 unit tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Users can now organize channels into named folders independent of the
underlying platform groupings. v1 ships single-level (no nesting) since
that's where ~90% of the organizational value lives — nested folders +
drag-to-reparent + per-folder cascading options are deferred to v2 once
we have real-world feedback on what the workflow needs.
Data layer
- New `folders` SQLite table: id, name (UNIQUE), position, created_at.
- New `channel_assignments` table: (platform, handle) → folder_id with
ON DELETE CASCADE so deleting a folder cleanly unfiles its channels.
- Database methods: create_folder, rename_folder, delete_folder (with
PRAGMA foreign_keys ON for the cascade), list_folders,
set_channel_folder, get_all_channel_assignments.
- New `FolderRecord` struct (id/name/position) is `Serialize` so it
rides on the /api/library response.
Library
- `Channel` gains `folder_id: Option<i64>` (None = Unfiled).
- New `library::apply_channel_folders` mirror of `apply_channel_options`
populates it from the bulk DB map after every scan / rescan.
Web API
- `POST /api/folders` create with `{name}` → `{id}`.
- `POST /api/folders/:id/rename` rename with `{name}`.
- `DELETE /api/folders/:id` cascade-deletes assignments.
- `POST /api/channels/:platform/:handle/folder` upsert/clear via
`{folder_id: <i64|null>}`. All endpoints bump the library ETag.
- `/api/library` response now carries a `folders: [...]` array so the
client can render the sidebar grouping without a second round trip.
Web UI
- Sidebar gains a "📁 Folders" section above the platform groupings.
Each folder lists its member channels (with platform icon prefix);
channels with `folder_id` are pulled out of their platform section
to avoid duplication. Unfiled channels keep platform grouping.
- "manage" / "new" button next to the Folders heading opens a modal
with the full folder list (member counts inline) plus rename /
delete buttons and a Create input.
- New per-channel sub-action "📁 Move to folder…" opens a small
picker modal with one row per folder + "— Unfiled —". Current
assignment highlighted.
Desktop UI
- Sidebar refactored to build a `Vec<SidebarItem>` ordered list
(FolderHeader / FolderManageBtn / PlatformHeader / Channel) and
iterate it once, instead of the previous flat
`for i in 0..library.len()` loop. Folder section renders above
the platform sections; channels with `folder_id` are skipped from
the platform-grouped loop so they appear in exactly one place.
- New `folder_manager_window` + `move_to_folder_window` egui windows
cover create / rename (via the create-buffer field) / delete and
per-channel folder picking. Right-click context menu on a channel
gains "📁 Move to folder…".
- App struct gains: `folders`, `show_folder_manager`,
`folder_create_buffer`, `show_move_to_folder`, `move_to_folder_target`.
Tartube spec checklist
- Folder hierarchy box ticked (with the v2 deferrals noted in the
inline annotation).
55 unit tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three Tartube-parity items shipped together since they touch the same
schema and rendering paths.
Per-video state flags
- New `video_flags` SQLite table with `bookmark`/`favourite`/`waiting`/
`archive` columns (one row per video that has at least one flag set).
- `Database::set_video_flag` validates the flag name against an
allow-list before interpolating into the SQL — column names can't be
parameterised in SQLite, but the allow-list keeps it injection-free.
- `Database::get_video_flags` bulk-loads into a new
`VideoFlagsBundle { bookmark, favourite, waiting, archive: HashSet<String> }`
so subsequent reads are O(1) in-memory hits.
- `WebState.flags` (Mutex<VideoFlagsBundle>) + `App.flags`
(VideoFlagsBundle, GUI is single-threaded) populated at startup.
- `POST /api/videos/:id/flags/:flag` toggles a flag; the in-memory
bundle is mirrored immediately so the next /api/library reflects the
change without a rescan. Library ETag is bumped.
- `VideoInfo` JSON gains `bookmark` / `favourite` / `waiting` /
`archive` boolean fields.
- `archive` is wired through everywhere but no UI gate yet — it's
reserved for the future auto-delete feature.
Smart folders
- Both UIs gain "★ Favourites", "🔖 Bookmarks", "⏳ Waiting" sidebar
entries between Continue Watching and Channels. Each is hidden when
its set is empty, so a fresh install isn't polluted with zero-count
rows.
- Desktop: new `SidebarView::Favourites / Bookmarks / Waiting`
variants. `compute_cards` filters the library by the matching
`self.flags.<set>` HashSet and returns the resulting cards using the
same rendering path as the other views.
- Web: `showFavourites/Bookmarks/Waiting` flags + smart-folder branch
in `currentVideos()`. Refactored the view-mode selectors through a
new `resetViewSelectors()` helper so adding three more booleans
didn't quadruple every `setX` function.
Per-card flag toggles
- Web: each video card grows three new action buttons (☆/★ favourite,
🔖 bookmark, ⏳ waiting) next to the existing watched toggle.
Optimistic UI — flip in-memory immediately, undo on server error.
- Desktop: same three buttons next to Watched in the card row, deferred
through a `toggle_flag_card: Option<&'static str>` to keep the
borrow checker happy. New `App::toggle_video_flag` helper persists
the flip to SQLite and invalidates the card cache.
Comments capture
- `DownloadOptions` gains `fetch_comments: bool`. When set, the
downloader emits `--write-comments` so yt-dlp embeds the full
comment tree into info.json.
- New `GET /api/comments/:id` reads the array out of info.json and
returns a slimmed-down version (id, author, text, likes, parent,
time) — strips the multi-MB extras yt-dlp can otherwise inline.
- Web player modal grows a 💬 button that opens a separate modal
with a threaded viewer: replies indented by parent id, "N comments"
header, helpful message when nothing's captured pointing at the
channel-options toggle.
- Channel-options dialog in both UIs gains a "Fetch comments"
checkbox + hint about the slowdown.
Tartube spec checklist
- Three more boxes ticked: per-video state flags, smart folders,
comments capture. Score now: 5 ahead, 5 behind, 1 tied + matching
on 4 parity items. (Half done.)
55 unit tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the single largest gap with Tartube documented in docs/tartube-spec.md.
Users can now attach a small set of yt-dlp overrides to any individual
channel; the overrides apply automatically during scheduled re-checks
and the right-click "Check for new videos" action.
Data layer
- New `channel_options` SQLite table keyed on (platform, handle), with
`options_json TEXT` blob + updated_at. PK ensures one row per channel.
- Database gains get_channel_options / set_channel_options /
delete_channel_options / get_all_channel_options. The bulk getter is
used by the library scanner so each rescan hydrates options without
per-channel SQL round-trips.
- `DownloadQuality` derives Serialize/Deserialize so it can roundtrip
through the options blob.
New module: src/download_options.rs
- `DownloadOptions` struct with 9 fields:
quality (Option<DownloadQuality>) — per-channel quality cap
audio_only (bool) — force --extract-audio chain
limit_rate_kb (Option<u32>) — --limit-rate
min_filesize_mb / max_filesize_mb — yt-dlp filesize filters
date_after (Option<String>) — --dateafter YYYYMMDD
match_filter (Option<String>) — yt-dlp --match-filter passthrough
subtitle_langs (Vec<String>) — --sub-langs CSV
extra_args (Vec<String>) — raw yt-dlp passthrough (Tartube's
`extra_cmd_string`)
- `apply(&mut Command)` appends the right flags conditionally.
- `is_empty()` lets the API layer collapse no-op writes into DELETEs so
we don't keep useless rows around.
- `from_json()` falls back to defaults on malformed rows so a schema
drift doesn't take a channel offline.
- 8 new unit tests cover apply-emits-correct-flags, JSON roundtrip,
corrupt-fallback, and is_empty.
Library
- `Channel` gains `download_options: DownloadOptions` (default empty).
- New `library::apply_channel_options(channels, map)` helper that the
caller invokes after `scan_channels` to hydrate from the bulk DB load.
- All five `scan_channels` call sites updated.
Downloader
- `Downloader::start` gains a new `channel_options: Option<&DownloadOptions>`
parameter. Applied last so per-channel overrides win over global
defaults. The explicit "submit from URL bar" flow passes None
(channel unknown until yt-dlp resolves it). Scheduled re-checks +
right-click checks pass the channel's stored options. Channel-options'
`quality` field overrides the hard-coded DownloadQuality::Best for
re-checks.
Web API
- New routes on /api/channels/:platform/:handle/options:
GET — fetch (returns defaults when no row exists)
POST — upsert (empty body collapses to DELETE)
DELETE — clear overrides
- All three update the in-memory library snapshot immediately so the
next re-check sees the change without waiting for a rescan, and bump
the library_version ETag so polling clients pick up the new state.
Web UI
- Sidebar each channel's expanded section gains a "⚙ Channel options…"
entry next to "Check for new videos".
- openChannelOptions() fetches the current options + renders a modal
with one input per field. Save / Clear / Cancel buttons. The Clear
button confirms via window.confirm() because it's destructive.
- All form fields validate locally before POSTing; empty numeric
fields map to null.
Desktop UI
- New `channel_options_window` egui::Window opened from the existing
channel right-click context menu (new "⚙ Channel options…" item).
- Form mirrors the web side: ComboBox for quality, checkbox for
audio-only, TextEdit (singleline + multiline) for the other fields.
- Save / Clear / Cancel buttons; saves go straight to SQLite and the
live `App::library` so the next scheduled-check or right-click runs
with the new overrides.
Documentation
- `docs/tartube-spec.md` checklist: per-target download options ticked
with a note describing the v1 scope.
55 unit tests pass (47 + 8 new).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
To make the "surpass Tartube" roadmap concrete, document Tartube's
v2.5.231 codebase as a specification we can refer to:
- Module map: 146k lines of Python across 19 modules, with per-module
responsibilities (mainapp / mainwin / config / downloads / media /
options / wizwin / process / refresh / tidy / info / updates /
ffmpeg_tartube / formats / ttutils / dialogue / files / xdg).
- Data model: container hierarchy (Folder → Channel | Playlist → Video)
with N-level Folder nesting, system folders that act as smart filters
(Bookmarks, Favourites, Waiting, New, Live, Missing, Recent,
Temporary), and the full per-video flag inventory.
- OptionsManager: the headline feature — 164 distinct yt-dlp flags
attachable to any container or video with cascading resolution. The
spec lists the option groups (Behaviour, Network, Playlist, Filters,
Download, Filesystem, Auth, Anti-bot, Format, Subtitles,
Post-processing, Cookies, Output, Tartube-specific) and points to a
concrete SQLite + Rust struct design for our Phase 1.1.
- Seven threaded operation managers (Download, Update, Refresh, Tidy,
Info, Process, Livestream/Stream) and three downloader strategies
(VideoDownloader, ClipDownloader, StreamDownloader).
- Custom downloads (named recipes) and profiles (saved container
groups) — feeding our Phase 1 + 3.
- UI surface inventory: five primary tabs (Videos, Progress, Classic
Mode, Drag and Drop, Errors), ~80 specialised dialogs, three
wizards (Setup, Import YT subs, Tutorial), and the ~30-tab
SystemPrefWin.
- SponsorBlock + chapter persistence (slice_list / stamp_list) — a
Phase 1 add-on after per-channel options ship.
- Notable corners: master/slave dbid dedup, RSS-driven discovery,
external_dir per-channel storage, livestream message parsing, full
gettext i18n.
- Things Tartube does better (15 items, mapped to roadmap entries),
things we already do better (10 items, to keep momentum honest).
- A 12-checkbox "Tartube parity" milestone definition — when all are
ticked, the comparison table at the top of ROADMAP.md goes from
8-ahead-8-behind to 8-ahead-0-behind, and "surpass" begins.
ROADMAP.md gains a pointer to the spec just under the north-star line.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Tartube is the mature open-source yt-dlp GUI in this space and the
obvious benchmark. As of today we're 8-ahead / 8-behind / 1-tied across
the comparison matrix at the top of the new ROADMAP.md.
Four phases:
1. Tartube feature parity — the 8 items they have that we don't, ordered
by user-visible impact. Per-channel custom download options is the
single biggest gap; everything else (folder hierarchy, filter UI,
comments capture, notes, system tray, format conversion pipeline,
per-distro packaging) follows.
2. Polish where Tartube is mature — integration tests, docs site,
structured error recovery / "rescue recipes", backup-restore, stability
hardening.
3. Surpass — cross-compile mac+win, Android client, WebSocket-driven
progress, smart auto-tagging, federation, a real comment viewer,
perceptual-hash dedup, plugin/scripting hooks.
4. Stretch / blue-sky — TV-mode layout, AI summarisation, multi-user,
Plex/Jellyfin source-plugin integration.
The architectural wins we already have (single Rust binary, real web
UI, bundled curl_cffi venv, Plex export with NFO sidecars, Argon2 auth +
CSP + rate-limit, 10 themes, ETag+gzip on the library response) are
called out so we keep building on them rather than re-litigating them.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four nice-to-have items, plus surfacing the upload date in both UIs
since the user explicitly asked.
Upload date display
- Desktop: card meta row now includes a `YYYY-MM-DD` cell between the
ID and the duration; detail panel header shows it as `📅 YYYY-MM-DD`.
- Web: video player modal title gains a date+duration subline so the
viewer has context without opening the metadata pane (card meta
already shows the date).
- New `format_upload_date()` helper on the desktop side; the web side
reuses the existing `fmtDate()` JS.
Extract HTML_UI to include_str!
- `LOGIN_HTML` and `HTML_UI` moved out of `src/web.rs` into
`src/web_ui/login.html` and `src/web_ui/index.html`. The Rust file
drops from 2915 lines to 1806; editors now syntax-highlight the
HTML/CSS/JS, and UI diffs no longer overwhelm the Rust diff view.
- `include_str!` bakes them into the binary at compile time, so there
is no runtime fs lookup and no extra deploy artifacts.
Twitch clips-only mode
- `classify_twitch` now recognises `twitch.tv/<user>/clips` as a
Channel URL (was Unknown), so yt-dlp's TwitchClips extractor is
reachable from our normal download pipeline.
- Both UIs gain a "Clips only" toggle visible only when the URL is
a Twitch channel (not a specific VOD/clip). On submit, the URL is
rewritten to `twitch.tv/<user>/clips` before dispatch.
- 1 new platform test covers the clips-listing classification.
DB connection pool (r2d2_sqlite)
- `Database` now wraps an `r2d2`-managed SQLite pool. File-backed
databases get up to 4 connections; the in-memory fallback is capped
at 1 (since each in-memory connection is a fresh empty DB by default
in SQLite).
- `WebState.db` drops its outer `Mutex` — concurrent handlers each
check out their own connection from the pool. Static-file fetches
through `/files/` no longer serialize on the watched/positions/
password lookups that the auth middleware performs.
- Pool init failures are converted to `rusqlite::Error` so the
existing `Result<T>` return types still work.
47 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Four items knocked off the unscoped list:
Auto-chmod 0600 cookies.txt
- web::write_cookies now sets mode 0600 on the file after writing, on Unix.
Mirrors the same guard we put on yt-offline.db. cookies.txt carries
live session credentials; no reason for it to stay umask-default.
Redact cookies path from job log
- Downloader::spawn_job now runs each stdout/stderr line through a small
redactor that replaces the absolute cookies.txt path with the bare
filename before forwarding to the Job log. yt-dlp can echo the path
in error strings, which leaks $HOME into the UI and /api/progress
responses.
- 2 new tests cover the strip + pass-through cases.
Bandcamp full-discography mode
- Bandcamp at the bare artist URL is already a Channel in our classifier
and yt-dlp's BandcampUserIE returns the full discography. The output
template for Bandcamp's channel case now organizes tracks into
per-album subfolders via `%(album|Unknown)s` so a discography pull
stays coherent instead of one flat directory.
- New `Downloader::apply_platform_extras` adds `--embed-thumbnail` for
audio-first platforms (Bandcamp, SoundCloud) so music players see the
cover art via embedded tags rather than scanning for a sidecar JPEG.
Library ETag + gzip
- WebState gains `library_version: AtomicU64`. `bump_library_version()`
is called from post_rescan, post_watched, post_maintenance_remove, and
post_resume (only when crossing the "Continue watching" >3.0 boundary
so playback-time updates don't constantly invalidate the cache).
- get_library is now Response-returning; it consults `If-None-Match` and
short-circuits with 304 Not Modified when the client's ETag matches.
Otherwise it sends `ETag: "<n>"` alongside the JSON.
- JS `loadLibrary()` caches the ETag and sends it on subsequent calls.
Returns early on 304 keeping the existing in-memory library array.
- Adds `tower_http::compression::CompressionLayer` (gzip). Already-
compressed media served from ServeDir is auto-skipped by the layer;
JSON responses get ~10× smaller. New `compression-gzip` feature on
the tower-http dep.
46 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three roadmap items together since they touch overlapping code:
Live-stream recording
- New `live: bool` argument on Downloader::start. When true, attaches
`--live-from-start --wait-for-video 30`, suppresses
`--break-on-existing` (every recording is a unique file), and
prepends a UTC timestamp suffix to the output filename so
re-recordings of the same broadcaster don't collide.
- The job label gains a "🔴 LIVE" marker so a long-running record is
obviously different from a VOD pull.
- Both UIs gain a "🔴 Live stream" checkbox in the download panel,
hidden when Music mode is selected. Works for Twitch *and* YouTube
Live since yt-dlp recognizes both from the URL.
- Web `POST /api/download` accepts a `live: bool` body field; all
other callers (scheduled re-check, right-click "Check for new
videos") pass false.
Recent-additions feed
- `library::Video` gains `mtime_unix: Option<u64>` — populated from
the metadata we already read for `file_size`, so no extra fs hit.
- New `SidebarView::Recent` (desktop) + `showRecent` (web) sort the
whole library by mtime descending and cap at 100 entries.
- Sidebar entry "🕒 Recent additions" appears only when the library
has any dated content, so a fresh install doesn't show an empty view.
Per-platform impersonation
- New `Platform::impersonate_target()`:
- Twitch → None (skip impersonation; their OAuth/Helix dislikes
mismatched TLS fingerprints).
- TikTok → "Chrome-Android-131" (mobile profile matches their
first-party app surface).
- Everything else → "Chrome-146:Macos-26".
- `Downloader::apply_impersonation` now takes the platform and routes
through the override; `repair()` (always YouTube-implicit) and
`start_music()` (uses classify_url on the URL) wire through too.
- `/api/preview` does the same classification before dispatching to
yt-dlp.
- 3 new tests confirm Twitch → None, TikTok → mobile, YouTube → desktop.
44 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The README still described yt-offline as a YouTube-only desktop app.
Rewrite it to reflect what actually exists today:
- Multi-platform support (YouTube, TikTok, Twitch, Vimeo, Bandcamp,
SoundCloud, Odysee, and yt-dlp's full fallback list) with a table
of what each platform supports.
- Library, playback, downloads, web server, Plex integration, themes,
and misc sections — each grouping the relevant features so a reader
can spot what matters to them.
- Bundled-yt-dlp install walkthrough (venv + curl_cffi + deno).
- Updated config.toml example with every field the app reads.
- Library layout diagram showing the new platform sibling folders
and the `.source-url` sidecar.
- Usage section split between desktop GUI and web UI.
- Troubleshooting now lists the impersonation error path, missing
python3-venv, the transcoding-seek caveat, and rate-limit recovery.
- Pointer to SECURITY_AUDIT.md.
- AGPL §13 obligations spelled out.
Also expand the Cargo.toml description so `cargo metadata` and crates.io
listings convey the multi-platform / web UI scope at a glance.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous standalone PyInstaller binary couldn't run `--impersonate`
because curl_cffi isn't packaged into it. Bundled mode lost the
bot-detection bypass as a result.
Install layout is now:
~/.local/share/yt-offline/
bin/deno ← unchanged: prepended to PATH for JS deciphering
venv/bin/yt-dlp ← new: pip-installed yt-dlp[default] + curl_cffi
The install script:
- Errors clearly if python3 or the `venv` module isn't available
(the latter is split out on Debian as `python3-venv`).
- Creates the venv if missing, otherwise reuses it for upgrades.
- pip-installs `yt-dlp[default]` and `curl_cffi`, both with PyPI's
built-in SHA-256 verification.
- Cleans up the legacy `bin/yt-dlp` PyInstaller binary if a prior
install left one behind.
- Probes `--list-impersonate-targets` at the end and logs whether
curl_cffi loaded successfully.
Downloader::apply_impersonation now passes --impersonate
unconditionally — both bundled (via the venv) and system yt-dlp (via
pip-installed curl_cffi) honor it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The bundled PyInstaller binary doesn't include curl_cffi, so passing
--impersonate Chrome-146:Macos-26 makes it error out with
"Impersonate target ... is not available". System yt-dlp installed via
pip can have curl_cffi alongside it and works fine.
Centralize the decision in `Downloader::apply_impersonation` and apply
the same gate to the `/api/preview` handler.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
yt-dlp already speaks ~1,800 sites; the changes here teach the rest of
the app to route them through platform-specific output folders and
remember where each creator was downloaded from.
New `platform` module
- `Platform` enum: YouTube, TikTok, Twitch, Vimeo, Bandcamp, SoundCloud,
Odysee, Other.
- `classify_url` returns both the source platform and a `UrlKind`
(Channel / Playlist / Video / Unknown) — per-platform path parsing
for each supported host.
- `platform_root(channels_root, platform)` returns the on-disk folder
per platform. YouTube keeps the legacy `channels/` for backward
compat; the others live as siblings (`tiktok/`, `twitch/`, …).
- `write_source_url` / `read_source_url` persist the originating URL
in a `.source-url` sidecar at each creator folder so re-checks no
longer rely on the YouTube-only "UC + 22 chars" heuristic.
Downloader
- `Downloader::start` now takes `&UrlInfo` instead of `&UrlKind` and
routes the yt-dlp `-o` template into the right platform folder.
- For channel downloads it also writes `.source-url` so future
re-checks recover the exact URL.
- Output template uses `%(uploader,channel,creator|Unknown)s` so
non-YouTube sites that don't expose `channel` still get a sensible
creator folder.
- Per-platform `archive.txt` keeps cross-platform IDs (TikTok numeric
vs YouTube base64) from colliding.
- New `recheck_url(&Channel)` prefers `source_url` and falls back to
the legacy YouTube heuristic for libraries created before this
change.
Library scanner
- `scan_channels` now walks every platform's folder and tags each
`Channel` with its `platform` + `source_url`. Sort key puts the
platform first, then name, so the sidebar groups cleanly.
- `Channel` gains `platform: Platform` and `source_url: Option<String>`
fields, populated from the on-disk sidecar.
Web UI
- `WebState` gains `library_root` (= parent of channels_root). The
`/files/` ServeDir is mounted at library_root so non-YouTube videos
resolve at `/files/<platform>/<creator>/<file>`. Existing YouTube
URLs still work — they're now `/files/channels/...` instead of
`/files/...` and the server-rendered JSON updates accordingly.
- Maintenance scan + remove use `library_root` so non-YouTube content
is included.
- `ChannelInfo` JSON exposes `platform`, `platform_label`,
`platform_icon`, and `source_url`.
- Sidebar groups channels by platform with each platform's icon.
- "Check for new videos" now uses the stored `source_url` (with the
YouTube heuristic as a last-resort fallback).
- Download dialog preview shows the detected source + destination
folder.
Desktop UI
- App gains a `library_root` field mirroring the web side; maintenance
ops scan/remove against it.
- Sidebar channel labels are prefixed with the platform icon for
non-YouTube channels; tooltip carries the platform name.
- Download URL field surfaces both source platform and folder path
preview as you type.
Plex
- Non-YouTube creators get their show folder prefixed with the
platform name (e.g., `TikTok - cooluser`) so a YouTube channel and
a TikTok account with the same name don't collide.
Backward compatibility
- Existing YouTube libraries (`channels/<handle>/...`) are untouched.
New platform folders are created on first run as siblings.
- Channels without a `.source-url` (i.e., everything that pre-dates
this commit) fall through to the YouTube heuristic, preserving the
existing re-check behavior.
Tests
- 14 new `platform` tests cover URL classification per platform,
channel handle extraction, and `platform_root` layout. The old
`detect_url_kind` / `extract_after` tests move into platform.rs
since the canonical implementation lives there now.
- 41 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Statistics
- New `stats` module computes totals, top-N channels, per-year upload
histogram, and per-week download activity from the in-memory library
+ watched/positions data.
- GET /api/stats endpoint returns the full report as JSON.
- Web UI: 📊 button in the header opens a stats modal with summary
tiles, two CSS bar charts (weeks + years), and side-by-side top-N
tables.
- Desktop GUI: 📊 Stats toggle in the top bar opens a window with
the same data, rendered via egui rects (stdlib calendar math so
there's no chrono dep).
- Bar chart helper `draw_bars` is reusable across stats sections.
Source URL (AGPL §13)
- `web.source_url` is now editable in both Settings UIs — was
previously documented-only and required hand-editing config.toml.
- Settings POST persists it and the footer link updates immediately.
- Default config.toml ships with the Codeberg URL set.
SECURITY_AUDIT.md
- Re-audited the codebase as of commit 5999673; rewrote the file to
document the threat model and which adversary each defense addresses.
- Resolved-from-2026-05-17 column shows what landed: session expiry,
Secure cookie, rate-limit, body cap, CSP, X-Frame-Options,
X-Content-Type-Options, safeUrl(), SHA-256 verify, chmod 0600 db.
- Documented accepted risks (plain-HTTP LAN, cookies.txt umask,
job log stderr verbatim).
- Future-hardening checklist preserved for posterity.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bug fixes
- Desktop auto-rescan: snapshot was_running before check_notifications
updates prev_job_states, so the post-download rescan actually fires.
- web::run shutdown: replace `let _ = tx;` (which dropped the sender
immediately and short-circuited the shutdown select!) with
std::mem::forget(tx); add an idle fallback so the `-> !` is honored.
- /api/preview now resolves the bundled yt-dlp path and extends PATH
for deno discovery, matching the rest of the downloader.
- Scheduler interval clamped to >=1h on read so a manually-edited
config.toml with 0 hours can't trigger every tick.
- maintenance::is_within canonicalizes the parent + filename when the
target is missing; remove_files treats NotFound as success.
Security
- Session tokens stored with issued-at Instant and pruned past 30 day
TTL on every touch (was: unbounded HashSet).
- Login rate-limited per source IP: 5 failures → 60s lockout (429).
- Secure cookie flag added when X-Forwarded-Proto: https is present.
- 4 MiB body size cap via DefaultBodyLimit.
- Security headers middleware: CSP, X-Content-Type-Options,
X-Frame-Options, Referrer-Policy.
- JS safeUrl() defangs javascript:/vbscript:/data:text URLs before
interpolation into src= attributes.
- Bundled yt-dlp install verifies SHA-256 against the release's
SHA2-256SUMS; deno hash printed for visual inspection. Install also
reports byte counts every 3s so users see live progress.
- yt-offline.db chmod 0600 at open time on Unix.
New features
- Bundled yt-dlp + deno mode: settings toggle (System/Bundled) plus
one-click Install/Update from settings. Ensures executable bit on
every launch in case the install chmod was missed.
- Download queue with configurable max_concurrent (1–10); pending
jobs surfaced in both UIs.
- Music download mode: --extract-audio into music/<artist>/, separate
sidebar entry, /api/music + /music-files/ endpoints, inline <audio>.
- Download quality selector: Best / 1080p / 720p / 480p / 360p.
- Sort videos by Newest / Oldest (upload_date from info.json with
release_date fallback).
- Plex metadata: per-episode .nfo (title, season/episode, aired,
runtime, plot from .description), <stem>-thumb.jpg symlink, and
show-level tvshow.nfo.
- yt-dlp retry/throttle defaults: --retries 30 --fragment-retries 30
--retry-sleep linear=1:30:2 --sleep-requests 1 to ride out the
"Connection reset by peer" failures.
Performance
- password_required cached in AtomicBool; refreshed only on password
change. Eliminates a SQLite query per static-file fetch.
- Job::log → VecDeque for O(1) front-pop instead of O(n) drain.
- Library scan parallelized across cores via stdlib parallel_map.
- Web UI poll: 600ms while active, 5s when idle, 2s after errors.
- Music scan now recursive (Artist/Album/Track.opus).
- percent_encode_segment uses write! to avoid per-byte String allocs.
Code quality
- 27 unit tests covering parse_progress, parse_stem, extract_after,
detect_url_kind, srt_to_vtt, count_cookies, percent_encode_segment,
bind_mode_of, hash_password, lang_label, is_within, sanitize,
year_of, aired_date, xml_escape.
- Unified library::find_video + Channel::all_videos() iterator,
replacing three duplicated linear searches.
- Downloader::browser wired to --cookies-from-browser as a fallback
when no cookies.txt is present.
- mpv detection now matches the binary basename, not substring.
- Settings modal scroll fix; web settings expose all new fields.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Full-scan download toggle: omits --break-on-existing so all unarchived
videos are fetched regardless of order, filling gaps in partial archives
- Filter "Aborting remaining downloads" from job logs (break-on-existing noise)
- Password hash moved from config.toml to SQLite settings table; survives
git checkouts and rebuilds. config.toml and yt-offline.db added to .gitignore;
config auto-generated on first run if absent
- Channel re-check URLs derived from folder name (/@handle or /channel/UCxxx)
instead of info.json's canonical channel_url, preventing duplicate UCxxx folders
- SRT subtitles detected by library scanner and served via /api/sub-vtt/* endpoint
that converts to WebVTT on the fly; browser <track> element can now play them
- Cookie clear button in both desktop and web UI (DELETE /api/cookies)
- Per-job X close button on finished download jobs in desktop GUI
- Fix widget ID clash when multiple download jobs are shown (push_id per job)
- Watched videos excluded from Continue Watching list and badge count
- settings table added to SQLite schema (key/value store)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Update cookies.txt by pasting Netscape-format contents in web UI settings and
the desktop GUI settings window (shared write/validation in web.rs)
- Treat yt-dlp exit code 101 (--break-on-existing reached an archived video) as
a successful "up to date" outcome instead of a failed job
- Add --no-write-playlist-metafiles so channel avatars/info aren't written as
phantom "Title [CHANNEL_ID]" files
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Features:
- Port all 7 desktop themes (Dracula, Trans, Emo variants, etc.) to the web UI
- Configurable bind interface (localhost/Tailscale/LAN/all) with detected IPs,
settable from both the web UI and desktop GUI
- Optional password that gates the whole UI and all API access via session
cookies (Argon2), with a login page, logout, and middleware; settable from
both UIs
- Library maintenance: scan for duplicate video IDs and missing assets
(thumbnail/info.json/description), remove duplicates (scan-and-confirm, with
out-of-library path guard), and re-fetch missing sidecars via yt-dlp.
Surfaced in both the web UI and a desktop GUI window.
Optimizations / cleanup:
- Cache has_chapters at scan time (library.rs) instead of re-reading and
parsing every info.json on each /api/library request
- Use the cached channel size in get_library
- Pool a single SQLite connection in WebState instead of opening one per request
- Remove a 1.2 GB stray nested src/yt-offline duplicate tree
Fixes:
- Rescan no longer clears all thumbnail textures (caused thumbnails to unload);
prune only removed entries and bump library_generation so the card grid
refreshes correctly
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Default makepkg.conf has OPTIONS=(... lto ...) which enables LTO globally.
With LTO, the cc crate compiles bundled sqlite3 as LLVM IR objects, and
rust-lld then fails to find any sqlite3_* symbols when linking the final
binary (undefined symbol errors for sqlite3_prepare_v3, sqlite3_finalize,
sqlite3_column_*, etc.).
Adding options=('!lto') tells makepkg to not enable LTO for this package,
which lets the bundled sqlite link normally as machine-code static lib.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Web UI:
- Replace SSE with simple 600ms polling (GET /api/progress) — SSE added
complexity with no real benefit for a local tool and was the source of
connection failures
- Poll task moved into request handler (dl.poll() on each /api/progress call)
so no background goroutine needed; simplifies state greatly
- Remove tokio-stream dependency (no longer needed)
- Fix: create channels dir before opening DB in web serve()
- Rescan also refreshes watched set from DB
- Bulk watched, sort, channel download button all working in web UI
Sort labels:
- Rename "Dur ↑" / "Dur ↓" → "Shortest" / "Longest"
- Rename "Size ↑" / "Size ↓" → "Smallest" / "Largest"
Channel right-click context menu:
- Right-click any channel in sidebar → "Check for new videos" starts a
yt-dlp job on the stored channel_url from its info.json
- Also shows "Open folder" shortcut
- Grayed out with tooltip if no URL is stored yet (channel has no info.json)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
License:
- GPL3 → AGPL3
New features:
- Web interface (--web [port]): axum/tokio server with SSE progress, library browser,
download trigger, watched toggle; embedded single-page HTML/JS UI
- Desktop notifications via notify-rust when downloads complete or fail
- Continue Watching sidebar entry showing videos with saved resume positions
- Bulk select mode: select multiple videos, mark all watched/unwatched at once
- Storage stats: channel disk usage shown in sidebar next to video count
- Scheduled channel checks: configurable interval (hours), auto re-downloads all
tracked channels using channel_url from info.json; enabled in Settings
Theme fixes:
- Scene Queen: remove override_text_color so active button text uses fg_stroke
(dark navy on neon green) instead of near-white — fixes unreadable contrast
- Trans: override_text_color → hot pink #cc0066 for pink-tinted text throughout
Settings additions:
- Auto-check channels toggle + interval_hours DragValue
- Web UI port setting
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- SQLite DB for watched/position persistence (database.rs)
- Mark watched toggle on cards and detail panel
- mpv IPC socket integration for resume position tracking (unix)
- Sort by title, duration, file size
- Thumbnail density slider in top bar
- Channel metadata banner (subscriber count, uploader, channel URL)
- Cookie browser selector dropdown in settings
- Auto-rescan library when a download job finishes
- Fix yt-dlp --no-progress-bar flag (does not exist; removed)
- Browser field wired through config → downloader
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- library.rs: scans channels/<name>/ into channels + videos by filename stem
- downloader.rs: runs yt-dlp in a background thread, streams progress to the UI
- app.rs / main.rs: channel sidebar, searchable thumbnail list, detail/description
panel, downloads panel; plays videos via mpv (falls back to xdg-open)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Rust/egui desktop app:
- library.rs: scans channels/<name>/ into channels + videos by filename stem
- downloader.rs: runs yt-dlp in a background thread, streams progress to the UI
- app.rs: channel sidebar, searchable video list with thumbnails, detail/description
panel, downloads panel, plays videos via mpv (falls back to xdg-open)
channels/ is gitignored (holds the downloaded media).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
