Commit graph

19 commits

Author SHA1 Message Date
Luna
3f611849af docs: rename yt-offline → Catacomb across README, ROADMAP, guides, packaging
Prose + commands updated to the new name: brand reads "Catacomb", while
binary/path/package references are the lowercase `catacomb` (commands, deb/rpm
names, ~/.local/share/catacomb, catacomb.db, catacomb.desktop). Codeberg repo
URLs left pointing at the existing repo. Also folds in the pending doc edits
from earlier in the session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 03:34:18 -07:00
Luna
9ed62935f5 Download manager: cancel/retry/queue controls + dedup off-switch
Download management UI (web + desktop):
- Cancel a running job (SIGKILL, marked "cancelled" so it isn't auto-retried
  and shows a distinct state rather than a misleading error class)
- Cancel a still-queued job before it starts
- Manually retry a failed/cancelled job (fresh auto-retry budget)
- Live speed · ETA · % on running jobs, parsed from the yt-dlp progress line
- Expandable full per-job log fetched on demand
New endpoints: POST /api/jobs/:idx/{cancel,retry}, GET /api/jobs/:idx/log,
DELETE /api/queued/:idx.

Perceptual-dedup off-switch:
- backup.dedup_enabled (default true) hard-disables the "similar content"
  scan in both UIs; the web scan endpoint returns {disabled:true} and the
  desktop scan no-ops with a note. Wired through all five settings touchpoints.

Docs: README seeking note + ROADMAP recently-shipped updates for the prior
seekable-playback / federation / auto-tagging work.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 16:35:05 -07:00
Luna
033572d3bd Federation: read-only browsing of a peer instance's library (3.5)
Configure peers as [[remote]] entries (name, url, optional password) and
browse their libraries from this instance, read-only.

- remote.rs: RemoteClient (blocking reqwest + rustls + cookie jar). Fetches
  the peer's /api/library, logging in first if the peer has a password and
  retrying on a 401. Media is not proxied: the library JSON's media URLs
  (/files/, /music-files/, /api/transcode/, /api/sub-vtt/) are rewritten to
  absolute peer URLs with the peer's read-only feed token appended, so video
  streams straight from the peer to the browser/mpv while only the small JSON
  passes through us.
- web.rs: auth_middleware now also accepts the feed token for GET
  /api/transcode/ and /api/sub-vtt/ (read-only media, same class as /files/),
  so a transcoded peer streams remotely. New GET /api/remotes and
  /api/remotes/:id/library (the RemoteClient runs under spawn_blocking).
- web UI: a 🌐 Remotes sidebar switcher; remote mode is read-only (hides the
  download bar + per-card mutating actions, keeps Play).
- desktop: a 🌐 Remotes screen that fetches a peer's library off-thread and
  plays via mpv on the tokenized URL.
- config.rs: [[remote]] section list; reqwest dep added (rustls/blocking/
  cookies, no system OpenSSL).

Verified end-to-end against a second local instance, including a transcoded
stream returning HTTP 200 via the token. ROADMAP 3.5 + CLAUDE.md updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 17:51:44 -07:00
Luna
aed577ea2f web/desktop: grid + subtitle fixes, comment toggle, faster startup
Several related UI and startup fixes from one session:

- Web grid: channel and music views rendered in a single column because
  their content was wrapped in one nested div that became a single cell of
  the outer CSS grid. Span the wrappers (and .empty) across all columns
  with grid-column:1/-1.

- Subtitles: route every track through /api/sub-vtt and strip per-cue
  position/alignment settings (align:start position:0% etc.) from both SRT
  and VTT, so auto-generated captions render centered instead of left-
  aligned. New strip_cue_settings/normalize_vtt + sub_vtt_url helpers.

- Comment downloads: add a global backup.fetch_comments toggle (the
  per-channel option already existed; it's now a tri-state override that
  defers to the global). Full five-touchpoint wiring across config,
  download_options, the downloader's apply_comments resolver, both UIs'
  global + per-channel controls, and downloader seeding.

- Desktop startup: run the initial library scan + search-index sync on a
  background thread (mirrors the web server's deferred bind) so the window
  appears immediately instead of blocking on a cold-cache scan; update()
  swaps the library in when it lands.

- Perceptual dedup: cap worker threads via fingerprint::default_workers()
  (~half the cores, always leaving at least one free) instead of using
  every core, so the hashing pass doesn't starve the UI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 01:43:50 -07:00
Luna
3031b5b0e5 Enhance comment viewer + make SponsorBlock configurable (3.6 + parity)
Comment viewer (web UI): the flat tree gains search (highlight + keep
ancestor context), sort (threaded / top / newest / oldest), per-thread
collapse/expand with collapse-all/expand-all, an OP badge for the
uploader, and a "new since last visit" highlight + count (localStorage
per-video timestamp). API now returns each comment's `timestamp` and
`author_is_uploader`.

SponsorBlock: was a hardcoded `--sponsorblock-mark all`. Now a real
setting (off / mark / remove) with a per-channel override, threaded
through the full five touchpoints — config `[backup].sponsorblock_mode`
(default "mark", preserving prior behavior), DownloadOptions override,
a downloader resolver (apply_sponsorblock), and both UIs (egui Settings
+ channel dialog; web Settings modal + channel dialog + SettingsPayload).

Integration tests extended: settings round-trip asserts the sponsorblock
default + persistence; channel-options round-trip asserts the override.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 03:53:49 -07:00
Luna
b8a742535b Desktop: global UI scale (egui zoom_factor), persisted
The card-density slider only resized the channel-grid cards — the
sidebar, settings, detail panel, and all text stayed fixed. Add a real
global scale that grows/shrinks the *entire* UI crisply.

egui already handles Ctrl + / Ctrl - / Ctrl 0 via zoom_factor (it
re-rasterizes fonts rather than bitmap-stretching), but it wasn't
persisted and had no visible control. Now:

- New `[ui] ui_scale` config (default 1.0), applied with
  set_zoom_factor at startup.
- update() mirrors ctx.zoom_factor() back into config each frame and
  saves on a 500ms debounce, so both the slider and the built-in
  keyboard zoom persist across restarts without writing config.toml
  every frame of a drag.
- A "UI scale" slider (0.5–3.0×) + Reset button in the Settings screen,
  driving set_zoom_factor live.

The card-density "Size:" slider stays as a separate grid-density
preference (it composes with the global zoom). 95 tests pass; GUI
verified running with the per-frame zoom-sync path.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-04 19:14:23 -07:00
Luna
fd9063b5cb Format conversion pipeline (Tartube parity 1.7)
Post-download ffmpeg pass with three modes, global + per-channel.
Closes the last real Tartube parity gap.

## Modes
- remux-mp4: fast container change mkv→mp4 (stream-copy video, aac audio)
- h264-mp4: re-encode to H.264/AAC at a configurable CRF + x264 preset
- audio: extract to mp3 / m4a / opus / flac

## Design (separate ffmpeg pass)
The main download adds `--print after_move:CONVERT_PATH:%(filepath)s`
when conversion is active. poll() scans freshly-Done download jobs for
those lines and enqueues a distinct ffmpeg transcode Job per finished
file — so the UI shows transcoding as its own phase. On success the
source is renamed to <name>.original.<ext> (keep_original) or deleted;
the converted file lands next to it. Transcodes don't auto-retry and
don't themselves convert (no infinite loop).

## Config
New [convert] section (mode / crf / preset / audio_format /
keep_original) + per-channel DownloadOptions.convert_mode override
(None = defer to global, "off" = force-disable for that channel).
ConvertSpec::resolve merges them; CRF 0→23, empty preset→medium, empty
audio→mp3 defaults.

## UI
Desktop + web: a "Format conversion (global defaults)" Settings section
with mode-dependent rows (CRF/preset for h264, format for audio, keep-
original toggle), and a per-channel Convert dropdown in the channel-
options dialog.

Verified: all three ffmpeg modes produce valid output on a real library
file (remux instant, h264 CRF23 ~12s for a 3.4min clip, mp3 extraction);
settings round-trip (global → config.toml [convert] → echo; per-channel
override saves + reads back). 4 resolver tests; 95 pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-03 23:20:06 -07:00
Luna
2f711604f0 Anti-bot: configurable YouTube player clients (global + per-channel)
YouTube's bot-detection cracks down on different player clients over
time — the `web` client gets captcha-walled, then later `tv`/`mweb` are
the safe ones, etc. Let the user route around whichever client is
currently being targeted instead of being stuck with yt-dlp's pick.

- New backup.youtube_player_clients config (comma-separated, blank =
  yt-dlp default). Maps to --extractor-args youtube:player_client=…
- Per-channel DownloadOptions.youtube_player_clients override (None =
  defer to global). Resolved in the downloader's apply_player_client,
  which omits the flag entirely when empty so yt-dlp keeps its default
  client set (the recommended baseline).
- Threaded into the live downloader at construction + settings save
  (app + web).

UI: a "YouTube player clients" field in both global Settings (desktop +
web) and the per-channel options dialog, with hints pointing at
"tv,mweb" as the current least-bot-checked combo for captcha-prone
channels.

Verified the settings round-trip (global persists to config + echoes;
per-channel override saves + reads back). 87 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 09:27:57 -07:00
Luna
0814ba0c3b Subtitle settings: global [subtitles] config + per-channel overrides
Adds full subtitle control (backend + desktop UI; web UI follows).

## Model
- New [subtitles] config section: enabled / auto_generated / embed /
  format / langs. Sensible defaults (subs on, auto on, no embed/convert,
  all langs).
- DownloadOptions gains per-channel Option overrides: subtitles_enabled,
  subtitles_auto, subtitles_embed, subtitle_format (subtitle_langs
  already existed). None = defer to global.

## Resolver
- Downloader gains subtitle_defaults (the global section) + a single
  apply_subtitle_flags() that merges global + per-channel and emits the
  yt-dlp flags (--write-subs / --write-auto-subs / --sub-langs /
  --convert-subs / --embed-subs). Disabled = emit nothing.
- Replaces the hardcoded --write-subs --write-auto-subs at the main
  download + repair builders. Moved the --sub-langs emission out of
  DownloadOptions::apply into the resolver so global + override merge in
  one place.

## Desktop UI
- Settings screen: "Subtitles (global defaults)" section.
- Channel-options dialog: tri-state (Default/On/Off) combos for each
  override + a convert-format field.

## Wiring
- subtitle_defaults seeded at Downloader construction (app + web) and
  refreshed on settings save (desktop).

4 resolver tests cover global defaults, disabled, global format/embed/
langs, and per-channel override-wins. 84 pass.

Web UI rows are the remaining piece (next commit).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 05:44:33 -07:00
Luna
2696a51cc8 Add YouTube POT token provider (bgutil-pot, opt-in)
YouTube increasingly requires a per-video Proof-of-Origin token bound
to each video ID before handing back format URLs. Without one, yt-dlp
sees empty formats and downloads fail. The upstream solution is the
Brainicism bgutil-ytdlp-pot-provider Python plugin paired with a
long-running server that mints tokens via BotGuard.

We integrate the Rust port (jim60105/bgutil-ytdlp-pot-provider-rs) so
the server is a single static binary, no Node.js required.

## Architecture

- bgutil-pot binary lives alongside the bundled deno + yt-dlp at
  ~/.local/share/yt-offline/bin/bgutil-pot.
- The matching Python plugin is pip-installed into the bundled venv:
  python -m pip install bgutil-ytdlp-pot-provider.
- Downloader spawns the server lazily on first job (port 4416,
  loopback only) and kills it on Drop.
- Each yt-dlp invocation gets
  --extractor-args "youtubepot-bgutilhttp:base_url=http://127.0.0.1:4416"
  appended in spawn_job when use_pot_provider is on and the server
  child is alive.

## UX

- Settings → "POT token provider" row with toggle + Install/Update
  button (mirrors the bundled yt-dlp row).
- Disabled unless use_bundled_ytdlp is also on (plugin lives in that
  venv).
- Refuses install if the bundled yt-dlp venv isn't there yet, with a
  helpful 428 error.

## Lifecycle

- Lazy spawn: ensure_pot_server() runs at the top of start() so the
  server is up before yt-dlp queries the plugin.
- Drop: kill_server() on Downloader drop so we don't leave an orphaned
  process bound to 4416.
- start_pot_provider_update() kills the running child first so the
  install can overwrite the binary in place (no ETXTBSY).

3 new unit tests cover URL/extractor-args formatting; 77 total pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-31 06:08:25 -07:00
Luna
615bc0cd4b Tray: minimize-to-tray is opt-in via config
Make ui.minimize_to_tray default-off so users on desktops without a
working StatusNotifier host (GNOME sans AppIndicator extension being
the common case) don't get stuck — without an opt-in, clicking X
would hide an invisible window with no obvious way back.

The tray menu's Show/Hide/Quit and Ctrl+Q still work regardless of
the flag; the toggle only affects what happens when the user clicks
the window's close button.

Added a settings-screen checkbox for the flag with a tooltip explaining
the SNI dependency.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 02:18:28 -07:00
Luna
59996735f3 Add bundled yt-dlp, music library, Plex metadata, sort by date; security + perf hardening
Bug fixes
- Desktop auto-rescan: snapshot was_running before check_notifications
  updates prev_job_states, so the post-download rescan actually fires.
- web::run shutdown: replace `let _ = tx;` (which dropped the sender
  immediately and short-circuited the shutdown select!) with
  std::mem::forget(tx); add an idle fallback so the `-> !` is honored.
- /api/preview now resolves the bundled yt-dlp path and extends PATH
  for deno discovery, matching the rest of the downloader.
- Scheduler interval clamped to >=1h on read so a manually-edited
  config.toml with 0 hours can't trigger every tick.
- maintenance::is_within canonicalizes the parent + filename when the
  target is missing; remove_files treats NotFound as success.

Security
- Session tokens stored with issued-at Instant and pruned past 30 day
  TTL on every touch (was: unbounded HashSet).
- Login rate-limited per source IP: 5 failures → 60s lockout (429).
- Secure cookie flag added when X-Forwarded-Proto: https is present.
- 4 MiB body size cap via DefaultBodyLimit.
- Security headers middleware: CSP, X-Content-Type-Options,
  X-Frame-Options, Referrer-Policy.
- JS safeUrl() defangs javascript:/vbscript:/data:text URLs before
  interpolation into src= attributes.
- Bundled yt-dlp install verifies SHA-256 against the release's
  SHA2-256SUMS; deno hash printed for visual inspection. Install also
  reports byte counts every 3s so users see live progress.
- yt-offline.db chmod 0600 at open time on Unix.

New features
- Bundled yt-dlp + deno mode: settings toggle (System/Bundled) plus
  one-click Install/Update from settings. Ensures executable bit on
  every launch in case the install chmod was missed.
- Download queue with configurable max_concurrent (1–10); pending
  jobs surfaced in both UIs.
- Music download mode: --extract-audio into music/<artist>/, separate
  sidebar entry, /api/music + /music-files/ endpoints, inline <audio>.
- Download quality selector: Best / 1080p / 720p / 480p / 360p.
- Sort videos by Newest / Oldest (upload_date from info.json with
  release_date fallback).
- Plex metadata: per-episode .nfo (title, season/episode, aired,
  runtime, plot from .description), <stem>-thumb.jpg symlink, and
  show-level tvshow.nfo.
- yt-dlp retry/throttle defaults: --retries 30 --fragment-retries 30
  --retry-sleep linear=1:30:2 --sleep-requests 1 to ride out the
  "Connection reset by peer" failures.

Performance
- password_required cached in AtomicBool; refreshed only on password
  change. Eliminates a SQLite query per static-file fetch.
- Job::log → VecDeque for O(1) front-pop instead of O(n) drain.
- Library scan parallelized across cores via stdlib parallel_map.
- Web UI poll: 600ms while active, 5s when idle, 2s after errors.
- Music scan now recursive (Artist/Album/Track.opus).
- percent_encode_segment uses write! to avoid per-byte String allocs.

Code quality
- 27 unit tests covering parse_progress, parse_stem, extract_after,
  detect_url_kind, srt_to_vtt, count_cookies, percent_encode_segment,
  bind_mode_of, hash_password, lang_label, is_within, sanitize,
  year_of, aired_date, xml_escape.
- Unified library::find_video + Channel::all_videos() iterator,
  replacing three duplicated linear searches.
- Downloader::browser wired to --cookies-from-browser as a fallback
  when no cookies.txt is present.
- mpv detection now matches the binary basename, not substring.
- Settings modal scroll fix; web settings expose all new fields.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 04:54:26 -07:00
Luna
1d72069913 Add full-scan mode, persist password in DB, fix channel re-check URLs, SRT subs, cookie clear, UI fixes
- Full-scan download toggle: omits --break-on-existing so all unarchived
  videos are fetched regardless of order, filling gaps in partial archives
- Filter "Aborting remaining downloads" from job logs (break-on-existing noise)
- Password hash moved from config.toml to SQLite settings table; survives
  git checkouts and rebuilds. config.toml and yt-offline.db added to .gitignore;
  config auto-generated on first run if absent
- Channel re-check URLs derived from folder name (/@handle or /channel/UCxxx)
  instead of info.json's canonical channel_url, preventing duplicate UCxxx folders
- SRT subtitles detected by library scanner and served via /api/sub-vtt/* endpoint
  that converts to WebVTT on the fly; browser <track> element can now play them
- Cookie clear button in both desktop and web UI (DELETE /api/cookies)
- Per-job X close button on finished download jobs in desktop GUI
- Fix widget ID clash when multiple download jobs are shown (push_id per job)
- Watched videos excluded from Continue Watching list and badge count
- settings table added to SQLite schema (key/value store)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-21 05:49:55 -07:00
Luna
1ef3fe56c6 Add web themes, network binding, UI auth, library maintenance; optimize + fix rescan
Features:
- Port all 7 desktop themes (Dracula, Trans, Emo variants, etc.) to the web UI
- Configurable bind interface (localhost/Tailscale/LAN/all) with detected IPs,
  settable from both the web UI and desktop GUI
- Optional password that gates the whole UI and all API access via session
  cookies (Argon2), with a login page, logout, and middleware; settable from
  both UIs
- Library maintenance: scan for duplicate video IDs and missing assets
  (thumbnail/info.json/description), remove duplicates (scan-and-confirm, with
  out-of-library path guard), and re-fetch missing sidecars via yt-dlp.
  Surfaced in both the web UI and a desktop GUI window.

Optimizations / cleanup:
- Cache has_chapters at scan time (library.rs) instead of re-reading and
  parsing every info.json on each /api/library request
- Use the cached channel size in get_library
- Pool a single SQLite connection in WebState instead of opening one per request
- Remove a 1.2 GB stray nested src/yt-offline duplicate tree

Fixes:
- Rescan no longer clears all thumbnail textures (caused thumbnails to unload);
  prune only removed entries and bump library_generation so the card grid
  refreshes correctly

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 02:58:09 -07:00
Luna
74b3efd990 Major update with lots of new features and fixes 2026-05-17 05:36:04 -07:00
Luna
80cce00b99 Add web UI, AGPL3, notifications, continue watching, bulk watched, storage stats, scheduler
License:
- GPL3 → AGPL3

New features:
- Web interface (--web [port]): axum/tokio server with SSE progress, library browser,
  download trigger, watched toggle; embedded single-page HTML/JS UI
- Desktop notifications via notify-rust when downloads complete or fail
- Continue Watching sidebar entry showing videos with saved resume positions
- Bulk select mode: select multiple videos, mark all watched/unwatched at once
- Storage stats: channel disk usage shown in sidebar next to video count
- Scheduled channel checks: configurable interval (hours), auto re-downloads all
  tracked channels using channel_url from info.json; enabled in Settings

Theme fixes:
- Scene Queen: remove override_text_color so active button text uses fg_stroke
  (dark navy on neon green) instead of near-white — fixes unreadable contrast
- Trans: override_text_color → hot pink #cc0066 for pink-tinted text throughout

Settings additions:
- Auto-check channels toggle + interval_hours DragValue
- Web UI port setting

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-05-11 04:27:04 -07:00
Luna
5b3b8fc901 Add watched tracking, resume positions, sort, density slider, channel metadata, mpv IPC, browser selector, auto-rescan
- SQLite DB for watched/position persistence (database.rs)
- Mark watched toggle on cards and detail panel
- mpv IPC socket integration for resume position tracking (unix)
- Sort by title, duration, file size
- Thumbnail density slider in top bar
- Channel metadata banner (subscriber count, uploader, channel URL)
- Cookie browser selector dropdown in settings
- Auto-rescan library when a download job finishes
- Fix yt-dlp --no-progress-bar flag (does not exist; removed)
- Browser field wired through config → downloader

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-05-11 03:47:29 -07:00
Luna
c11d1c3366 add themes, settings GUI, URL-aware download routing, and playlist view 2026-05-11 03:18:16 -07:00
Luna
abf3af5768 First version of youtube-backup, a tool to backup your YouTube channel. 2026-05-11 02:33:52 -07:00