luna/catacomb
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

README + Cargo description: cover every shipped feature

Browse source 
The README still described yt-offline as a YouTube-only desktop app.
Rewrite it to reflect what actually exists today:

- Multi-platform support (YouTube, TikTok, Twitch, Vimeo, Bandcamp,
  SoundCloud, Odysee, and yt-dlp's full fallback list) with a table
  of what each platform supports.
- Library, playback, downloads, web server, Plex integration, themes,
  and misc sections — each grouping the relevant features so a reader
  can spot what matters to them.
- Bundled-yt-dlp install walkthrough (venv + curl_cffi + deno).
- Updated config.toml example with every field the app reads.
- Library layout diagram showing the new platform sibling folders
  and the `.source-url` sidecar.
- Usage section split between desktop GUI and web UI.
- Troubleshooting now lists the impersonation error path, missing
  python3-venv, the transcoding-seek caveat, and rate-limit recovery.
- Pointer to SECURITY_AUDIT.md.
- AGPL §13 obligations spelled out.

Also expand the Cargo.toml description so `cargo metadata` and crates.io
listings convey the multi-platform / web UI scope at a glance.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Luna 2026-05-24 23:21:48 -07:00
parent 770ef66623
commit c0be4bb533
2 changed files with 250 additions and 135 deletions
Download patch file Download diff file Expand all files Collapse all files

2
Cargo.toml
View file

@ -2,7 +2,7 @@
name = "yt-offline"
version = "0.1.0"
edition = "2021"
description = "A desktop app for archiving YouTube channels with yt-dlp"
description = "Self-hosted archive for YouTube, TikTok, Twitch, Vimeo, Bandcamp, SoundCloud, Odysee and more. Desktop GUI + web UI, bundled yt-dlp with curl_cffi impersonation, Plex export, SQLite-backed resume tracking. AGPL-3.0."
[dependencies]
eframe = "0.29"

383
README.md
View file

@ -2,204 +2,319 @@
![yt-offline icon](icon.png)
A desktop app for archiving YouTube channels using [yt-dlp](https://github.com/yt-dlp/yt-dlp). Browse your downloaded channels, manage downloads, and play videos — all from a native GUI.
A self-hosted media archive for YouTube and friends. Pastes any URL, routes it
to the right folder by source, tracks what you've watched, and lets you play
everything back from a desktop GUI or browser — even when you're offline or
the source video has been taken down.
Built on [yt-dlp](https://github.com/yt-dlp/yt-dlp); written in Rust.
## What it backs up
| Platform | Channels | Playlists | Single videos |
| ---------------------------- | ----------- | ----------- | ------------- |
| YouTube | ✅ | ✅ | ✅ |
| TikTok | ✅ | — | ✅ |
| Twitch (VODs + clips) | ✅ | — | ✅ |
| Vimeo | ✅ | ✅ | ✅ |
| Bandcamp | ✅ (artist) | ✅ (albums) | ✅ (tracks) |
| SoundCloud | ✅ | ✅ (sets) | ✅ |
| Odysee | ✅ | — | ✅ |
| Anything else yt-dlp accepts | `other/` | `other/` | `other/` |
Each source lands in its own sibling folder under your backup directory
(`channels/` for YouTube, `tiktok/`, `twitch/`, etc.). A `.source-url`
sidecar is dropped in every creator folder so re-checks always know the
exact URL to refresh from.
## Features
- **Channel Browser** — Browse downloaded channels and videos with thumbnails
- **Smart Download Routing** — Paste any YouTube URL; channels, playlists, and single videos are automatically routed to the right folder
- **Playlist View** — Channels with playlist subdirectories show them in the sidebar
- **Themes** — Dark, Light, Dracula, Trans, and three Emo/Scene themes
- **Settings GUI** — Configure everything from inside the app
- **Video Playback** — Launch videos in mpv, VLC, or any player
- **Search** — Filter across your entire library in real time
- **System Tray** — Minimize to tray
### Library
- **Multi-platform sidebar** grouped by source, with platform icons.
- **Search** across your whole library, instant.
- **Sort by:** Newest / Oldest (upload date), Largest / Smallest, Longest /
Shortest, Title.
- **Watched tracking** + **resume positions** persisted in SQLite.
- **Continue watching** view of partially-played videos.
- **Subtitles** (auto + manual, SRT and WebVTT) + **chapters** + **SponsorBlock**
marks embedded into the video file.
- **Statistics** — totals, top channels by size/count, weekly downloads
histogram, upload-year histogram.
- **Maintenance** — finds duplicate video IDs across folders and missing
sidecars (thumbnail / info.json / description), with one-click repair.
### Playback
- **Desktop GUI** launches your configured player (mpv by default; mpv gets
resume-position tracking via JSON-IPC).
- **Web UI** plays videos inline in any browser, with subtitle tracks and a
chapters panel. Optional on-the-fly **ffmpeg transcoding** for browsers
that can't decode MKV.
### Downloads
- **Bundled yt-dlp + curl_cffi + deno** — one click in Settings sets up a
self-contained Python venv with `yt-dlp[default]` plus `curl_cffi`
(browser-impersonation for bot-detection bypass) and `deno` for player-JS
evaluation. No system installation required.
- Or use **system yt-dlp** — toggleable in Settings.
- **Concurrent-download limit** (default 3) with a visible queue.
- **Quality picker:** Best / 1080p / 720p / 480p / 360p, or **Music mode**
for audio-only extraction into `music/<artist>/`.
- **Retry hardening:** `--retries 30 --fragment-retries 30
--retry-sleep linear=1:30:2 --sleep-requests 1` so connection resets self-heal
instead of failing the job.
- **Scheduler** — periodically re-check every channel for new uploads.
- **Cookies** — paste / file-pick a Netscape `cookies.txt`, or fall back to
`--cookies-from-browser` for whatever browser you pick.
### Web server
- Single binary, no Node / Python / Docker required (Python only for the
optional bundled yt-dlp venv).
- **Bind interface picker** — localhost only (default), Tailscale, LAN, or
all interfaces.
- **Password-gated UI** when enabled — Argon2 hashed, 256-bit session
tokens, 30-day TTL with lazy pruning, per-IP rate-limit on `/api/login`
(5 failures → 60s lockout).
- **Security headers:** Content-Security-Policy, X-Frame-Options DENY,
X-Content-Type-Options nosniff, Referrer-Policy no-referrer.
- **`Secure` cookie flag** when `X-Forwarded-Proto: https` is present
(i.e. behind a reverse proxy doing TLS).
- 4 MiB request body limit on every route.
### Plex integration
- One click generates a symlink tree your Plex "TV Shows" library can ingest
directly: `<plex_root>/<show>/Season 2024/<show> - S2024E001 - title.mkv`.
- Writes `.nfo` sidecars per episode (title, season, episode, aired date,
runtime, plot) plus a show-level `tvshow.nfo`, so Plex's "Personal Media
(TV Shows)" agent picks up YouTube metadata correctly.
- Symlinks the thumbnail as `<stem>-thumb.jpg` so Plex shows episode art.
- Non-YouTube creators get `<Platform> - <handle>` show folders to avoid
collisions across sources.
### Themes
Ten themes shared across desktop and web — Dark, Light, Dracula, Trans,
plus three Emo/Scene flavours (Nocturnal, Coffin, Scene Queen).
### Misc
- Native notifications when downloads finish (notify-rust / zbus, no GTK
build dep).
- `/api/stats`, `/api/ytdlp/update`, `/api/scheduler/run` — every UI button
has an equivalent JSON endpoint.
## Quick start
```bash
git clone https://codeberg.org/anassaeneroi/yt-offline
cd yt-offline
cargo build --release
# Desktop GUI
./target/release/yt-offline
# Or web server only (headless)
./target/release/yt-offline --web 8080
```
On first run a `config.toml` is created in the working directory. Settings
can also be changed from inside either UI — no manual editing required.
### Bundled yt-dlp setup
1. Open **Settings**.
2. Under **yt-dlp binary**, choose **Bundled** and click **Install**.
3. The installer creates `~/.local/share/yt-offline/venv/` with
`yt-dlp[default]` + `curl_cffi`, plus `~/.local/share/yt-offline/bin/deno`.
Progress streams into a regular job entry.
Updates are the same button. Switching to **System** uses whatever `yt-dlp`
is on PATH instead.
## Building
### Arch Linux / Manjaro
A `PKGBUILD` is included:
### Arch / CachyOS / Manjaro
```bash
sudo pacman -S --needed rust mpv # python3 + python-pip already present
git clone https://codeberg.org/anassaeneroi/yt-offline
cd yt-offline
makepkg -si
makepkg -si # or: cargo build --release
```
Or build manually:
A `PKGBUILD` is included.
```bash
sudo pacman -S --needed rust yt-dlp mpv
cargo build --release
sudo install -Dm755 target/release/yt-offline /usr/bin/yt-offline
```
---
### Debian / Ubuntu / Linux Mint
Install build dependencies:
### Debian / Ubuntu
```bash
sudo apt install \
build-essential pkg-config curl git \
build-essential pkg-config curl git python3-venv \
libxcb-render0-dev libxcb-shape0-dev libxcb-xfixes0-dev \
libxkbcommon-dev libssl-dev \
libgtk-3-dev libayatana-appindicator3-dev
```
Install Rust (if not already installed):
```bash
libxkbcommon-dev libssl-dev
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"
```
Install runtime dependencies:
```bash
sudo apt install yt-dlp mpv
```
Build and install:
```bash
git clone https://codeberg.org/anassaeneroi/yt-offline
cd yt-offline
cargo build --release
sudo install -Dm755 target/release/yt-offline /usr/bin/yt-offline
cp youtube-backup.desktop ~/.local/share/applications/yt-offline.desktop
```
---
`python3-venv` is required for the bundled yt-dlp install path (skipable
if you only ever use system yt-dlp).
### macOS
Install Xcode command line tools and Homebrew dependencies:
```bash
xcode-select --install
brew install yt-dlp mpv
```
Install Rust:
```bash
brew install mpv
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"
cargo build --release
```
Build:
### Windows
```bash
Install [Rust](https://rustup.rs/) (MSVC toolchain), Visual Studio Build
Tools with the **Desktop development with C++** workload, then:
```powershell
winget install mpv.net python.python.3.12
git clone https://codeberg.org/anassaeneroi/yt-offline
cd yt-offline
cargo build --release
```
The binary is at `target/release/yt-offline`. Copy it wherever you like, or run it in place.
---
### Windows
1. Install [Rust](https://rustup.rs) — accept the default MSVC toolchain when prompted
2. Install [Visual Studio Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/) with the **Desktop development with C++** workload
3. Install yt-dlp and mpv:
```powershell
winget install yt-dlp.yt-dlp
winget install mpv.net
```
4. Clone and build:
```powershell
git clone https://codeberg.org/anassaeneroi/yt-offline
cd yt-offline
cargo build --release
```
The binary is at `target\release\yt-offline.exe`. Copy it wherever you like and run it.
> **Note:** The first build takes a while — Rust compiles all dependencies from scratch.
---
## Configuration
On first run, create a `config.toml` next to the binary (or edit it from the Settings button inside the app):
`config.toml` lives next to the binary (or in the working directory). All
fields are also editable in Settings.
```toml
[backup]
directory = "/path/to/your/video/library"
directory = "/path/to/library/channels" # YouTube root; siblings hold other platforms
max_concurrent = 3 # parallel yt-dlp processes
use_bundled_ytdlp = false # true = use the venv set up by the Install button
[player]
command = "mpv"
command = "mpv" # any executable that takes a file path as last arg
browser = "firefox" # used by --cookies-from-browser when no cookies.txt is set
[ui]
theme = "dark"
theme = "dark" # dark | light | dracula | trans | emo-nocturnal | emo-coffin | emo-scene-queen
[scheduler]
enabled = false
interval_hours = 24
[web]
port = 8080
bind = "127.0.0.1" # 127.0.0.1 | 0.0.0.0 | a Tailscale/LAN address
transcode = false # MKV → MP4 on the fly for browsers (needs ffmpeg)
source_url = "https://codeberg.org/anassaeneroi/yt-offline" # AGPL §13 footer link
[plex]
library_path = "/path/to/plex/TV/youtube" # leave unset to disable
```
### Options
| Setting | Default | Description |
| ------- | ------- | ----------- |
| `backup.directory` | `./channels` | Where downloaded videos are stored |
| `player.command` | `mpv` | Command used to launch the video player |
| `ui.theme` | `dark` | `dark`, `light`, `dracula`, `trans`, `emo-nocturnal`, `emo-coffin`, `emo-scene-queen` |
---
## Usage
1. **Download a channel/video/playlist** — click **⬇ Downloads**, paste a YouTube URL. The app detects the URL type and routes it automatically:
- Channel URL → `channels/<handle>/`
- Single video → `channels/<channel-name>/`
- Playlist → `channels/<channel-name>/<playlist-name>/`
### Desktop GUI
2. **Browse your library** — channels appear in the left sidebar. If a channel has playlist subdirectories they show as collapsible sub-items.
1. Click **⬇ Downloads**, paste any URL — channel, video, playlist, or
profile from any supported platform. The dialog shows the detected
source and target folder before you confirm.
2. Choose quality (or **🎵 Music** for audio-only).
3. Pick **Fast mode** to stop at the first already-archived video, or leave
it off for a full back-fill.
3. **Play a video** — click **▶ Play** on any video card, or double-click the thumbnail.
The sidebar groups channels by platform. Right-click any channel for
**Check for new videos** and **Open folder**. The download bar shows the
active job count plus what's queued.
4. **Change settings** — click **⚙ Settings** to change the backup directory, player, or theme without editing the file.
Other top-bar buttons:
---
- **⟳ Rescan** — re-read the library directory.
- **📊 Stats** — totals, top channels, weekly download histogram.
- **🩺 Maintenance** — find and resolve duplicates / missing sidecars.
- **⚙ Settings** — everything in `config.toml`, plus cookies, password,
bundled-yt-dlp install/update, and starting the web server.
### Web UI
Start the server (`--web` CLI flag or **Start** button in Settings →
**Web server**), open `http://localhost:8080` (or your configured bind
address). The UI mirrors the desktop one and adds inline video playback.
If a download password is set, you'll be redirected to a login page;
the session cookie is HttpOnly + SameSite=Strict, valid 30 days.
## Library layout
```text
/path/to/library/
├── channels/ ← YouTube
│ ├── @creator-name/
│ │ ├── .source-url ← original URL, for re-checks
│ │ ├── archive.txt ← yt-dlp's downloaded-ID record
│ │ ├── Video Title [abc123].mkv
│ │ ├── Video Title [abc123].webp
│ │ ├── Video Title [abc123].info.json
│ │ ├── Video Title [abc123].description
│ │ └── Video Title [abc123].en.vtt
│ └── @another/
├── tiktok/
├── twitch/
├── vimeo/
├── bandcamp/
├── soundcloud/
├── odysee/
├── other/
├── music/
│ └── <artist>/
│ └── Track Title [xyz].opus
└── yt-offline.db ← watched + positions + password hash
```
## Troubleshooting
**`yt-dlp` not found**
Make sure it's installed and on your `$PATH`. On Windows, restart your terminal after install.
**"Impersonate target ... is not available"**
You're on bundled mode but the venv install hasn't completed. Open
Settings → yt-dlp binary → click **Install** and watch the job for the
"impersonation targets available" line.
**Build fails on Debian/Ubuntu with missing headers**
Make sure you installed all packages in the build dependencies block above, especially `libayatana-appindicator3-dev` and `libgtk-3-dev`.
**`yt-dlp` not found (system mode)**
Install yt-dlp via your package manager or `pip install yt-dlp`, or
switch to **Bundled** in Settings.
**Build fails on Windows with linker errors**
Make sure Visual Studio Build Tools are installed with the C++ workload selected, not just the base tools.
**Web UI says "authentication required" but no password is set**
Sessions cleared after the password was changed. Refresh the page.
**Videos won't play**
Check `player.command` in `config.toml` (or Settings). The command must accept a file path as its last argument.
**Videos play but seeking jumps to the start**
That's the transcoding mode (`web.transcode = true`). MKV streams can't be
seeked while transcoding live; turn it off if your browser can play MKV
directly.
**Thumbnails not loading**
Supported formats: JPEG, PNG, WebP. Other formats are silently skipped.
**Connection-reset / "Recv failure" errors on download**
The defaults already retry 30× with linear backoff. If they're persistent,
your IP is being rate-limited — paste fresh cookies from a browser session
into Settings → Cookies.
---
**Build fails on Debian/Ubuntu**
Make sure all packages from the build dependencies block are installed,
plus `python3-venv` if you want the bundled yt-dlp install path.
## Project Structure
## Security
```text
src/
main.rs entry point
app.rs UI and main loop
downloader.rs yt-dlp integration and URL detection
library.rs channel/playlist/video scanner
config.rs config file loading and saving
theme.rs all colour themes
database.rs SQLite (reserved for future use)
tray.rs system tray
```
See [SECURITY_AUDIT.md](SECURITY_AUDIT.md) — the codebase has been audited
end-to-end, covering command/SQL injection, path traversal, XSS, session
management, rate-limiting, supply-chain (bundled binary SHA verification),
and dependency status.
## License
See [LICENSE](LICENSE).
[AGPL-3.0-or-later](LICENSE). When you run this software as a network
service, the AGPL requires that you offer network users access to the
Corresponding Source — the running version, with any modifications.
Set `web.source_url` in config to a public URL where the source is
available; the web UI surfaces it in the footer and `GET /api/settings`.