# Tartube Specification A structured analysis of [Tartube v2.5.231](https://github.com/axcore/tartube) (2026-05-24), our project's primary feature benchmark. Written to give yt-offline a concrete target to work toward — every section ends with a "What we can do" note about whether to adopt, adapt, or skip. Tartube codebase: ~146,000 lines of Python 3 across 19 modules under `tartube/`, plus locale, icons, sounds, screenshots. GTK 3 frontend. GPL-3-or-later. Single-user desktop application. ## 1. Module map | File | Lines | Responsibility | | --- | --- | --- | | `mainapp.py` | 30,565 | The `TartubeApp` Gtk.Application — owns 1,601 instance variables / 791 boolean toggles, every config field, every operation entry point. | | `mainwin.py` | 42,074 | `MainWin` Gtk window + ~80 specialised `Gtk.Dialog` subclasses + the Video Catalogue widgets. | | `config.py` | 36,659 | Preferences and per-object editor windows (per-channel options, FFmpeg options, scheduled actions, custom-download recipes). | | `downloads.py` | 12,188 | Download orchestration: `DownloadManager` (worker pool), `DownloadList` (queue), `DownloadWorker`, three downloader strategies (`VideoDownloader`, `ClipDownloader`, `StreamDownloader`), JSON fetchers. | | `ttutils.py` | 5,015 | Cross-cutting helpers: path manipulation, livestream-message parsing, locale, sorting. | | `media.py` | 4,684 | Hierarchical data model: `Video`, `Channel`, `Playlist`, `Folder`, `Scheduled`, plus `GenericMedia` / `GenericContainer` / `GenericRemoteContainer` mixins. | | `wizwin.py` | 4,113 | Setup wizard, YouTube subscriptions import wizard, tutorial wizard. | | `options.py` | 2,284 | `OptionsManager`: a 164-field bag of yt-dlp / youtube-dl flags that can be attached to any Video / Channel / Playlist / Folder, plus an `OptionsParser` that compiles them into a yt-dlp CLI invocation. | | `tidy.py` | 1,636 | `TidyManager`: verify-on-disk operation that finds phantom DB records, missing files, orphaned thumbnails. | | `ffmpeg_tartube.py` | 1,303 | `FFmpegManager` + `FFmpegOptionsManager`: post-processing pipeline with named presets. | | `formats.py` | 1,289 | Canonical lists: SponsorBlock categories/actions, audio/video format codes, language codes, livestream message patterns. | | `updates.py` | 1,180 | `UpdateManager`: self-update yt-dlp via pip / system package / git inside Tartube. | | `process.py` | 1,012 | `ProcessManager`: chains FFmpeg invocations after a download completes. | | `info.py` | 671 | `InfoManager`: yt-dlp `--dump-json` + capability probes, version checks. | | `refresh.py` | 617 | `RefreshManager`: re-import disk state into the media registry. | | `dialogue.py` | 522 | Small/common dialog primitives (yes-no, message, OK). | | `classes.py` | small | Misc shared helpers. | | `files.py` | 167 | File-existence checks. | | `xdg_tartube.py` | 183 | XDG base-directory resolution for config/data paths. | Auxiliary directories: - `locale/` — `gettext` translations (many languages). - `icons/`, `sounds/` — assets. - `nsis/`, `pack/` — Windows installer + per-distro packaging. - `docs/` — manual pages. **What we can do:** Our entire codebase is ~9k lines of Rust right now. Tartube's volume comes from (a) the sheer number of UI dialogs (~80) and (b) `OptionsManager` cascading through every code path. Both are mechanical work to match, not architectural blockers. ## 2. Data model Tartube's central insight is a **container hierarchy** where everything nests: ``` Folder ─┬─ Folder (recursive) ├─ Channel ── Video └─ Playlist ── Video ``` `Folder` is the freeform organisation layer (user-created and system-created — see §2.1). `Channel` and `Playlist` are "remote containers" with a `source` URL and download semantics. `Video` is a leaf with on-disk file references plus state flags. ### 2.1 System folders Tartube ships with a fixed set of system `Folder` objects users can't delete: - **All Videos** — every video across the registry. - **Bookmarks** — videos with `bookmark_flag = True`. - **Favourite Videos** — videos with `fav_flag = True`, inherited from any ancestor. - **Live Videos** — videos with `live_mode > 0`. - **Missing Videos** — videos that existed but are gone from the source. - **New Videos** — downloaded but not yet watched. - **Recent Videos** — last N downloads. - **Waiting Videos** — videos manually queued for later attention. - **Temporary Videos** — one-off Classic-Mode downloads. System folders are virtual: they don't own their videos, they're filtered views of the registry by flag. Adding a video to "Bookmarks" sets `bookmark_flag` on the underlying video; removing the bookmark removes the video from the folder. **What we can do:** This pattern is *exactly* the activity feed / smart filter mechanism we should adopt. Our `SidebarView::Recent` is one of these, hand-coded. Generalising into a `SmartFolder` enum with a predicate would let us add Bookmarks / Favourites / Waiting / New mechanically. Tartube parity-and-then-some. ### 2.2 `media.Video` field inventory Beyond what we have today, Tartube tracks per video: | Field | Purpose | Have it? | | --- | --- | --- | | `dbid` | Integer primary key | No (we key by yt-dlp ID string) | | `vid` | Extractor's video ID | Yes (`Video::id`) | | `name` vs `nickname` | Filename vs display title | No — one `title` field | | `natname` | Natural-sort key | No — sorted at query time | | `source` | Original URL | No on `Video`; we keep `source_url` per Channel | | `file_name`, `file_ext` | Separate parts | No — one `PathBuf` | | `file_size` | bytes | Yes | | `upload_time` (unix) | Upload date as seconds | Partial — we have `YYYYMMDD` string | | `receive_time` (unix) | When Tartube downloaded it | We have `mtime_unix`, equivalent | | `duration` (int seconds) | Yes (`duration_secs`) | | `index` | Position in channel/playlist | No | | `author` | Uploader (distinct from channel) | No | | `descrip`, `short` | Full + summary description | We have full via sidecar | | `live_mode` (0/1/2) | Not-live / scheduled / broadcasting | We have `live: bool` at download time only | | `live_debut_flag` | YouTube "premiere" videos | No | | `was_live_flag` | Once-was-live, prevents re-marking | No | | `live_time` | Approximate start time | No | | `live_msg` | Parsed livestream message | No | | `archive_flag` | Skip auto-deletion | No | | `bookmark_flag` | Bookmarked | No | | `fav_flag` | Favourited (cascades from ancestors) | No | | `missing_flag` | Removed from source after download | No (we have a maintenance scan) | | `new_flag` | Downloaded but unwatched | Inverse of our `watched` set | | `waiting_flag` | Manually queued for attention | No | | `block_flag` | Censored / age-restricted | No | | `split_flag` | Clip extracted from a parent video | No | | `orig_parent_obj` | Where the video originally came from | No | | `subs_list` | Available subtitle languages | Yes (`subtitles: Vec`) | | `stamp_list` | Chapter timestamps (manual or extracted) | Partial — we read chapters from info.json, no manual editing | | `slice_list` | Per-video SponsorBlock data | No — we use `--sponsorblock-mark` at download time | | `comment_list` | Comments (yt-dlp `--write-comments`) | No | | `error_list`, `warning_list` | Last download attempt messages | No persisted; we have rolling Job log only | ### 2.3 `media.Channel` / `media.Playlist` / `media.Folder` Container fields (shared via `GenericContainer`): - `dbid`, `name`, `nickname`, `natname`, `parent_obj`, `child_list`. - `options_obj` — per-container `OptionsManager` override (§3). - Counters: `vid_count`, `bookmark_count`, `dl_count`, `fav_count`, `live_count`, `missing_count`, `new_count`, `waiting_count`. - `fav_flag` — cascades to all descendants. - `error_list`, `warning_list`. - `external_dir` — optional override storage path. - `master_dbid` / `slave_dbid_list` — alias resolution (a video uploaded to two channels with different IDs can be deduplicated by pointing one to the other). Channels & Playlists additionally have: - `source` (URL). - `rss` — RSS feed URL (for fast new-video discovery). - `playlist_id_dict` — set of playlist IDs already imported. - `dl_no_db_flag` — download to disk but skip the database. - `dl_sim_flag` — always simulate (check-only, never actually fetch). - `dl_disable_flag` — skip during operations. ### 2.4 Storage - **`tartube.db`** — Python `pickle` of the entire media registry, not SQLite. Atomic save-on-modify with periodic backups (configurable `db_backup_mode = 'always' | 'every_session' | 'daily' | 'never'`). - **Per-channel folder layout**: ``` / / .mp4 .info.json .description ...vtt .ytdl-archive (yt-dlp's downloaded-IDs record) ... ``` - **External directories** — a channel can be redirected to a separate drive via `external_dir`. **What we can do:** Pickle is a misfeature — schema migrations are nightmarish and a corrupt pickle locks out the whole library. Our SQLite-backed approach is strictly better. Don't adopt the storage model; do adopt the per-channel layout (which matches ours already). ## 3. `OptionsManager` — the headline feature `options.OptionsManager` is the single biggest gap between us and Tartube. It holds **164 distinct yt-dlp / youtube-dl flags** and can be attached to: - A specific `Video`. - A `Channel` / `Playlist` / `Folder`. - The application globally (the "default" manager). Resolution at download time: `Video.options_obj` → `parent.options_obj` → ancestor chain → app global. First non-`None` wins. A user can create *named* OptionsManager instances and apply the same one to many channels. ### 3.1 Canonical option groups From the docstring at `options.py:69`: - **Behaviour**: ignore_errors, abort_on_error, live_from_start, wait_for_video_min. - **Network**: proxy, socket_timeout, source_address, force_ipv4/6, geo_verification_proxy, geo_bypass*, geo_bypass_country, geo_bypass_ip_block. - **Playlist selection**: playlist_start, playlist_end, playlist_items, max_downloads, playlist_reverse, playlist_random, skip_playlist_after_errors, break_on_existing, break_on_reject. - **Filters**: min_filesize, max_filesize, date, date_before, date_after, min_views, max_views, match_filter, age_limit, include_ads, match_title_list, reject_title_list. - **Download**: limit_rate, retries, abort_on_unavailable_fragment, native_hls, hls_prefer_ffmpeg, external_downloader, external_arg_string, concurrent_fragments, throttled_rate. - **Filesystem**: restrict_filenames, nomtime, write_description, write_info, write_annotations, cookies_path, write_thumbnail, force_encoding, output_format, output_template, output_format_list, output_path_list, windows_filenames, trim_filenames, no_overwrites, force_overwrites. - **Auth**: username, password, two_factor, net_rc, video_password, ap_mso, ap_username, ap_password. - **Anti-bot**: no_check_certificate, prefer_insecure, user_agent, referer, min_sleep_interval, max_sleep_interval. - **Format**: video_format, all_formats, prefer_free_formats, yt_skip_dash, merge_output_format, video_format_list, video_format_mode. - **Subtitles**: write_subs, write_auto_subs, write_all_subs, subs_format, subs_lang, subs_lang_list. - **Post-processing**: extract_audio, audio_format, audio_quality, recode_video, pp_args, keep_video, embed_subs, embed_thumbnail, add_metadata, fixup_policy, prefer_avconv, prefer_ffmpeg. - **Cookies**: no_cookies, cookies_from_browser, no_cookies_from_browser. - **Output sidecars**: write_link, write_url_link, write_webloc_link, write_desktop_link. - **Tartube-specific**: keep_description / keep_info / keep_thumbnail / keep_annotations (move sidecars vs delete), sim_keep_* (same in simulate mode), move_description (move out of data dir after download), extra_cmd_string (raw passthrough), direct_cmd_flag / direct_url_flag (escape hatches), fetch_formats_cmd_string, fetch_subtitles_cmd_string, downloader_config (write a yt-dlp config file), check_fetch_comments / dl_fetch_comments / store_comments_in_db, extractor_args_list. ### 3.2 `OptionsParser` A second class that walks the canonical option list (a registry of `OptionHolder` entries) and emits a yt-dlp CLI: `--retries 30 --match-filter "duration > 60" -o "