Add YouTube POT token provider (bgutil-pot, opt-in)
YouTube increasingly requires a per-video Proof-of-Origin token bound to each video ID before handing back format URLs. Without one, yt-dlp sees empty formats and downloads fail. The upstream solution is the Brainicism bgutil-ytdlp-pot-provider Python plugin paired with a long-running server that mints tokens via BotGuard. We integrate the Rust port (jim60105/bgutil-ytdlp-pot-provider-rs) so the server is a single static binary, no Node.js required. ## Architecture - bgutil-pot binary lives alongside the bundled deno + yt-dlp at ~/.local/share/yt-offline/bin/bgutil-pot. - The matching Python plugin is pip-installed into the bundled venv: python -m pip install bgutil-ytdlp-pot-provider. - Downloader spawns the server lazily on first job (port 4416, loopback only) and kills it on Drop. - Each yt-dlp invocation gets --extractor-args "youtubepot-bgutilhttp:base_url=http://127.0.0.1:4416" appended in spawn_job when use_pot_provider is on and the server child is alive. ## UX - Settings → "POT token provider" row with toggle + Install/Update button (mirrors the bundled yt-dlp row). - Disabled unless use_bundled_ytdlp is also on (plugin lives in that venv). - Refuses install if the bundled yt-dlp venv isn't there yet, with a helpful 428 error. ## Lifecycle - Lazy spawn: ensure_pot_server() runs at the top of start() so the server is up before yt-dlp queries the plugin. - Drop: kill_server() on Downloader drop so we don't leave an orphaned process bound to 4416. - start_pot_provider_update() kills the running child first so the install can overwrite the binary in place (no ETXTBSY). 3 new unit tests cover URL/extractor-args formatting; 77 total pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
parent
4f15ab873f
commit
2696a51cc8
7 changed files with 439 additions and 4 deletions
|
|
@ -185,10 +185,23 @@ pub struct Downloader {
|
|||
/// If true, invoke the bundled yt-dlp under [`ytdlp_bin::bundled_dir`]
|
||||
/// instead of the system PATH yt-dlp.
|
||||
pub use_bundled_ytdlp: bool,
|
||||
/// If true and the bundled yt-dlp is in use, spawn the bgutil-pot
|
||||
/// HTTP server on first download and pass its extractor-args to
|
||||
/// every yt-dlp invocation. See [`crate::pot_provider`].
|
||||
pub use_pot_provider: bool,
|
||||
/// Running bgutil-pot server child. Lazily spawned in [`Self::start`]
|
||||
/// on the first job after the flag turns on; killed in [`Drop`].
|
||||
pot_server: Option<std::process::Child>,
|
||||
}
|
||||
|
||||
impl Downloader {
|
||||
pub fn new(channels_root: PathBuf, browser: String, max_concurrent: usize, use_bundled_ytdlp: bool) -> Self {
|
||||
pub fn new(
|
||||
channels_root: PathBuf,
|
||||
browser: String,
|
||||
max_concurrent: usize,
|
||||
use_bundled_ytdlp: bool,
|
||||
use_pot_provider: bool,
|
||||
) -> Self {
|
||||
Self {
|
||||
jobs: Vec::new(),
|
||||
pending: VecDeque::new(),
|
||||
|
|
@ -196,6 +209,24 @@ impl Downloader {
|
|||
browser,
|
||||
max_concurrent,
|
||||
use_bundled_ytdlp,
|
||||
use_pot_provider,
|
||||
pot_server: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Lazy-spawn the POT server on first use. We don't spin it up on
|
||||
/// app start because most users won't have it installed; doing the
|
||||
/// check + spawn on first download means there's no per-launch
|
||||
/// penalty when the feature is off, and an obvious place to surface
|
||||
/// "did you install the binary?" errors when it's on.
|
||||
fn ensure_pot_server(&mut self) {
|
||||
if !self.use_pot_provider { return; }
|
||||
if !self.use_bundled_ytdlp { return; } // plugin is in the bundled venv
|
||||
if self.pot_server.is_some() { return; } // already running
|
||||
if !crate::pot_provider::installed() { return; } // not installed, skip silently
|
||||
match crate::pot_provider::spawn_server() {
|
||||
Ok(child) => { self.pot_server = Some(child); }
|
||||
Err(_) => { /* failure surfaces as missing POT → yt-dlp warns */ }
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -325,6 +356,12 @@ impl Downloader {
|
|||
live: bool,
|
||||
channel_options: Option<&DownloadOptions>,
|
||||
) {
|
||||
// Make sure the POT server is up before we hand the URL to
|
||||
// yt-dlp — the Python plugin checks the HTTP endpoint at
|
||||
// extractor-time, so a too-late start means the first request
|
||||
// misses POT and YouTube hands back empty formats.
|
||||
self.ensure_pot_server();
|
||||
|
||||
let platform_dir = platform::platform_root(&self.channels_root, info.platform);
|
||||
// Per-platform download archive keeps cross-platform IDs from colliding
|
||||
// (TikTok IDs are numeric, YouTube IDs are 11-char base64, etc.).
|
||||
|
|
@ -551,8 +588,37 @@ impl Downloader {
|
|||
self.enqueue(cmd, url, label);
|
||||
}
|
||||
|
||||
/// Enqueue a job that installs (or updates) the bgutil-pot binary
|
||||
/// and pip-installs the matching Python plugin into the bundled
|
||||
/// venv. Same UX shape as [`Self::start_ytdlp_update`] — progress
|
||||
/// streams into the Downloads modal.
|
||||
///
|
||||
/// Before enqueueing we kill any already-running server child so
|
||||
/// the install can overwrite the binary in place without an "ETXTBSY"
|
||||
/// from the OS. The next download after install completes will
|
||||
/// re-spawn the server via [`Self::ensure_pot_server`].
|
||||
pub fn start_pot_provider_update(&mut self) {
|
||||
if let Some(mut child) = self.pot_server.take() {
|
||||
crate::pot_provider::kill_server(&mut child);
|
||||
}
|
||||
let cmd = crate::pot_provider::install_command();
|
||||
let url = "https://github.com/jim60105/bgutil-ytdlp-pot-provider-rs/releases/latest".to_string();
|
||||
let label = "install bgutil-pot + Python plugin".to_string();
|
||||
self.enqueue(cmd, url, label);
|
||||
}
|
||||
|
||||
/// Spawn `cmd` on a background thread, streaming its output into a new [`Job`].
|
||||
fn spawn_job(&mut self, mut cmd: Command, url: String, label: String) {
|
||||
// POT provider extractor-arg. yt-dlp lets us pass multiple
|
||||
// --extractor-args flags; this one points the bgutil plugin at
|
||||
// our local server. Only emitted when the user opted in *and*
|
||||
// the server child is actually running — yt-dlp would warn
|
||||
// about an unreachable base_url otherwise.
|
||||
if self.use_pot_provider && self.pot_server.is_some() {
|
||||
cmd.arg("--extractor-args")
|
||||
.arg(crate::pot_provider::extractor_args());
|
||||
}
|
||||
|
||||
let (tx, rx) = channel();
|
||||
cmd.stdin(Stdio::null())
|
||||
.stdout(Stdio::piped())
|
||||
|
|
@ -709,6 +775,18 @@ impl Downloader {
|
|||
}
|
||||
}
|
||||
|
||||
impl Drop for Downloader {
|
||||
/// Tear down the bgutil-pot child if we spawned one. Without this
|
||||
/// the server keeps running after yt-offline exits — orphaned and
|
||||
/// still bound to port 4416, blocking the next launch from
|
||||
/// re-spawning.
|
||||
fn drop(&mut self) {
|
||||
if let Some(mut child) = self.pot_server.take() {
|
||||
crate::pot_provider::kill_server(&mut child);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Current UNIX timestamp in seconds. Used to disambiguate live-recording
|
||||
/// filenames at job-start time.
|
||||
fn now_unix() -> u64 {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue