catacomb/docs/superpowers/plans/2026-06-27-android-engine-feasibility-research.md

Android Engine Feasibility Research — Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Produce an evidence-backed go/no-go decision on a standalone on-device Android download engine for Catacomb (yt-dlp + JS runtime + POT), plus a recommended next sub-project if "go".

Architecture: This is a research spike, not a feature build. Each task investigates one question via desk research, performs a cheap emulator reproduction where it materially reduces uncertainty, records evidence in a findings file, and commits. The final task synthesizes a go/no-go verdict. No product code, no UI, no shipped Rust JNI library.

Tech Stack (under investigation, not committed): Android SDK/NDK, the catacomb_test emulator (API 34, x86_64), adb, Chaquopy / youtubedl-android, an embeddable JS engine (QuickJS / WebView), Rust aarch64-linux-android cross-compile + JNI.

Global Constraints

• Spec authority: docs/superpowers/specs/2026-06-27-android-engine-feasibility-research.md. Every task traces to a research question (Q1–Q4) or the synthesis.
• No product code. No app, no Gradle product project, no UI, no shipped JNI lib. Throwaway experiment artifacts only, kept under /tmp/.../scratchpad or a clearly-marked android-spike/ scratch dir — not committed into the crate.
• Only the findings doc is committed to the repo: docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md.
• Android tooling env: always export JAVA_HOME=/usr/lib/jvm/java-17-openjdk and ANDROID_HOME=~/Android/Sdk (system JDK 26 breaks AGP/Gradle). See the android-emulator-env memory.
• Emulator: AVD catacomb_test, reachable as emulator-5554. It gets reaped between turns — relaunch headless before any adb work (see Task 0).
• Reproduce-where-cheap policy: desk research is primary; run on-device proofs only when cheap and uncertainty-reducing; flag (do not attempt) anything needing the user's real device, a Google login, or live-YouTube behavior that won't reproduce headlessly.
• Arch caveat: emulator is x86_64; a real phone is aarch64. On-device runs prove behavior, not the aarch64 toolchain. Keep build-proof (aarch64 compiles) separate from run-proof (x86_64 executes).
• Sandbox download corruption: large SDK/NDK zips must be fetched via curl + manual extract, never bare sdkmanager (it truncates them).
• Verdict vocabulary: every question ends in exactly one of proven / risky / unsolved, with the evidence behind it.

---

Task 0: Bootstrap the findings doc + confirm the emulator is live

Files:

• Create: docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md

Interfaces:

• Consumes: nothing.

• Produces: the findings file with one ## Q1…## Q4 + ## Synthesis heading per later task to append under. Section headings are the contract every later task writes into.

• Step 1: Relaunch the emulator headless and wait for full boot

export ANDROID_HOME=~/Android/Sdk
export PATH="$ANDROID_HOME/platform-tools:$PATH"
pkill -f "avd catacomb_test" 2>/dev/null; sleep 1
nohup "$ANDROID_HOME/emulator/emulator" -avd catacomb_test \
  -no-window -no-audio -no-boot-anim -gpu swiftshader_indirect \
  -no-snapshot -memory 2048 > /tmp/emu.log 2>&1 &
adb wait-for-device
until [ "$(adb shell getprop sys.boot_completed 2>/dev/null | tr -d '\r')" = "1" ]; do sleep 2; done
adb shell getprop ro.build.version.sdk | tr -d '\r'

Expected: prints 34 once boot completes (usually < 60s with KVM).

• Step 2: Create the findings skeleton

# Android Engine Feasibility — Findings

Companion to `2026-06-27-android-engine-feasibility-research.md`.
Each section ends with a verdict: **proven / risky / unsolved**.

## Q1 — yt-dlp on Android

## Q2 — JS runtime (deno's job)

## Q3 — POT / Proof-of-Origin

## Q4 — Rust core via JNI

## Synthesis — go / no-go

• Step 3: Commit

cd /home/luna/code/catacomb
git add docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md
git commit -m "docs(android-spike): scaffold feasibility findings doc"

---

Task 1: Q1 — yt-dlp on Android

Files:

• Modify: docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md (the ## Q1 section)

Interfaces:

• Consumes: a booted emulator (Task 0).

• Produces: Q1 findings + verdict; specifically a named recommended mechanism ("Chaquopy" | "youtubedl-android" | "other") that Task 5 synthesis reads.

• Step 1: Desk research — how real apps run yt-dlp on-device

Investigate and capture, with source links:

• youtubedl-android (yausername) — the lib Seal/Tubular-adjacent apps use: how it bundles Python, how it ships/updates the yt-dlp payload (its updateYoutubeDL/channel mechanism).

• Chaquopy — embedding CPython + pip in an APK: what it costs (APK size, supported ABIs, Python version), and whether pip-installed yt-dlp works at runtime.

• Seal app specifically: which of the above it uses and any documented yt-dlp version-pinning. Record: bundling mechanism, supported ABIs, approximate size cost.

• Step 2: Capture the version/update story (Catacomb-specific worry)

Catacomb desktop relies on nightly yt-dlp for working curl_cffi impersonation (see ytdlp_bin.rs). Document: can an Android yt-dlp be updated post-install (youtubedl-android's download channel? in-app pip?) or is it pinned/stale, and what staleness costs for impersonation. State this explicitly — a pinned-only answer is a partial-no-go signal.

• Step 3: Cheap emulator repro — prove a Python yt-dlp can run on-device

Goal: prove yt-dlp's Python actually imports/executes on Android, not just in theory. Cheapest path that avoids a full Gradle app: push a CPython-for-Android

• yt-dlp wheel via adb into an app-data dir, or run the youtubedl-android sample. Minimum acceptable proof = yt-dlp --version (or python -m yt_dlp --version) executing under Android and printing a version. Capture the exact commands and output.

# illustrative shape — actual mechanism determined in Step 1:
adb shell "cd /data/local/tmp && ./python -m yt_dlp --version"

Expected: a yt-dlp version string printed from within the Android environment. If this proves not-cheap (needs a full app build), STOP, document why, and mark the repro "deferred to Stage-1 prototype" rather than sinking hours here.

• Step 4: Write Q1 findings + verdict

Append to the ## Q1 section: mechanism comparison table, the update-story paragraph, the repro transcript (or the documented reason it was deferred), and a one-line verdict: proven / risky / unsolved with the recommended mechanism.

• Step 5: Commit

git add docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md
git commit -m "docs(android-spike): Q1 yt-dlp-on-Android findings + verdict"

---

Task 2: Q2 — JS runtime (deno's job)

Files:

• Modify: docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md (the ## Q2 section)

Interfaces:

• Consumes: Q1's chosen yt-dlp mechanism (the JS runtime must integrate with it).

• Produces: Q2 findings + verdict; a named JS-runtime recommendation ("QuickJS" | "system WebView JS" | "embedded V8" | "none viable") that Task 5 reads.

• Step 1: Desk research — what JS interpreter yt-dlp uses on Android

yt-dlp needs JS for nsig/signature/challenge solving (deno on desktop). Investigate and capture, with links:

• yt-dlp's --exec/jsinterp and external-interpreter support: which interpreters it accepts (deno, node, quickjs?), and what youtubedl-android wires in.

• QuickJS on Android (e.g. quickjs-android bindings) vs. system WebView JS vs. J2V8/embedded V8: availability, size, and whether yt-dlp can drive them.

• Known fragility: is the JS-challenge path a recurring break point on mobile?

• Step 2: Cheap emulator repro (if isolable)

If yt-dlp's JS path can be exercised in isolation on-device (e.g. running its bundled jsinterp against a sample nsig challenge), do it via adb and capture output. If it can't be isolated without the full extractor flow, document precisely why and what that implies — do not force it.

• Step 3: Write Q2 findings + verdict

Append to ## Q2: the interpreter-options comparison, integration notes with Q1's mechanism, the repro (or why-not), and verdict: proven / risky / unsolved with the recommended runtime.

• Step 4: Commit

git add docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md
git commit -m "docs(android-spike): Q2 JS-runtime findings + verdict"

---

Task 3: Q3 — POT / Proof-of-Origin (biggest unknown)

Files:

• Modify: docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md (the ## Q3 section)

Interfaces:

• Consumes: Q1/Q2 context (POT generation may need the JS runtime / a WebView).

• Produces: Q3 findings + verdict — the verdict that most strongly gates the overall go/no-go in Task 5.

• Step 1: Desk research — POT on mobile

Investigate and capture, with links:

• How bgutil-ytdlp-pot-provider / bgutil-pot work, and whether a WebView-based POT generation path exists for Android (no Node sidecar).

• Crucially: does YouTube gate the mobile client surface the same way the desktop clients Catacomb uses get gated? If mobile/tv/embedded clients aren't POT-gated, the desktop POT machinery may be unnecessary on-device — a materially simpler design. Cite yt-dlp issues / NewPipeExtractor discussion.

• What Seal/NewPipe do about POT today (or don't).

• Step 2: Identify what needs the user's real device

POT/login behavior likely won't reproduce headlessly. Explicitly list which checks need the user's real device or a Google login, and frame them as flagged-for-device rather than blockers on the spike.

• Step 3: Write Q3 findings + verdict

Append to ## Q3: the POT-on-mobile landscape, the "is mobile gated like desktop?" answer (the pivotal one), the flagged-for-device list, and verdict: proven / risky / unsolved. Note explicitly how this verdict propagates to synthesis.

• Step 4: Commit

git add docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md
git commit -m "docs(android-spike): Q3 POT/Proof-of-Origin findings + verdict"

---

Task 4: Q4 — Rust core via JNI

Files:

• Modify: docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md (the ## Q4 section)
• Throwaway scratch only (NOT committed): an android-spike/ dir under the scratchpad for the cross-compile experiment.

Interfaces:

• Consumes: the existing crate's module layout (src/vtt.rs is the chosen pure-logic probe).

• Produces: Q4 findings + verdict + a reuse-vs-reimplement recommendation and a per-module port-cleanliness list.

• Step 1: Desk research + module triage

Read the current crate to classify modules as pure-logic (portable) vs. desktop/subprocess-bound (not portable). Candidates to confirm portable: vtt, library, database, fingerprint, platform, error_class. For each, note dependencies that would block an Android .so (subprocess spawning, GTK/ egui, OS-specific calls). Capture the table.

• Step 2: Install the NDK + Rust android targets (curl workaround if needed)

export JAVA_HOME=/usr/lib/jvm/java-17-openjdk
export ANDROID_HOME=~/Android/Sdk
"$ANDROID_HOME/cmdline-tools/latest/bin/sdkmanager" --install "ndk;26.3.11579264" 2>&1 | tail -3
rustup target add aarch64-linux-android x86_64-linux-android

Expected: NDK present under $ANDROID_HOME/ndk/; both Rust targets installed. If the NDK zip corrupts (sandbox bug), curl it from the repo manifest and extract manually per the Global Constraints.

• Step 3: Build-proof — cross-compile vtt to aarch64 (does the toolchain build?)

In the throwaway scratch dir, create a minimal cdylib crate that depends on a copy of vtt.rs (or a path-dep into the repo, read-only) exposing one JNI fn, and build it for aarch64. This proves the toolchain, per the arch caveat.

cd /tmp/claude-1000/-home-luna-code-catacomb/<session>/scratchpad/android-spike
# configure CC/AR/linker from the NDK toolchain for the target, then:
cargo build --release --target aarch64-linux-android
ls target/aarch64-linux-android/release/*.so

Expected: a .so is produced for aarch64. Capture the command + result.

• Step 4: Run-proof — call an x86_64 build over JNI on the emulator

Build the same lib for x86_64-linux-android, push it + a tiny test harness via adb, and invoke the JNI function on the emulator to prove the call path works end-to-end. (x86_64 because that's the emulator ABI.) If a full JNI harness is not-cheap, downgrade to: push the .so and confirm it dlopens / readelf shows the expected symbol, and document the JNI call as deferred.

adb push target/x86_64-linux-android/release/libcatacomb_spike.so /data/local/tmp/
adb shell "cd /data/local/tmp && readelf -d ./libcatacomb_spike.so | head"

Expected: the lib loads / exposes the symbol on-device. Capture output.

• Step 5: Write Q4 findings + verdict

Append to ## Q4: the module port-cleanliness table, the aarch64 build-proof result, the x86_64 run/load proof, the effort estimate, a reuse-vs-reimplement recommendation, and verdict: proven / risky / unsolved.

• Step 6: Commit (findings only; scratch dir stays out of git)

cd /home/luna/code/catacomb
git status --short   # confirm only the findings doc is staged; no android-spike/ leaked in
git add docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md
git commit -m "docs(android-spike): Q4 Rust-core-via-JNI findings + verdict"

---

Task 5: Synthesis — go/no-go + recommended next sub-project

Files:

• Modify: docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md (the ## Synthesis section)

Interfaces:

• Consumes: the four verdicts (Q1–Q4).

• Produces: the spike's terminal deliverable — a go/no-go recommendation.

• Step 1: Tabulate the four verdicts

Append a table: Question | Verdict | One-line reason. Pull each verbatim from Q1–Q4.

• Step 2: Write the go/no-go recommendation

Apply the decision logic from the spec:

• Go (standalone engine) only if Q1, Q2, Q3 are each at worst risky with a credible path, and Q3 (POT) isn't unsolved.

• Partial / pivot if Q3 is unsolved or yt-dlp can only be stale: recommend a client-to-server app first (reuses the existing web API per remote.rs), revisit standalone later. Name the exact blocking unknown.

• No-go if multiple cores are unsolved. State the recommendation in one paragraph, then the recommended next sub-project with a one-line scope (e.g. "Stage-1 engine prototype: Q1 mechanism + Q2 runtime" OR "Client-to-server Android app against the web API").

• Step 3: Update roadmap pointer

Add a one-line note under ROADMAP.md §3.2 pointing at the findings doc and the verdict, so the roadmap reflects the decision.

# edit ROADMAP.md 3.2 to reference docs/.../findings.md and the go/no-go outcome

• Step 4: Commit

git add docs/superpowers/specs/2026-06-27-android-engine-feasibility-findings.md ROADMAP.md
git commit -m "docs(android-spike): synthesis — go/no-go + next sub-project; link from roadmap"

• Step 5: Tear down the emulator

pkill -f "avd catacomb_test" 2>/dev/null; echo "emulator stopped"

---

Self-Review

Spec coverage:

• Q1 → Task 1 ✅; Q2 → Task 2 ✅; Q3 → Task 3 ✅; Q4 → Task 4 ✅.
• Spec "Deliverable" (findings file, per-question evidence, go/no-go, next sub-project) → Task 0 (scaffold) + Tasks 1–4 (evidence) + Task 5 (go/no-go + next) ✅.
• Spec "reproduce where cheap" → each repro step has an explicit "if not cheap, document and defer" escape ✅.
• Spec arch caveat (build-proof vs run-proof) → Task 4 Steps 3 & 4 are split exactly along that line ✅.
• Spec "success = even a well-evidenced no-go" → Task 5 Step 2 handles no-go / pivot explicitly ✅.
• Spec non-goal "no committed product code" → Global Constraints + Task 4 scratch dir kept out of git, with a git status guard ✅.

Placeholder scan: No TBD/TODO. Repro commands that depend on a Step-1 finding (the yt-dlp bundling mechanism) are marked "illustrative shape — actual mechanism determined in Step 1", which is honest for a research spike rather than a fake concrete command. The <session> token in the scratch path is a real filesystem placeholder the executor substitutes, not a planning gap.

Type consistency: Findings section headings (## Q1…## Q4, ## Synthesis) are defined once in Task 0 Step 2 and referenced identically by every later task. Verdict vocabulary (proven/risky/unsolved) is uniform. AVD name catacomb_test, target triples aarch64-linux-android / x86_64-linux-android, and the findings filename are identical across all tasks.