enodia-sentinal/docs/GO_PORT_HANDOFF.md

Go Port Handoff

Saved: 2026-07-22T04:08:40-07:00 Updated: 2026-07-24T04:45:55-07:00 Branch: main Base commit: a25c73a (feat(go): report integrity anchor context); the FIM/ package-DB/rootcheck engines and the Python go-sidecar dashboard consumer described below landed on top of that commit and remain uncommitted. Status: items 1 and 2 of "Resume here" are now implemented and green; see the updated sections below. Desktop GUI work referenced by the original checkpoint has since been committed (457c760, 1d1dee7, and related commits) and is no longer part of this checkout's uncommitted state.

Worktree warning

The checkout is intentionally dirty and contains work from multiple related continuations. Do not reset, clean, or broadly restage it.

• The validation-sidecar, incident-persistence, bounded-enrichment, local- assurance, asynchronous-enrichment, incident-reader, package-ownership, and integrity-anchor tranches are signed commits f85c2e8, 538d1d9, fd2bbba, 1ab4add, eff31b3, 6355fb4, 88d7a6e, and a25c73a.
• On top of that, uncommitted work now adds go-agent/internal/{fim,pkgdb, rootcheck}/, wires them into agent.Sweep/Initialize behind --state-dir, and adds Python's read-only /api/go-sidecar/* consumer (enodia_sentinel/web.py, schemas.GO_SIDECAR_V1) plus a Sidecar dashboard tab. git status --short currently shows this as modified files across both trees plus the three new untracked Go package directories.
• build/ is ignored and contains only local build output.

Implemented Go surface

The Go module requires Go 1.25 or newer and is currently verified with go1.26.5-X:nodwarf5. It uses github.com/cilium/ebpf v0.22.0 and builds as a static binary.

Polling and lifecycle

• All current Python poll detectors have Go parity: reverse_shell, ld_preload, deleted_exe, input_snooper, credential_access, stealth_network, memory_obfuscation, egress, first_seen, new_listener, persistence, and new_suid.
• --state-dir enables isolated listener/SUID baselines, startup grace, persistence scans, and atomic first-seen retention.
• The Go state directory must never be the Python daemon's /var/log/enodia-sentinel directory.

Event rules and native sources

• Exec rules: built-in SIDs 100001-100004 plus configured rules.
• Syscall rules: SIDs 100060-100066.
• Typed host rules: SIDs 100067-100073 and 100076-100078.
• Mixed JSONL routing, rule catalog operations, correlation SID 100080, and one shared cooldown gate are implemented.
• Embedded CO-RE exec and filtered raw-syscall probes use cilium/ebpf and fail open to polling on load/attach/read failures.
• Native typed transports include UID/GID changes, chmod/chown variants, capset, finit_module, confirmed interpreter write/writev/pwrite64/ pwritev/pwritev2, and successful TCP connect/bind/listen/accept calls with /proc endpoint resolution.
• Generated little- and big-endian BPF objects live under go-agent/internal/ebpfsource/. Run make generate-go only after changing go-agent/internal/ebpfsource/bpf/*.bpf.c.

Production validation path

• make build-go creates a stripped, static, VCS-independent binary.
• make install-go installs the separate binary and enodia-sentinel-go-sidecar.service without replacing or enabling the Python service.
• The hardened systemd unit has a current systemd-analyze security exposure score of 4.3 (OK). It preserves the host /proc, /tmp, and network view because detectors require them, while hiding the Python state tree.
• The unit uses Type=notify; READY is sent only after baseline setup, event log preflight, and both requested probe-load attempts.
• JSONL remains visible in journald and is also retained as private bounded /var/lib/enodia-sentinel-go/events.jsonl plus events.jsonl.1 (64 MiB per active segment boundary). Non-status events are synchronized to disk.
• /var/lib/enodia-sentinel-go/heartbeat is atomically refreshed after each successful status emission.
• enodia-sentinel-go --health returns enodia.go.health.v1 JSON and a watchdog-friendly exit status.
• enodia-sentinel-go --events-tail N reads active and rotated retained events chronologically, validates JSON, fails closed on corruption, and deliberately works without loading the agent configuration.
• The unit now retains private alert-YYYYMMDD-HHMMSS.{json,log} pairs using the required enodia.alert.snapshot.v1 fields. Same-second alerts merge into one pair, process context is collected best-effort from /proc, and shared max_snapshots / max_snapshot_age_days limits are applied at startup, capture, and status time.
• enodia.status.v1 totals, severity counts, and last-alert time are populated only from parseable retained Go snapshots. Python's unchanged list_alerts and get_alert data layer successfully consumed a Go-produced pair.
• Snapshot incident IDs now reference an atomic private incidents.json using the required enodia.incident.v1 fields. Grouping mirrors Python's process- lineage-first, time-window fallback behavior, the index is bounded to 1000 recent incidents, and same-second alert batches update one incident without duplicating the snapshot name.
• Incident severity/signatures/SIDs/PIDs/lineage/timestamps are updated additively and correlation SID 100080 is persisted. Python's unchanged list_incidents and get_incident consumers successfully loaded a live Go-produced index, snapshot, and timeline.
• A bounded no-I/O enrichment stage now derives Python-shaped process/parent context, lineage, remote-IP classification, and candidate paths entirely from already-captured records. Alert/process/indicator caps bound attacker- controlled work on the emission path.
• Snapshot log/JSON revisions now append synchronized private enodia.hash_chain.v1 records. Artifact SHA-256 is streamed, previous-hash lookup reads a bounded chain tail, and append completion is synced before the alert event is emitted.
• A single 16-job asynchronous worker now streams SHA-256 only for regular executables at most 8 MiB and adds lstat file metadata. Slow I/O occurs outside the snapshot lock; a full queue simply leaves optional fields unknown.
• The asynchronous worker now resolves pacman/dpkg/rpm package-owner strings with a two-second bound and cache; ownership lookup never holds a detector lock or blocks alert capture.
• The worker reports FIM-baseline, package-DB-anchor, and Go hash-chain presence/age, plus the configured pacman -Qkk and rootcheck cadences now that those engines are ported (see below). This remains metadata only: recording a snapshot never starts integrity work itself.
• Notification fan-out is not yet ported.
• --incidents-list and --incident-show <id> now read the sidecar's isolated state without starting the agent. The latter returns enodia.incident.view.v1 with the incident, available snapshots, and time-ordered timeline.
• No live system service was installed or enabled during development.

FIM, package-DB anchor, and rootcheck engines (new since the base commit)

• go-agent/internal/fim is a sidecar-only engine, gated on --state-dir, that persists a SHA-256 security-path baseline as fim-baseline.json and performs non-overlapping background comparisons on fim_scan_interval, emitting the Python-identical SIDs 100017-100019. fim_pkg_verify = true opts into a bounded pacman -Qkk verification pass emitting SID 100020.
• go-agent/internal/pkgdb fingerprints pacman's local checksum database on pkgdb_interval and alerts (SID 100021) only on changes lacking a corresponding transaction-log record; it never refreshes an unexplained changed anchor.
• go-agent/internal/rootcheck bounds PID cross-view probing with rootcheck_pid_cap, runs one background comparison per rootcheck_interval, and reproduces the Python rootcheck SIDs (hidden process/module/port families, promiscuous mode, module/kernel taint) from procfs, sysfs, ps, and socket views.
• All three run asynchronously off the sweep loop (agent.Sweep calls MaybeStart before detector evaluation and Drain after, so a slow hash tree or pacman invocation never delays a sweep) and return alerts through the existing cooldown/snapshot/incident pipeline rather than emitting directly.
• enodia_sentinel/web.py adds a read-only, schema-checked consumer for this evidence: go_sidecar_state_dir (config key) exposes /api/go-sidecar/ {status,alerts,incidents,events} and a dashboard Sidecar tab. It never starts, stops, or writes into the Go sidecar's state directory, and a corrupted retained event log fails the request closed rather than showing a partial stream. The new enodia.go.sidecar.v1 schema lives in enodia_sentinel/schemas.py and is pinned by tests/test_schema_contracts.py::test_go_sidecar_v1_contract.

Last green verification

Run from the repository root:

python3 -m unittest discover -s tests -q
cd go-agent
GOCACHE=/tmp/enodia-go-cache go test ./...
GOCACHE=/tmp/enodia-go-cache go test -race ./...
GOCACHE=/tmp/enodia-go-cache go vet ./...
go mod verify
cd ..
make parity-go
make check-go-service
git diff --check

Results at original save time (a25c73a):

• Python: 377 tests passed.
• Go unit and race suites: all packages passed through a25c73a.
• go vet and go mod verify: passed.
• Parity: 22 poll alerts, 5 exec alerts, 7 syscall alerts, 10 typed-host alerts, 22 catalog records, 3 mixed-stream alerts, and 2 correlations through 88d7a6e; the later integrity-anchor change is enrichment-only.
• systemd-analyze verify and the three Go sidecar packaging/isolation tests: passed.
• Static builds: linux/amd64, linux/arm64, linux/riscv64, and linux/s390x passed.
• Integration smoke tests passed for static binary identity, JSON emission, READY/STOPPING notification, heartbeat freshness/staleness, retained event equivalence, tail selection, corruption rejection, and evidence reading with malformed TOML. Snapshot smoke tests also passed for same-second batching, severity aggregation, private JSON/text pairs, truthful status counts, startup/quiet-period retention, process context, and Python dashboard-reader compatibility.

Results re-verified 2026-07-24 with the FIM/pkgdb/rootcheck engines and go-sidecar dashboard consumer added on top:

• Python: 385 tests passed (unittest discover).
• Go unit and -race suites: all packages passed, including the new fim, pkgdb, and rootcheck packages.
• go vet and go mod verify: passed.
• Parity counts are unchanged from the original checkpoint (22/5/7/10/22/3/2), as expected: the new engines are integrity checks outside the poll/exec/ syscall/host-rule parity fixture.
• systemd-analyze verify and the three Go sidecar packaging/isolation tests: passed.
• git diff --check: passed (no whitespace errors).
• Static cross-arch builds were not re-run this pass.

The Python suite prints three expected CLI error lines from negative tests and one existing ResourceWarning for an implicitly cleaned-up HTTP 401 object; the suite still exits successfully.

Resume here

1. Port the actual FIM/package-verification/rootcheck engines behind explicit, bounded acceptance gates — done: go-agent/internal/{fim,pkgdb, rootcheck}, gated on --state-dir, tested, and wired into agent.Sweep.
2. Add a read-only management API consumer for isolated Go snapshot/event state while keeping Python authoritative — done: /api/go-sidecar/* in enodia_sentinel/web.py plus the dashboard Sidecar tab, schema-checked and never controlling the Go process.
3. Continue broader Phase 3 rule metadata/state parity before considering any default-service or package cutover. This is the remaining item from the original checkpoint.
4. Notification fan-out for the Go sidecar (item noted above) is still unported; the FIM/pkgdb/rootcheck engines currently return alerts only through the shared JSONL/snapshot/incident pipeline, not push backends.
5. Consider whether 04-integrity.md and 05-rootcheck.md in docs/behavior/ (currently "planned") should be written now that these engines exist on both sides, so future Go-side changes have the same reviewable behavioral contract as the poll detectors and event rules do.

Keep the validation sidecar opt-in, preserve its separate state tree, and keep new attack/event primitives disconnected from destructive response paths.