feat(go): port FIM/pkgdb/rootcheck integrity engines and add go-sidecar dashboard consumer

Adds sidecar-only, --state-dir-gated FIM, package-DB-anchor, and rootcheck
engines to the Go migration sidecar, wired into agent.Sweep/Initialize
behind the existing baseline lifecycle. Adds a read-only, schema-checked
Python dashboard consumer (go_sidecar_state_dir, /api/go-sidecar/*, Sidecar
tab) that never starts, stops, or mutates the Go sidecar's state.

Also fixes drift found while reconciling this work: enodia_sentinel/web.py
had reinvented local schema constants instead of using the canonical
enodia_sentinel/schemas.py catalog (now registers enodia.go.sidecar.v1
there and reuses ALERT_SNAPSHOT_V1/INCIDENT_V1); docs/RULES.md had drifted
from what `rules docs` actually generates for SIDs 100069-100078 (ruleops.py
was missing metadata for four SIDs and had stale drill text for four more),
now back in sync with a regression test pinning them together.

Updates CLAUDE.md, README.md, go-agent/README.md, and docs/{ROADMAP,
GO_PORT_HANDOFF,SURICATA_ASSIMILATION,THREAT_MODEL,OPERATIONS,SCHEMAS,
COMMAND_REFERENCE}.md to reflect the landed work.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQivSKBqQJsayz1xcYqWzZ

docs/COMMAND_REFERENCE.md

@@ -471,6 +471,12 @@ HTTPS is mandatory. If `web_tls_cert` / `web_tls_key` are not configured,
 Sentinel creates a self-signed pair under `log_dir`; add a browser exception for
 that certificate or replace it with a private/local CA certificate.
 
+When `go_sidecar_state_dir` is explicitly configured, the same authenticated
+read-only server also exposes the isolated migration-sidecar evidence at
+`/api/go-sidecar/status`, `/api/go-sidecar/alerts`, `/api/go-sidecar/events`,
+and `/api/go-sidecar/incidents`. These paths never start, stop, or alter the Go
+sidecar, and do not replace the Python daemon's normal API data.
+
 The console exposes incidents, alert snapshots, posture findings,
 integrity/watchdog state, event-rule metadata, event tail, and dry-run response
 plans. It remains read-only; response commands are displayed for review but not

docs/GO_PORT_HANDOFF.md

@@ -1,9 +1,15 @@
 # Go Port Handoff
 
 Saved: 2026-07-22T04:08:40-07:00
+Updated: 2026-07-24T04:45:55-07:00
 Branch: `main`
-Base commit: `a25c73a` (`feat(go): report integrity anchor context`)
-Status: current Go-port state saved in this checkpoint; separate GUI work remains uncommitted
+Base commit: `a25c73a` (`feat(go): report integrity anchor context`); the FIM/
+package-DB/rootcheck engines and the Python go-sidecar dashboard consumer
+described below landed on top of that commit and remain uncommitted.
+Status: items 1 and 2 of "Resume here" are now implemented and green; see the
+updated sections below. Desktop GUI work referenced by the original checkpoint
+has since been committed (`457c760`, `1d1dee7`, and related commits) and is no
+longer part of this checkout's uncommitted state.
 
 ## Worktree warning
 
@@ -14,10 +20,12 @@ continuations. Do not reset, clean, or broadly restage it.
   assurance, asynchronous-enrichment, incident-reader, package-ownership, and
   integrity-anchor tranches are signed commits `f85c2e8`, `538d1d9`,
   `fd2bbba`, `1ab4add`, `eff31b3`, `6355fb4`, `88d7a6e`, and `a25c73a`.
-- At this checkpoint, `git status --short` has 11 entries with untracked
-  directories collapsed.
-- The Python GUI files and tests are separate pre-existing work. Preserve them
-  while continuing the Go port.
+- On top of that, uncommitted work now adds `go-agent/internal/{fim,pkgdb,
+  rootcheck}/`, wires them into `agent.Sweep`/`Initialize` behind
+  `--state-dir`, and adds Python's read-only `/api/go-sidecar/*` consumer
+  (`enodia_sentinel/web.py`, `schemas.GO_SIDECAR_V1`) plus a Sidecar dashboard
+  tab. `git status --short` currently shows this as modified files across both
+  trees plus the three new untracked Go package directories.
 - `build/` is ignored and contains only local build output.
 
 ## Implemented Go surface
@@ -106,15 +114,46 @@ static binary.
 - The asynchronous worker now resolves pacman/dpkg/rpm package-owner strings
   with a two-second bound and cache; ownership lookup never holds a detector
   lock or blocks alert capture.
-- The worker now reports FIM-baseline, package-DB-anchor, and Go hash-chain
-  presence/age. It explicitly marks package verification and rootcheck disabled
-  until their actual engines are ported.
-- Richer integrity engines and notification fan-out are not yet ported.
+- The worker reports FIM-baseline, package-DB-anchor, and Go hash-chain
+  presence/age, plus the configured `pacman -Qkk` and rootcheck cadences now
+  that those engines are ported (see below). This remains metadata only:
+  recording a snapshot never starts integrity work itself.
+- Notification fan-out is not yet ported.
 - `--incidents-list` and `--incident-show <id>` now read the sidecar's isolated
   state without starting the agent. The latter returns `enodia.incident.view.v1`
   with the incident, available snapshots, and time-ordered timeline.
 - No live system service was installed or enabled during development.
 
+### FIM, package-DB anchor, and rootcheck engines (new since the base commit)
+
+- `go-agent/internal/fim` is a sidecar-only engine, gated on `--state-dir`,
+  that persists a SHA-256 security-path baseline as `fim-baseline.json` and
+  performs non-overlapping background comparisons on `fim_scan_interval`,
+  emitting the Python-identical SIDs `100017`-`100019`. `fim_pkg_verify = true`
+  opts into a bounded `pacman -Qkk` verification pass emitting SID `100020`.
+- `go-agent/internal/pkgdb` fingerprints pacman's local checksum database on
+  `pkgdb_interval` and alerts (SID `100021`) only on changes lacking a
+  corresponding transaction-log record; it never refreshes an unexplained
+  changed anchor.
+- `go-agent/internal/rootcheck` bounds PID cross-view probing with
+  `rootcheck_pid_cap`, runs one background comparison per
+  `rootcheck_interval`, and reproduces the Python rootcheck SIDs (hidden
+  process/module/port families, promiscuous mode, module/kernel taint) from
+  procfs, sysfs, `ps`, and socket views.
+- All three run asynchronously off the sweep loop (`agent.Sweep` calls
+  `MaybeStart` before detector evaluation and `Drain` after, so a slow hash
+  tree or `pacman` invocation never delays a sweep) and return alerts through
+  the existing cooldown/snapshot/incident pipeline rather than emitting
+  directly.
+- `enodia_sentinel/web.py` adds a read-only, schema-checked consumer for this
+  evidence: `go_sidecar_state_dir` (config key) exposes `/api/go-sidecar/
+  {status,alerts,incidents,events}` and a dashboard Sidecar tab. It never
+  starts, stops, or writes into the Go sidecar's state directory, and a
+  corrupted retained event log fails the request closed rather than showing a
+  partial stream. The new `enodia.go.sidecar.v1` schema lives in
+  `enodia_sentinel/schemas.py` and is pinned by
+  `tests/test_schema_contracts.py::test_go_sidecar_v1_contract`.
+
 ## Last green verification
 
 Run from the repository root:
@@ -132,7 +171,7 @@ make check-go-service
 git diff --check
 ```
 
-Results at save time:
+Results at original save time (`a25c73a`):
 
 - Python: 377 tests passed.
 - Go unit and race suites: all packages passed through `a25c73a`.
@@ -152,19 +191,44 @@ Results at save time:
   startup/quiet-period retention, process context, and Python dashboard-reader
   compatibility.
 
+Results re-verified 2026-07-24 with the FIM/pkgdb/rootcheck engines and
+go-sidecar dashboard consumer added on top:
+
+- Python: 385 tests passed (`unittest discover`).
+- Go unit and `-race` suites: all packages passed, including the new `fim`,
+  `pkgdb`, and `rootcheck` packages.
+- `go vet` and `go mod verify`: passed.
+- Parity counts are unchanged from the original checkpoint (22/5/7/10/22/3/2),
+  as expected: the new engines are integrity checks outside the poll/exec/
+  syscall/host-rule parity fixture.
+- `systemd-analyze verify` and the three Go sidecar packaging/isolation tests:
+  passed.
+- `git diff --check`: passed (no whitespace errors).
+- Static cross-arch builds were not re-run this pass.
+
 The Python suite prints three expected CLI error lines from negative tests and
 one existing `ResourceWarning` for an implicitly cleaned-up HTTP 401 object;
 the suite still exits successfully.
 
 ## Resume here
 
-1. Port the actual FIM/package-verification/rootcheck engines behind explicit,
-   bounded acceptance gates; do not treat enrichment anchor status as detection
-   parity.
-2. Add a read-only management API consumer for isolated Go snapshot/event state
-   while keeping Python authoritative.
+1. ~~Port the actual FIM/package-verification/rootcheck engines behind
+   explicit, bounded acceptance gates~~ — done: `go-agent/internal/{fim,pkgdb,
+   rootcheck}`, gated on `--state-dir`, tested, and wired into `agent.Sweep`.
+2. ~~Add a read-only management API consumer for isolated Go snapshot/event
+   state while keeping Python authoritative~~ — done: `/api/go-sidecar/*` in
+   `enodia_sentinel/web.py` plus the dashboard Sidecar tab, schema-checked and
+   never controlling the Go process.
 3. Continue broader Phase 3 rule metadata/state parity before considering any
-   default-service or package cutover.
+   default-service or package cutover. This is the remaining item from the
+   original checkpoint.
+4. Notification fan-out for the Go sidecar (item noted above) is still
+   unported; the FIM/pkgdb/rootcheck engines currently return alerts only
+   through the shared JSONL/snapshot/incident pipeline, not push backends.
+5. Consider whether `04-integrity.md` and `05-rootcheck.md` in
+   `docs/behavior/` (currently "planned") should be written now that these
+   engines exist on both sides, so future Go-side changes have the same
+   reviewable behavioral contract as the poll detectors and event rules do.
 
 Keep the validation sidecar opt-in, preserve its separate state tree, and keep
 new attack/event primitives disconnected from destructive response paths.

docs/OPERATIONS.md

@@ -75,7 +75,10 @@ Expected healthy output:
 
 The HTTPS dashboard shows the same posture findings under the Posture tab and
 summarizes FIM/package anchors plus heartbeat freshness under the Integrity tab
-for quick remote review.
+for quick remote review. If the optional `go_sidecar_state_dir` setting points
+at an isolated Go migration-sidecar state directory, a Sidecar tab also shows
+that sidecar's own retained heartbeat, alert snapshots, incidents, and event
+log, read-only and clearly separate from Python's production data.
 
 For SSH/tmux operators, `enodia-sentinel tui` provides a terminal dashboard over
 the same local read-only management state: status, recent alerts, incidents,

docs/ROADMAP.md

@@ -300,9 +300,23 @@ Exit criteria:
   now provide truthful retained counts and basic process context. A private,
   atomic `enodia.incident.v1` index now applies the shared lineage-first/time-
   window grouping, assigns real snapshot `incident_id` values, and persists
-  correlation SID `100080`. The Python service remains the production default
-  until rich enrichment/assurance, management-console consumers, and broader
-  subsystem parity land.
+  correlation SID `100080`. Hot-path enrichment adds bounded process/parent
+  context, lineage, remote-IP classification, and candidate paths synchronously,
+  while a 16-job asynchronous worker adds streamed executable hashes, file
+  metadata, and package-owner strings under a query bound. The sidecar now also
+  has sidecar-only, `--state-dir`-gated FIM (`fim-baseline.json`, SIDs
+  `100017`–`100020`, optional `pacman -Qkk` verification), package-DB anchor
+  (`pkgdb_verify`, SID `100021`), and rootcheck (procfs/sysfs/`ps`/socket/
+  module-taint/kernel-taint/promiscuous-mode cross-view checks) engines that
+  run asynchronously off the sweep loop and return alerts through the same
+  cooldown/snapshot/incident pipeline. Python's dashboard gained a first,
+  explicitly read-only management-console consumer for this migration
+  evidence: an opt-in `go_sidecar_state_dir` setting exposes `/api/go-sidecar/*`
+  endpoints and a Sidecar dashboard tab that display the Go sidecar's retained
+  heartbeat, alert snapshots, incidents, and event log without starting,
+  stopping, or otherwise controlling the Go process. The Python service remains
+  the production default until notification fan-out and broader subsystem
+  parity land.
 
 ## Backlog
 

docs/SCHEMAS.md

@@ -139,6 +139,33 @@ Required fields:
 | `hash_chain` | object | Local append-only hash-chain summary for snapshots and event logs: path, exists flag, record count, and last chain hash. |
 | `read_only` | boolean | Always true for this dashboard API. |
 
+## `enodia.go.sidecar.v1`
+
+Status object returned by `/api/go-sidecar/status`, describing the optional,
+separately configured Go migration-sidecar evidence tree (`go_sidecar_state_dir`).
+This is a read-only view of another process's retained files: the Python web
+process never starts, stops, or probes the Go sidecar, and this schema does not
+appear anywhere in the production Python daemon's own state.
+
+Required fields:
+
+| Field | Type | Meaning |
+|---|---|---|
+| `schema` | string | `enodia.go.sidecar.v1`. |
+| `read_only` | boolean | Always true. |
+| `configured` | boolean | Whether `go_sidecar_state_dir` is set. |
+| `state_dir` | string | The configured path, or `""` if unset. |
+| `available` | boolean | Whether the configured directory currently exists. |
+| `heartbeat` | object | `{"status": "not-configured"\|"missing"\|"fresh"\|"stale", ...}`; `fresh`/`stale` reports also include `timestamp`, `age_seconds`, and `max_age_seconds`. |
+
+The related `/api/go-sidecar/alerts`, `/api/go-sidecar/incidents`, and
+`/api/go-sidecar/events` endpoints replay the Go sidecar's own
+`enodia.alert.snapshot.v1`, `enodia.incident.v1`, and `enodia.event.v1` records
+after schema-checking each one; malformed or unrecognized files are dropped
+(alerts/incidents) or reported as an explicit error rather than partially
+displayed (events), so a damaged retained file cannot be misread as a clean
+evidence stream.
+
 ## `enodia.hash_chain.v1`
 
 JSONL record appended to `hash-chain.jsonl` when Sentinel captures forensic

docs/SURICATA_ASSIMILATION.md

@@ -251,9 +251,13 @@ signature keeps `sid` / `signature` / `classtype` / tests / docs.
   transports. Bounded `enodia.alert.snapshot.v1` JSON/text pairs now retain
   alerts with basic process context and feed truthful status counts. An atomic
   `enodia.incident.v1` index supplies lineage/time grouping, snapshot IDs, and
-  correlation evidence. Rich enrichment/assurance, management-consumer
-  integration, and broader parity must land before the bcc/Python runtime
-  dependency can be dropped.
+  correlation evidence. Hot-path plus asynchronous enrichment (hashes, file
+  metadata, package ownership) now populate snapshots, and sidecar-only,
+  `--state-dir`-gated FIM, package-DB-anchor, and rootcheck integrity engines
+  now run asynchronously off the sweep loop. Python's dashboard has a first
+  read-only management-console consumer (`go_sidecar_state_dir`,
+  `/api/go-sidecar/*`) over this isolated evidence. Broader Phase 3 rule-engine
+  parity must land before the bcc/Python runtime dependency can be dropped.
 - **Phase 3 — Rule-engine parity + assimilation.** `rev`/`reference`/`metadata`,
   statebits (flowbits), thresholding, suppression, and — if rule count warrants
   — the MPM prefilter. `rules list/show/test/docs` parity.

docs/THREAT_MODEL.md

@@ -55,6 +55,26 @@ The default service is hardened and read-mostly. Enabling the bcc eBPF monitor
 requires additional capabilities and memory permissions. That tradeoff is
 explicit: stronger event visibility for a wider runtime permission set.
 
+### Go Migration-Sidecar Evidence Is Isolated and Read-Only
+
+The experimental `go-agent/` sidecar (see `docs/ROADMAP.md`'s Active Migration
+Track) is a separate process with its own state directory, baselines, FIM/
+package-DB/rootcheck engines, and hash-chain. It never shares or mutates the
+Python daemon's `log_dir`, baselines, or anchors, and it is not the production
+agent.
+
+When an operator opts in with `go_sidecar_state_dir`, the Python dashboard adds
+a read-only consumer (`/api/go-sidecar/*`, the Sidecar tab) over that directory.
+It treats the Go sidecar's retained files as untrusted input from another
+process rather than trusted application state: every alert snapshot and
+incident record is schema-checked before display, a corrupted retained event
+log fails the request closed instead of showing a partial stream, and the web
+process never starts, stops, signals, or writes into the Go sidecar's state
+directory. Compromising this consumer therefore requires tampering with the Go
+sidecar's own retained files, which is the same class of local-tampering
+problem `enodia.hash_chain.v1` and the sidecar's own FIM self-watch already
+cover.
+
 ## Detection Assumptions
 
 Sentinel assumes: