feat(go): port FIM/pkgdb/rootcheck integrity engines and add go-sidecar dashboard consumer

Adds sidecar-only, --state-dir-gated FIM, package-DB-anchor, and rootcheck
engines to the Go migration sidecar, wired into agent.Sweep/Initialize
behind the existing baseline lifecycle. Adds a read-only, schema-checked
Python dashboard consumer (go_sidecar_state_dir, /api/go-sidecar/*, Sidecar
tab) that never starts, stops, or mutates the Go sidecar's state.

Also fixes drift found while reconciling this work: enodia_sentinel/web.py
had reinvented local schema constants instead of using the canonical
enodia_sentinel/schemas.py catalog (now registers enodia.go.sidecar.v1
there and reuses ALERT_SNAPSHOT_V1/INCIDENT_V1); docs/RULES.md had drifted
from what `rules docs` actually generates for SIDs 100069-100078 (ruleops.py
was missing metadata for four SIDs and had stale drill text for four more),
now back in sync with a regression test pinning them together.

Updates CLAUDE.md, README.md, go-agent/README.md, and docs/{ROADMAP,
GO_PORT_HANDOFF,SURICATA_ASSIMILATION,THREAT_MODEL,OPERATIONS,SCHEMAS,
COMMAND_REFERENCE}.md to reflect the landed work.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQivSKBqQJsayz1xcYqWzZ
This commit is contained in:
Luna 2026-07-24 09:59:09 -07:00
parent 1d1dee7f6e
commit d835386381
No known key found for this signature in database
43 changed files with 3419 additions and 72 deletions

View file

@ -1,9 +1,15 @@
# Go Port Handoff
Saved: 2026-07-22T04:08:40-07:00
Updated: 2026-07-24T04:45:55-07:00
Branch: `main`
Base commit: `a25c73a` (`feat(go): report integrity anchor context`)
Status: current Go-port state saved in this checkpoint; separate GUI work remains uncommitted
Base commit: `a25c73a` (`feat(go): report integrity anchor context`); the FIM/
package-DB/rootcheck engines and the Python go-sidecar dashboard consumer
described below landed on top of that commit and remain uncommitted.
Status: items 1 and 2 of "Resume here" are now implemented and green; see the
updated sections below. Desktop GUI work referenced by the original checkpoint
has since been committed (`457c760`, `1d1dee7`, and related commits) and is no
longer part of this checkout's uncommitted state.
## Worktree warning
@ -14,10 +20,12 @@ continuations. Do not reset, clean, or broadly restage it.
assurance, asynchronous-enrichment, incident-reader, package-ownership, and
integrity-anchor tranches are signed commits `f85c2e8`, `538d1d9`,
`fd2bbba`, `1ab4add`, `eff31b3`, `6355fb4`, `88d7a6e`, and `a25c73a`.
- At this checkpoint, `git status --short` has 11 entries with untracked
directories collapsed.
- The Python GUI files and tests are separate pre-existing work. Preserve them
while continuing the Go port.
- On top of that, uncommitted work now adds `go-agent/internal/{fim,pkgdb,
rootcheck}/`, wires them into `agent.Sweep`/`Initialize` behind
`--state-dir`, and adds Python's read-only `/api/go-sidecar/*` consumer
(`enodia_sentinel/web.py`, `schemas.GO_SIDECAR_V1`) plus a Sidecar dashboard
tab. `git status --short` currently shows this as modified files across both
trees plus the three new untracked Go package directories.
- `build/` is ignored and contains only local build output.
## Implemented Go surface
@ -106,15 +114,46 @@ static binary.
- The asynchronous worker now resolves pacman/dpkg/rpm package-owner strings
with a two-second bound and cache; ownership lookup never holds a detector
lock or blocks alert capture.
- The worker now reports FIM-baseline, package-DB-anchor, and Go hash-chain
presence/age. It explicitly marks package verification and rootcheck disabled
until their actual engines are ported.
- Richer integrity engines and notification fan-out are not yet ported.
- The worker reports FIM-baseline, package-DB-anchor, and Go hash-chain
presence/age, plus the configured `pacman -Qkk` and rootcheck cadences now
that those engines are ported (see below). This remains metadata only:
recording a snapshot never starts integrity work itself.
- Notification fan-out is not yet ported.
- `--incidents-list` and `--incident-show <id>` now read the sidecar's isolated
state without starting the agent. The latter returns `enodia.incident.view.v1`
with the incident, available snapshots, and time-ordered timeline.
- No live system service was installed or enabled during development.
### FIM, package-DB anchor, and rootcheck engines (new since the base commit)
- `go-agent/internal/fim` is a sidecar-only engine, gated on `--state-dir`,
that persists a SHA-256 security-path baseline as `fim-baseline.json` and
performs non-overlapping background comparisons on `fim_scan_interval`,
emitting the Python-identical SIDs `100017`-`100019`. `fim_pkg_verify = true`
opts into a bounded `pacman -Qkk` verification pass emitting SID `100020`.
- `go-agent/internal/pkgdb` fingerprints pacman's local checksum database on
`pkgdb_interval` and alerts (SID `100021`) only on changes lacking a
corresponding transaction-log record; it never refreshes an unexplained
changed anchor.
- `go-agent/internal/rootcheck` bounds PID cross-view probing with
`rootcheck_pid_cap`, runs one background comparison per
`rootcheck_interval`, and reproduces the Python rootcheck SIDs (hidden
process/module/port families, promiscuous mode, module/kernel taint) from
procfs, sysfs, `ps`, and socket views.
- All three run asynchronously off the sweep loop (`agent.Sweep` calls
`MaybeStart` before detector evaluation and `Drain` after, so a slow hash
tree or `pacman` invocation never delays a sweep) and return alerts through
the existing cooldown/snapshot/incident pipeline rather than emitting
directly.
- `enodia_sentinel/web.py` adds a read-only, schema-checked consumer for this
evidence: `go_sidecar_state_dir` (config key) exposes `/api/go-sidecar/
{status,alerts,incidents,events}` and a dashboard Sidecar tab. It never
starts, stops, or writes into the Go sidecar's state directory, and a
corrupted retained event log fails the request closed rather than showing a
partial stream. The new `enodia.go.sidecar.v1` schema lives in
`enodia_sentinel/schemas.py` and is pinned by
`tests/test_schema_contracts.py::test_go_sidecar_v1_contract`.
## Last green verification
Run from the repository root:
@ -132,7 +171,7 @@ make check-go-service
git diff --check
```
Results at save time:
Results at original save time (`a25c73a`):
- Python: 377 tests passed.
- Go unit and race suites: all packages passed through `a25c73a`.
@ -152,19 +191,44 @@ Results at save time:
startup/quiet-period retention, process context, and Python dashboard-reader
compatibility.
Results re-verified 2026-07-24 with the FIM/pkgdb/rootcheck engines and
go-sidecar dashboard consumer added on top:
- Python: 385 tests passed (`unittest discover`).
- Go unit and `-race` suites: all packages passed, including the new `fim`,
`pkgdb`, and `rootcheck` packages.
- `go vet` and `go mod verify`: passed.
- Parity counts are unchanged from the original checkpoint (22/5/7/10/22/3/2),
as expected: the new engines are integrity checks outside the poll/exec/
syscall/host-rule parity fixture.
- `systemd-analyze verify` and the three Go sidecar packaging/isolation tests:
passed.
- `git diff --check`: passed (no whitespace errors).
- Static cross-arch builds were not re-run this pass.
The Python suite prints three expected CLI error lines from negative tests and
one existing `ResourceWarning` for an implicitly cleaned-up HTTP 401 object;
the suite still exits successfully.
## Resume here
1. Port the actual FIM/package-verification/rootcheck engines behind explicit,
bounded acceptance gates; do not treat enrichment anchor status as detection
parity.
2. Add a read-only management API consumer for isolated Go snapshot/event state
while keeping Python authoritative.
1. ~~Port the actual FIM/package-verification/rootcheck engines behind
explicit, bounded acceptance gates~~ — done: `go-agent/internal/{fim,pkgdb,
rootcheck}`, gated on `--state-dir`, tested, and wired into `agent.Sweep`.
2. ~~Add a read-only management API consumer for isolated Go snapshot/event
state while keeping Python authoritative~~ — done: `/api/go-sidecar/*` in
`enodia_sentinel/web.py` plus the dashboard Sidecar tab, schema-checked and
never controlling the Go process.
3. Continue broader Phase 3 rule metadata/state parity before considering any
default-service or package cutover.
default-service or package cutover. This is the remaining item from the
original checkpoint.
4. Notification fan-out for the Go sidecar (item noted above) is still
unported; the FIM/pkgdb/rootcheck engines currently return alerts only
through the shared JSONL/snapshot/incident pipeline, not push backends.
5. Consider whether `04-integrity.md` and `05-rootcheck.md` in
`docs/behavior/` (currently "planned") should be written now that these
engines exist on both sides, so future Go-side changes have the same
reviewable behavioral contract as the poll detectors and event rules do.
Keep the validation sidecar opt-in, preserve its separate state tree, and keep
new attack/event primitives disconnected from destructive response paths.