enodia-sentinal/go-agent/README.md

Enodia Sentinel Go Agent (Migration Sidecar)

This directory is the parallel Go implementation described in docs/SURICATA_ASSIMILATION.md. Python remains the production agent and the behavioral oracle until every subsystem reaches fixture and red-team parity.

Current scope:

• standard-library-only flat TOML loading for the settings used so far;
• an injectable /proc process/file-descriptor/memory-map snapshot plus fail-open ss socket collection;
• a cancellable sweep loop that emits JSONL enodia.event.v1 records;
• exact enodia.alert.v1 parity for ld_preload (SID 100011), deleted_exe (100012), input_snooper (100032), and credential_access (100033), plus reverse_shell (100010), egress (100016), stealth_network (100036), and the /proc/<pid>/maps memory_obfuscation cohort (100039, 100048, 100049), and first_seen (100074, 100075), new_listener (100013), persistence (100015), and new_suid (100014);
• an opt-in live baseline lifecycle (--state-dir) that builds listener/SUID baselines, honors the startup grace gate, scans persistence paths, and atomically retains first-seen.json using the Python-compatible schema;
• a sidecar-only FIM engine, enabled only with --state-dir, that persists a SHA-256 security-path baseline as fim-baseline.json, performs non-overlapping background comparisons, and can opt in to a bounded pacman -Qkk package verification pass with fim_pkg_verify = true;
• a sidecar-only rootcheck engine that bounds PID cross-view probing with rootcheck_pid_cap, runs one background comparison per interval, and checks procfs, sysfs, ps, socket, module-taint, kernel-taint, and promiscuous-mode views without blocking the polling loop;
• a sidecar-only package-DB anchor (pkgdb_verify) that fingerprints pacman's local checksum database and alerts only on changes that lack a corresponding transaction-log record; it never refreshes an unexplained changed anchor;
• pure exec and syscall event models plus replayable rule engines: built-in exec SIDs 100001–100004, configured [[exec_rules]], and syscall SIDs 100060–100066 now emit Python-identical alert envelopes without requiring a kernel probe;
• typed host-event SIDs 100067–100073 and 100076–100078, a unified mixed JSONL pipeline, Python-compatible rules-list/rules-show metadata, correlation SID 100080, and a shared per-key cooldown gate;
• opt-in native exec (--ebpf-exec) and security-syscall (--ebpf-syscall) tracepoint sources built with cilium/ebpf v0.22.0 and CO-RE parent-process lookup. They feed the same rule/cooldown/envelope paths as replay and degrade to polling when loading, attaching, or reading is unavailable. The syscall probe filters its security-relevant syscall numbers in-kernel and also routes setuid/setgid into typed host-event SIDs 100071/100072, plus chmod/chown variants into SID 100070, capset into SID 100077, and finit_module into SID 100078 after resolving the module FD path. Confirmed interpreter write, writev, pwrite64, pwritev, and pwritev2 calls are enter/exit-correlated for SID 100069. Successful IPv4/IPv6 connect, bind, listen, and accept calls feed SIDs 100067, 100073, 100068, and 100076;
• enodia.status.v1 heartbeat records whose alert totals, severity counts, and last-alert time are derived from the retained Go snapshot store;
• optional bounded JSONL retention (--event-log) with private permissions, one rotated segment, and synchronous flushes for non-status records;
• optional enodia.alert.snapshot.v1 JSON/text pairs (--snapshot-dir) with same-second alert batching, best-effort /proc context, and Python-compatible count/age pruning;
• Python-compatible enodia.incident.v1 grouping and atomic incidents.json persistence, using process lineage first and the shared time-window fallback, including additive SID 100080 correlation evidence;
• a shared fixture checked against the Python detector by scripts/check-go-parity.py.

It does not yet provide Python's full enrichment or notification fan-out, and it does not replace enodia-sentinel. Python's optional /api/go-sidecar/* consumer can display this sidecar's isolated retained evidence, but remains read-only and never starts or controls the Go process. The module requires Go 1.25 or newer; development is currently verified with Go 1.26.5. Run a single terminal-visible stateless sweep:

cd go-agent
GOCACHE=/tmp/enodia-go-cache go run ./cmd/enodia-sentinel-go --once

Print the packaged agent identity with enodia-sentinel-go --version.

Exercise the live baseline lifecycle in an isolated directory (do not point the migration sidecar at the production Python agent's state directory):

GOCACHE=/tmp/enodia-go-cache go run ./cmd/enodia-sentinel-go \
  --state-dir /tmp/enodia-sentinel-go-state

Development verification from the repository root:

make generate-go  # only after editing internal/ebpfsource/bpf/*.bpf.c
make test-go
make parity-go

Build and install the opt-in production validation sidecar from the repository root:

make build-go
sudo make install-go
sudo make enable-go
make status-go
make logs-go

enodia-sentinel-go-sidecar.service writes JSONL to the system journal and keeps baseline lifecycle files under /var/lib/enodia-sentinel-go. It also retains the same envelopes in events.jsonl, rotating to events.jsonl.1 before the active file exceeds 64 MiB. Both files are mode 0600; alert and incident records are synchronized before emission succeeds, while routine status records use normal kernel writeback. It never shares the Python daemon's /var/log/enodia-sentinel state. Both native probes are requested by the unit; a probe load failure is visible in status events and the polling loop remains active. Installing the sidecar does not enable it or alter enodia-sentinel.service. The unit reaches active only after baseline initialization, event-log and snapshot-store preflight, and both requested probe load attempts finish. Snapshot-store preflight includes the private assurance chain.

Alert envelopes are also batched into private alert-YYYYMMDD-HHMMSS.{json,log} pairs under the same isolated directory. The JSON side implements the required enodia.alert.snapshot.v1 fields and the service applies shared max_snapshots / max_snapshot_age_days limits. Status counts are populated only from parseable retained snapshots. Each snapshot is linked to an incident in a private atomic incidents.json index. Grouping uses the configured incident_tracking, incident_window, and incident_lineage_depth settings; same-second alert batches update one incident and one snapshot name idempotently. Hot-path enrichment derives bounded process/parent context, lineage, remote-IP classification, and candidate paths from already-captured data without new I/O. A 16-job asynchronous worker then adds streamed SHA-256 values for regular executables up to 8 MiB, file metadata, and pacman/dpkg/rpm package-owner strings with a two-second query bound; saturation leaves fields unknown rather than delaying alerts. The sidecar also has separate, bounded FIM/package-verification, package-DB-anchor, and rootcheck engines; notification fan-out remains future parity work. The worker reports truthful FIM-baseline, package-DB-anchor, and Go hash-chain presence/age, plus the configured pacman -Qkk and rootcheck cadences. This is metadata only: recording a snapshot never starts integrity work. Snapshot text/JSON revisions append synchronized enodia.hash_chain.v1 records to a private local hash-chain.jsonl.

After every successfully emitted status record, the service atomically updates /var/lib/enodia-sentinel-go/heartbeat with the current Unix timestamp (mode 0600). The file remains after shutdown so external watchdogs can identify a stale sensor.

Check that signal with JSON output and a watchdog-friendly exit status:

enodia-sentinel-go --health
# For an isolated development run:
enodia-sentinel-go --health --state-dir /tmp/enodia-sentinel-go-state

The command uses heartbeat_max_age from the shared configuration, defaults to the service state path when --state-dir is omitted, and exits nonzero for a missing, malformed, or stale heartbeat.

For an ad-hoc retained run, set an explicit path and byte bound:

enodia-sentinel-go --state-dir /tmp/enodia-go-state \
  --event-log /tmp/enodia-go-state/events.jsonl \
  --event-log-max-bytes 8388608

Inspect recent retained records across both the active and rotated segments:

sudo enodia-sentinel-go --events-tail 50
# Or inspect an ad-hoc state tree:
enodia-sentinel-go --events-tail 50 --state-dir /tmp/enodia-go-state

The reader returns chronological JSONL, validates every source record, and fails nonzero rather than forwarding corrupted evidence. It does not load the agent configuration, so retained evidence remains available while a broken configuration prevents the service from starting.

Inspect the isolated incident index without starting the agent:

enodia-sentinel-go --incidents-list --state-dir /tmp/enodia-go-state
enodia-sentinel-go --incident-show inc-20260722-101112-abcd --state-dir /tmp/enodia-go-state

--incident-show returns enodia.incident.view.v1 with the durable incident, available snapshot reports, and a time-ordered timeline. These read-only commands never modify or consult the Python daemon state.

--fixture, --host, --timestamp, --proc-root, and --suid-root exist for deterministic parity/testing. Fixture mode never activates the live baseline manager. Production work should continue to use the default live /proc, hostname, and local timestamp.

The kernel sources are explicitly opt-in while this remains a sidecar. They require the privileges needed to load BPF programs and create maps. A load failure is reported on stderr and in enodia.status.v1, then the poll loop continues:

sudo go run ./cmd/enodia-sentinel-go --ebpf-exec --ebpf-syscall

Offline event-rule replays remain available without BPF or root:

go run ./cmd/enodia-sentinel-go --exec-events ../tests/fixtures/go/exec-events.json
go run ./cmd/enodia-sentinel-go --syscall-events ../tests/fixtures/go/syscall-events.json
go run ./cmd/enodia-sentinel-go --host-events ../tests/fixtures/go/host-events.json
go run ./cmd/enodia-sentinel-go --event-stream ../tests/fixtures/go/mixed-events.jsonl
go run ./cmd/enodia-sentinel-go --rules-list