feat(go): advance validation sidecar toward production
This commit is contained in:
Luna
parent
6a06eba255
commit
f85c2e831a
92 changed files with 8881 additions and 91 deletions
|
|
@ -1,4 +1,4 @@
|
|||
# Enodia Sentinel Go Agent (Phase 1 Prototype)
|
||||
# Enodia Sentinel Go Agent (Migration Sidecar)
|
||||
|
||||
This directory is the parallel Go implementation described in
|
||||
`docs/SURICATA_ASSIMILATION.md`. Python remains the production agent and the
|
||||
|
|
@ -14,27 +14,160 @@ Current scope:
|
|||
`deleted_exe` (`100012`), `input_snooper` (`100032`), and
|
||||
`credential_access` (`100033`), plus `reverse_shell` (`100010`), `egress`
|
||||
(`100016`), `stealth_network` (`100036`), and the `/proc/<pid>/maps`
|
||||
`memory_obfuscation` cohort (`100039`, `100048`, `100049`);
|
||||
- `enodia.status.v1` heartbeat records with no persistence or retained-alert
|
||||
claims;
|
||||
`memory_obfuscation` cohort (`100039`, `100048`, `100049`), and
|
||||
`first_seen` (`100074`, `100075`), `new_listener` (`100013`),
|
||||
`persistence` (`100015`), and `new_suid` (`100014`);
|
||||
- an opt-in live baseline lifecycle (`--state-dir`) that builds listener/SUID
|
||||
baselines, honors the startup grace gate, scans persistence paths, and
|
||||
atomically retains `first-seen.json` using the Python-compatible schema;
|
||||
- pure exec and syscall event models plus replayable rule engines: built-in
|
||||
exec SIDs `100001`–`100004`, configured `[[exec_rules]]`, and syscall SIDs
|
||||
`100060`–`100066` now emit Python-identical alert envelopes without requiring
|
||||
a kernel probe;
|
||||
- typed host-event SIDs `100067`–`100073` and `100076`–`100078`, a unified
|
||||
mixed JSONL pipeline, Python-compatible `rules-list`/`rules-show` metadata,
|
||||
correlation SID `100080`, and a shared per-key cooldown gate;
|
||||
- opt-in native exec (`--ebpf-exec`) and security-syscall (`--ebpf-syscall`)
|
||||
tracepoint sources built with `cilium/ebpf` v0.22.0 and CO-RE parent-process
|
||||
lookup. They feed the same rule/cooldown/envelope paths as replay and degrade
|
||||
to polling when loading, attaching, or reading is unavailable. The syscall
|
||||
probe filters its security-relevant syscall numbers in-kernel and also routes
|
||||
`setuid`/`setgid` into typed host-event SIDs `100071`/`100072`, plus
|
||||
`chmod`/`chown` variants into SID `100070`, `capset` into SID `100077`, and
|
||||
`finit_module` into SID `100078` after resolving the module FD path. Confirmed
|
||||
interpreter `write`, `writev`, `pwrite64`, `pwritev`, and `pwritev2` calls are
|
||||
enter/exit-correlated for SID `100069`.
|
||||
Successful IPv4/IPv6 `connect`, `bind`, `listen`, and `accept` calls feed SIDs
|
||||
`100067`, `100073`, `100068`, and `100076`;
|
||||
- `enodia.status.v1` heartbeat records whose alert totals, severity counts, and
|
||||
last-alert time are derived from the retained Go snapshot store;
|
||||
- optional bounded JSONL retention (`--event-log`) with private permissions,
|
||||
one rotated segment, and synchronous flushes for non-status records;
|
||||
- optional `enodia.alert.snapshot.v1` JSON/text pairs (`--snapshot-dir`) with
|
||||
same-second alert batching, best-effort `/proc` context, and Python-compatible
|
||||
count/age pruning;
|
||||
- a shared fixture checked against the Python detector by
|
||||
`scripts/check-go-parity.py`.
|
||||
|
||||
It does not install a service, write Python state, capture eBPF events, retain
|
||||
snapshots, or replace `enodia-sentinel`. Run a single terminal-visible sweep:
|
||||
It does not yet provide Python's incident grouping, rich enrichment, assurance
|
||||
chain, or notification fan-out for those snapshots, and it does not replace
|
||||
`enodia-sentinel`. The module requires Go 1.25 or newer; development is currently
|
||||
verified with Go 1.26.5. Run a single terminal-visible stateless sweep:
|
||||
|
||||
```bash
|
||||
cd go-agent
|
||||
GOCACHE=/tmp/enodia-go-cache go run ./cmd/enodia-sentinel-go --once
|
||||
```
|
||||
|
||||
Print the packaged agent identity with `enodia-sentinel-go --version`.
|
||||
|
||||
Exercise the live baseline lifecycle in an isolated directory (do not point the
|
||||
migration sidecar at the production Python agent's state directory):
|
||||
|
||||
```bash
|
||||
GOCACHE=/tmp/enodia-go-cache go run ./cmd/enodia-sentinel-go \
|
||||
--state-dir /tmp/enodia-sentinel-go-state
|
||||
```
|
||||
|
||||
Development verification from the repository root:
|
||||
|
||||
```bash
|
||||
make generate-go # only after editing internal/ebpfsource/bpf/*.bpf.c
|
||||
make test-go
|
||||
make parity-go
|
||||
```
|
||||
|
||||
`--fixture`, `--host`, `--timestamp`, and `--proc-root` exist for deterministic
|
||||
parity/testing. Production work should continue to use the default live
|
||||
`/proc`, hostname, and local timestamp.
|
||||
Build and install the opt-in production validation sidecar from the repository
|
||||
root:
|
||||
|
||||
```bash
|
||||
make build-go
|
||||
sudo make install-go
|
||||
sudo make enable-go
|
||||
make status-go
|
||||
make logs-go
|
||||
```
|
||||
|
||||
`enodia-sentinel-go-sidecar.service` writes JSONL to the system journal and
|
||||
keeps baseline lifecycle files under `/var/lib/enodia-sentinel-go`. It also
|
||||
retains the same envelopes in `events.jsonl`, rotating to `events.jsonl.1`
|
||||
before the active file exceeds 64 MiB. Both files are mode `0600`; alert and
|
||||
incident records are synchronized before emission succeeds, while routine
|
||||
status records use normal kernel writeback. It never shares the Python daemon's
|
||||
`/var/log/enodia-sentinel` state. Both native probes are requested by the unit;
|
||||
a probe load failure is visible in status events and the polling loop remains
|
||||
active. Installing the sidecar does not enable it or alter
|
||||
`enodia-sentinel.service`. The unit reaches `active` only after baseline
|
||||
initialization, event-log and snapshot-store preflight, and both requested probe
|
||||
load attempts finish.
|
||||
|
||||
Alert envelopes are also batched into private
|
||||
`alert-YYYYMMDD-HHMMSS.{json,log}` pairs under the same isolated directory.
|
||||
The JSON side implements the required `enodia.alert.snapshot.v1` fields and the
|
||||
service applies shared `max_snapshots` / `max_snapshot_age_days` limits. Status
|
||||
counts are populated only from parseable retained snapshots. Enrichment is
|
||||
currently limited to a Go-source marker plus best-effort process detail;
|
||||
incident indexing and the Python forensic enrichment/assurance pipeline remain
|
||||
future parity work.
|
||||
|
||||
After every successfully emitted status record, the service atomically updates
|
||||
`/var/lib/enodia-sentinel-go/heartbeat` with the current Unix timestamp (mode
|
||||
`0600`). The file remains after shutdown so external watchdogs can identify a
|
||||
stale sensor.
|
||||
|
||||
Check that signal with JSON output and a watchdog-friendly exit status:
|
||||
|
||||
```bash
|
||||
enodia-sentinel-go --health
|
||||
# For an isolated development run:
|
||||
enodia-sentinel-go --health --state-dir /tmp/enodia-sentinel-go-state
|
||||
```
|
||||
|
||||
The command uses `heartbeat_max_age` from the shared configuration, defaults to
|
||||
the service state path when `--state-dir` is omitted, and exits nonzero for a
|
||||
missing, malformed, or stale heartbeat.
|
||||
|
||||
For an ad-hoc retained run, set an explicit path and byte bound:
|
||||
|
||||
```bash
|
||||
enodia-sentinel-go --state-dir /tmp/enodia-go-state \
|
||||
--event-log /tmp/enodia-go-state/events.jsonl \
|
||||
--event-log-max-bytes 8388608
|
||||
```
|
||||
|
||||
Inspect recent retained records across both the active and rotated segments:
|
||||
|
||||
```bash
|
||||
sudo enodia-sentinel-go --events-tail 50
|
||||
# Or inspect an ad-hoc state tree:
|
||||
enodia-sentinel-go --events-tail 50 --state-dir /tmp/enodia-go-state
|
||||
```
|
||||
|
||||
The reader returns chronological JSONL, validates every source record, and
|
||||
fails nonzero rather than forwarding corrupted evidence. It does not load the
|
||||
agent configuration, so retained evidence remains available while a broken
|
||||
configuration prevents the service from starting.
|
||||
|
||||
`--fixture`, `--host`, `--timestamp`, `--proc-root`, and `--suid-root` exist for
|
||||
deterministic parity/testing. Fixture mode never activates the live baseline
|
||||
manager. Production work should continue to use the default live `/proc`,
|
||||
hostname, and local timestamp.
|
||||
|
||||
The kernel sources are explicitly opt-in while this remains a sidecar. They
|
||||
require the privileges needed to load BPF programs and create maps. A load
|
||||
failure is reported on stderr and in `enodia.status.v1`, then the poll loop
|
||||
continues:
|
||||
|
||||
```bash
|
||||
sudo go run ./cmd/enodia-sentinel-go --ebpf-exec --ebpf-syscall
|
||||
```
|
||||
|
||||
Offline event-rule replays remain available without BPF or root:
|
||||
|
||||
```bash
|
||||
go run ./cmd/enodia-sentinel-go --exec-events ../tests/fixtures/go/exec-events.json
|
||||
go run ./cmd/enodia-sentinel-go --syscall-events ../tests/fixtures/go/syscall-events.json
|
||||
go run ./cmd/enodia-sentinel-go --host-events ../tests/fixtures/go/host-events.json
|
||||
go run ./cmd/enodia-sentinel-go --event-stream ../tests/fixtures/go/mixed-events.jsonl
|
||||
go run ./cmd/enodia-sentinel-go --rules-list
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue