enodia-sentinal/docs/behavior/01-sweep-and-state.md

1. Sweep, System State, and Baseline Lifecycle

Scope: how the daemon gathers one view of the host, runs poll detectors over it, and manages the baselines that the baseline-diff detectors (first_seen, new_listener, new_suid, persistence) depend on. This is the substrate every other detection subsystem runs on top of.

Source: system.py, daemon.py, alert.py, detectors/__init__.py, config.py.

---

1.1 The alert value

alert.py. Every detection is an immutable Alert:

Field	Type	Meaning
severity	Severity enum	MEDIUM=1, HIGH=2, CRITICAL=3; str() yields the name
signature	str	human rule name, e.g. new_listener
key	str	stable dedup identity, e.g. lis:31337/nc; drives cooldown
detail	str	one-line human description
pids	tuple[int, …]	processes the snapshot should deep-dive (default empty)
sid	int	stable Snort-style signature id (default 0)
classtype	str	category (default uncategorized)

Alert.to_dict() is the serialization contract — key order: sid, severity, signature, classtype, key, detail, pids (with severity as the string name and pids as a list). A port must preserve key values exactly: cooldown dedup is keyed on them, so a differing key changes re-fire behavior.

---

1.2 System state capture

SystemState (system.py) is one sweep's cached view, built once and shared by all detectors. It is fully injectable: constructing it with processes=/sockets= bypasses live collection (how tests and the parity harness avoid /proc and ss).

1.2.1 Processes

SystemState.processes enumerates /proc, keeping entries whose name isdigit(), each wrapped in a Process(pid). Every Process field is a cached_property read lazily from /proc/<pid>/… and fail-open (any OSError → empty value), because a process can exit mid-sweep:

Field	Source	Notes
comm	comm	stripped
cmdline	cmdline	NUL→space, stripped
exe	readlink exe	"" if unreadable
cwd	readlink cwd
environ	environ	NUL-split k=v → dict
fd_targets	fd/	dict fd(str) → link target, fds sorted numerically
memory_maps	maps	see parser below
status	status	key → value (value stripped)
ppid	status[PPid]	0 on parse failure
uid	status[Uid] field 0	0 if absent

stdio_socket_inode() returns the socket inode if fd 0, 1, or 2 links to socket:[…] — the reverse-shell tell — else None.

parse_memory_maps(text) splits each line maxsplit=5; needs ≥5 fields; address start-end parsed as hex (skip line if no - or non-hex); path is field 5 when present else "". Emits MemoryMap(start, end, perms, path). The port must preserve maxsplit=5 semantics so pathnames containing spaces stay intact.

1.2.2 Sockets

SystemState.sockets runs ss -H once per protocol family, in this order, tagging each result with kind:

-tanep tcp   -uanep udp   -wanep raw   -Sanep sctp   -danep dccp
-0anep packet   -Manep mptcp   --tipc -anep tipc   --xdp -anep xdp
--vsock -anep vsock

Each ss call is fail-open (OSError/SubprocessError/10s timeout → ""), so an unsupported family drops only its own rows. _parse_ss per line:

• Split on whitespace; need ≥5 fields.
• If field[0].lower() is a known netid (tcp udp raw sctp dccp mptcp p_raw p_dgr packet tipc xdp vsock) and ≥6 fields: state, local, peer = fields[1], fields[4], fields[5]; otherwise fields[0], fields[3], fields[4]. (Handles both ss layouts — with and without a leading netid column.)
• inode from regex \bino:(\d+) or None.
• comm, pid from regex users:\(\("([^"]+)",pid=(\d+),fd=\d+ or ""/None.

A None pid is meaningful and must round-trip: it serializes as ? in prose and as a distinct key component. Socket fields: state, local, peer, inode, comm, pid, kind.

Derived views: net_peer_by_inode (inode → "local -> peer", network sockets only), listening_sockets() (state == "LISTEN"), established_sockets() (state == "ESTAB").

listener_keys() maps current listeners to identity keys port/comm where port = local.rpartition(":")[2] and comm defaults to ?. This is the identity used by new_listener and by the listener baseline.

1.2.3 SUID scan (expensive, gated)

scan_suid_binaries(root="/", extra_dirs=()) (system.py):

1. Walk root staying on its device (the find -xdev equivalent) via an explicit os.scandir stack; collect regular files whose mode has S_ISUID | S_ISGID.
2. Also walk each extra_dirs entry that is a directory, ignoring the device check — these are the small writable mounts (/tmp, /dev/shm, …) that are usually separate tmpfs yet are exactly where a hostile SUID binary is dropped.
3. Return the sorted union.

Per-entry errors are skipped (fail-open). The walk is expensive, which is why the daemon gates its cadence (§1.5).

---

1.3 Detector registry and gating

detectors/__init__.py. Detectors are pure detect(state, cfg) -> Iterable [Alert]. REGISTRY order is the order signatures appear in a snapshot and must be preserved by the port:

reverse_shell, ld_preload, deleted_exe, input_snooper, credential_access,
stealth_network, memory_obfuscation, egress,
first_seen*, new_listener*, persistence*, new_suid*†

* = needs_baseline=True; † new_suid is also expensive=True.

run_all(state, cfg, include=None) yields, per registry entry, when:

1. cfg.enabled(name) is true, and
2. include is None or contains the name, and
3. if needs_baseline: state.listener_baseline is not None.

Condition 3 is the arming gate: listener_baseline is None skips all four baseline-diff detectors at once — including first_seen, even though first_seen never reads listener_baseline. first_seen's own gate is this shared flag, not its store.

---

1.4 The daemon sweep loop

daemon.py:run() sequence:

1. mkdir log_dir; write sentinel.pid; append a start line to events.log.
2. build_baselines() — see §1.5.1. Note: run() always builds; it does not load.
3. load_fim_baseline(); snapshot.prune(); start eBPF exec + syscall monitors (each logs enabled/disabled to events.log; failure degrades to None, never fatal).
4. Loop until stopped, once per sample_interval (default 4.0s, via self._stop.wait(...) so stop is prompt):
   • write_heartbeat() (dead-man's switch, every iteration).
   • Only once (now - start_time) >= baseline_grace (default 10s): kick the cadence-gated background scanners — SUID, FIM, package verify, pkgdb anchor, pkgdb signature verify, rootcheck. Each runs off-thread and self-limits by its own interval; results feed back through _on_exec_alert.
   • alerts = sweep() (§1.5).
   • fresh = fresh_alerts(alerts, now) (§1.6).
   • If any fresh: capture a snapshot off the loop thread (a slow snapshot must never stall detection; the forensic SystemState is rebuilt fresh inside that thread).
   • Every 20 sweeps: snapshot.prune().

stop() sets the event and stops both eBPF monitors.

Asynchronous producers (eBPF monitors, FIM/pkg/rootcheck threads) do not go through sweep(); they call _on_exec_alert(alert, live_fps=None), which runs the same fresh_alerts dedup and then captures off-thread. So the cooldown and reconciliation chokepoint (§1.6) is shared by loop and async paths.

---

1.5 Baseline lifecycle

1.5.1 Listener and SUID baselines

Held on the daemon as set[str]: listener_baseline, suid_baseline.

• build_baselines() (called by run() at start, and by cli check): listener_baseline = SystemState().listener_keys() (a fresh live capture); suid_baseline = set(scan_suid_binaries(extra_dirs=suid_scan_extra_dirs)). Both are then written to disk:

  • log_dir/listener-baseline.json — JSON array, sorted.
  • log_dir/suid-baseline.json — JSON array, sorted.

  Consequence: on daemon start the baseline is whatever is running / present at that moment. Anything already listening or already SUID at start is, by definition, baseline and will not alert. The stored files are for the one-shot CLI path, not reloaded by the running daemon.

• load_baselines() reads those two files ([] on any error). It is used only by cli check (cli.py), never by run().

1.5.2 Arming (the grace window)

Inside sweep(force_suid=False):

armed = (now - start_time) >= baseline_grace         # default 10s
gate  = armed or force_suid
state = SystemState(
    listener_baseline = self.listener_baseline if gate else None,
    suid_baseline     = self.suid_baseline,          # always passed
    suid_binaries     = suid_binaries if gate else None,
    persist_since     = last_persist_scan if gate else None,
)
alerts = list(detectors.run_all(state, cfg))
last_persist_scan = now                              # advance AFTER the sweep

So during the first baseline_grace seconds the baseline-diff detectors are inert (their inputs are None), suppressing the startup burst. suid_baseline is always populated but only consumed when suid_binaries is present.

cli check passes force_suid=True: it bypasses the grace gate and computes suid_binaries synchronously via scan_suid_binaries (rather than reading the async result), so a single-shot check gets a complete SUID view.

1.5.3 SUID scan cadence and baseline refresh

The live SUID list is produced off the loop thread:

• _maybe_scan_suid(now): skip if a scan thread is alive or (now - last_suid_scan) < suid_scan_interval (default 60s); else record last_suid_scan = now and spawn _scan_suid.
• _scan_suid(): self._suid_current = scan_suid_binaries(...). Then, every suid_refresh (default 3600s), fold the current scan back into the baseline (suid_baseline = set(result), re-save suid-baseline.json) so legitimately installed SUID binaries stop alerting after a while.

sweep() reads self._suid_current, which is None until the first async scan completes — so new_suid is a no-op for the first cycle or two after start even once armed. A port using synchronous-on-cadence scanning must reproduce the same observable rule: no new_suid alerts until a scan result exists, and periodic baseline refresh on the suid_refresh interval.

1.5.4 persist_since

last_persist_scan starts at start_time and is set to now at the end of every sweep(). The persistence detector alerts on watched files whose mtime > persist_since, i.e. modified since the previous sweep. Passing the previous sweep's timestamp (advanced afterward) is the contract.

1.5.5 first_seen store

detectors/first_seen.py owns log_dir/first-seen.json, independent of the listener/SUID baselines. Format (schema enodia.first_seen.v1):

{
  "schema": "enodia.first_seen.v1",
  "initialized": false,
  "public_destinations": { "<comm>": ["ip:port", ...] },
  "listener_ports":      { "<comm>": ["port", ...] }
}

Behavior each time the detector runs (recall: only when armed):

1. Load the store (missing/corrupt → the default above).
2. For each ESTAB socket to a public peer (netutil.is_public_ip) with a port: if comm's public_destinations set lacks ip:port, add it (list kept sorted); and if initialized, emit SID 100074 first_public_destination MEDIUM, key firstdest:<comm>:<ip:port>, classtype network-rarity.
3. For each LISTEN/UNCONN socket of kind tcp/udp/"" with a numeric non-zero port: if comm's listener_ports set lacks it, add it (sorted numerically); and if initialized, emit SID 100075 first_listener_port MEDIUM, key firstlisten:<comm>:<port>, classtype network-rarity.
4. Set initialized = true and save (atomic: write *.json.tmp, os.replace, chmod 0640 best-effort).

Net effect: the first armed run on a fresh store learns silently (initialized was false) and thereafter alerts once per genuinely new comm→destination / comm→port. Because initialized persists to disk, a daemon restart does not re-enter the silent-learning phase — unlike the listener/SUID baselines, which are rebuilt every start. This asymmetry is deliberate and must be preserved.

---

1.6 Dedup, cooldown, and reconciliation chokepoint

fresh_alerts(alerts, now, live_fps=None) is the single gate every alert passes (loop and async):

1. Reconciliation first — if any alerts, load ReconcileStore and filter_alerts(alerts, live_fps or {}), dropping operator-acknowledged drift so an acked item never even consumes a cooldown slot. live_fps carries current fingerprints for content kinds (e.g. {"fim": {...}}); identity kinds need none. (Detailed in §8.)
2. Cooldown — under _cooldown_lock, for each surviving alert: emit it and stamp cooldowns[key] = now only if (now - cooldowns.get(key, 0)) >= cooldown (default 60s). Same key inside the window is suppressed.

Order matters: reconcile before cooldown.

---

1.7 Timing knobs (config.py defaults)

Key	Default	Governs
sample_interval	4.0 s	loop period
cooldown	60 s	per-key alert dedup window
baseline_grace	10 s	arming delay before baseline-diff detectors + background scans
suid_scan_interval	60 s	SUID rescan cadence
suid_refresh	3600 s	fold current SUID scan into baseline
suid_scan_extra_dirs	writable mounts tuple	extra (off-device) SUID walk roots
first_seen_enabled	true	first_seen on/off
log_dir	/var/log/enodia-sentinel	store + evidence root; ENODIA_LOG_DIR override

Store paths derive from log_dir: listener-baseline.json, suid-baseline.json, first-seen.json, fim-baseline.json, events.log, sentinel.pid.

---

1.8 Port parity notes (Go 2.0)

What the Go agent must reproduce to be behavior-equivalent for this subsystem:

• Registry order and gating (§1.3), including the shared arming gate that silences first_seen while unarmed.
• Baseline provenance (§1.5.1): baseline captured at start, not loaded, in the daemon path; loaded in the one-shot path.
• Time-based arming (§1.5.2), not first-sweep arming.
• SUID cadence + refresh + "no alert until a scan exists" (§1.5.3) — the Go design uses synchronous-on-cadence scanning, which is allowed as long as the observable rules hold.
• persist_since = previous sweep time, advanced after the sweep (§1.5.4).
• first-seen.json format, learn-once semantics, and restart asymmetry (§1.5.5); the file format is byte-compatible so either agent can read a store the other wrote.
• Reconcile-before-cooldown ordering and 60s per-key cooldown (§1.6).

Current Go status (as of this writing): poll detectors are ported and pass the parity fixture, but baselines are still injected from fixtures — the store package, live SUID walk, and the daemon-equivalent lifecycle above are the work this document specifies.