# 1. Sweep, System State, and Baseline Lifecycle Scope: how the daemon gathers one view of the host, runs poll detectors over it, and manages the baselines that the baseline-diff detectors (`first_seen`, `new_listener`, `new_suid`, `persistence`) depend on. This is the substrate every other detection subsystem runs on top of. Source: `system.py`, `daemon.py`, `alert.py`, `detectors/__init__.py`, `config.py`. --- ## 1.1 The alert value `alert.py`. Every detection is an immutable `Alert`: | Field | Type | Meaning | |-------|------|---------| | `severity` | `Severity` enum | `MEDIUM=1`, `HIGH=2`, `CRITICAL=3`; `str()` yields the name | | `signature` | str | human rule name, e.g. `new_listener` | | `key` | str | **stable dedup identity**, e.g. `lis:31337/nc`; drives cooldown | | `detail` | str | one-line human description | | `pids` | tuple[int, …] | processes the snapshot should deep-dive (default empty) | | `sid` | int | stable Snort-style signature id (default `0`) | | `classtype` | str | category (default `uncategorized`) | `Alert.to_dict()` is the serialization contract — key order: `sid, severity, signature, classtype, key, detail, pids` (with `severity` as the string name and `pids` as a list). A port must preserve `key` values exactly: cooldown dedup is keyed on them, so a differing key changes re-fire behavior. --- ## 1.2 System state capture `SystemState` (`system.py`) is **one sweep's cached view**, built once and shared by all detectors. It is fully injectable: constructing it with `processes=`/`sockets=` bypasses live collection (how tests and the parity harness avoid `/proc` and `ss`). ### 1.2.1 Processes `SystemState.processes` enumerates `/proc`, keeping entries whose name `isdigit()`, each wrapped in a `Process(pid)`. Every `Process` field is a `cached_property` read lazily from `/proc//…` and **fail-open** (any `OSError` → empty value), because a process can exit mid-sweep: | Field | Source | Notes | |-------|--------|-------| | `comm` | `comm` | stripped | | `cmdline` | `cmdline` | NUL→space, stripped | | `exe` | readlink `exe` | `""` if unreadable | | `cwd` | readlink `cwd` | | | `environ` | `environ` | NUL-split `k=v` → dict | | `fd_targets` | `fd/` | dict `fd(str) → link target`, fds sorted numerically | | `memory_maps` | `maps` | see parser below | | `status` | `status` | `key → value` (value stripped) | | `ppid` | `status[PPid]` | `0` on parse failure | | `uid` | `status[Uid]` field 0 | `0` if absent | `stdio_socket_inode()` returns the socket inode if fd 0, 1, or 2 links to `socket:[…]` — the reverse-shell tell — else `None`. `parse_memory_maps(text)` splits each line `maxsplit=5`; needs ≥5 fields; address `start-end` parsed as hex (skip line if no `-` or non-hex); `path` is field 5 when present else `""`. Emits `MemoryMap(start, end, perms, path)`. **The port must preserve `maxsplit=5` semantics** so pathnames containing spaces stay intact. ### 1.2.2 Sockets `SystemState.sockets` runs `ss -H` **once per protocol family**, in this order, tagging each result with `kind`: ``` -tanep tcp -uanep udp -wanep raw -Sanep sctp -danep dccp -0anep packet -Manep mptcp --tipc -anep tipc --xdp -anep xdp --vsock -anep vsock ``` Each `ss` call is fail-open (`OSError`/`SubprocessError`/10s timeout → `""`), so an unsupported family drops only its own rows. `_parse_ss` per line: - Split on whitespace; need ≥5 fields. - If `field[0].lower()` is a known netid (`tcp udp raw sctp dccp mptcp p_raw p_dgr packet tipc xdp vsock`) **and** ≥6 fields: `state, local, peer = fields[1], fields[4], fields[5]`; otherwise `fields[0], fields[3], fields[4]`. (Handles both `ss` layouts — with and without a leading netid column.) - `inode` from regex `\bino:(\d+)` or `None`. - `comm`, `pid` from regex `users:\(\("([^"]+)",pid=(\d+),fd=\d+` or `""`/`None`. A `None` pid is **meaningful** and must round-trip: it serializes as `?` in prose and as a distinct key component. `Socket` fields: `state, local, peer, inode, comm, pid, kind`. Derived views: `net_peer_by_inode` (`inode → "local -> peer"`, network sockets only), `listening_sockets()` (`state == "LISTEN"`), `established_sockets()` (`state == "ESTAB"`). `listener_keys()` maps current listeners to identity keys `port/comm` where `port = local.rpartition(":")[2]` and `comm` defaults to `?`. This is the identity used by `new_listener` and by the listener baseline. ### 1.2.3 SUID scan (expensive, gated) `scan_suid_binaries(root="/", extra_dirs=())` (`system.py`): 1. Walk `root` **staying on its device** (the `find -xdev` equivalent) via an explicit `os.scandir` stack; collect regular files whose mode has `S_ISUID | S_ISGID`. 2. **Also** walk each `extra_dirs` entry that is a directory, **ignoring the device check** — these are the small writable mounts (`/tmp`, `/dev/shm`, …) that are usually separate tmpfs yet are exactly where a hostile SUID binary is dropped. 3. Return the **sorted union**. Per-entry errors are skipped (fail-open). The walk is expensive, which is why the daemon gates its cadence (§1.5). --- ## 1.3 Detector registry and gating `detectors/__init__.py`. Detectors are pure `detect(state, cfg) -> Iterable [Alert]`. `REGISTRY` order **is** the order signatures appear in a snapshot and must be preserved by the port: ``` reverse_shell, ld_preload, deleted_exe, input_snooper, credential_access, stealth_network, memory_obfuscation, egress, first_seen*, new_listener*, persistence*, new_suid*† ``` `*` = `needs_baseline=True`; `†` `new_suid` is also `expensive=True`. `run_all(state, cfg, include=None)` yields, per registry entry, when: 1. `cfg.enabled(name)` is true, and 2. `include` is `None` or contains the name, and 3. if `needs_baseline`: **`state.listener_baseline is not None`**. Condition 3 is the arming gate: `listener_baseline is None` skips **all four** baseline-diff detectors at once — including `first_seen`, even though `first_seen` never reads `listener_baseline`. `first_seen`'s own gate is this shared flag, not its store. --- ## 1.4 The daemon sweep loop `daemon.py:run()` sequence: 1. `mkdir log_dir`; write `sentinel.pid`; append a start line to `events.log`. 2. **`build_baselines()`** — see §1.5.1. Note: `run()` *always builds*; it does not load. 3. `load_fim_baseline()`; `snapshot.prune()`; start eBPF exec + syscall monitors (each logs enabled/disabled to `events.log`; failure degrades to `None`, never fatal). 4. Loop until stopped, once per `sample_interval` (default `4.0`s, via `self._stop.wait(...)` so stop is prompt): - `write_heartbeat()` (dead-man's switch, every iteration). - **Only once `(now - start_time) >= baseline_grace`** (default `10`s): kick the cadence-gated background scanners — SUID, FIM, package verify, pkgdb anchor, pkgdb signature verify, rootcheck. Each runs off-thread and self-limits by its own interval; results feed back through `_on_exec_alert`. - `alerts = sweep()` (§1.5). - `fresh = fresh_alerts(alerts, now)` (§1.6). - If any fresh: capture a snapshot **off the loop thread** (a slow snapshot must never stall detection; the forensic `SystemState` is rebuilt fresh inside that thread). - Every 20 sweeps: `snapshot.prune()`. `stop()` sets the event and stops both eBPF monitors. Asynchronous producers (eBPF monitors, FIM/pkg/rootcheck threads) do **not** go through `sweep()`; they call `_on_exec_alert(alert, live_fps=None)`, which runs the same `fresh_alerts` dedup and then captures off-thread. So the cooldown and reconciliation chokepoint (§1.6) is shared by loop and async paths. --- ## 1.5 Baseline lifecycle ### 1.5.1 Listener and SUID baselines Held on the daemon as `set[str]`: `listener_baseline`, `suid_baseline`. - **`build_baselines()`** (called by `run()` at start, and by `cli check`): `listener_baseline = SystemState().listener_keys()` (a *fresh* live capture); `suid_baseline = set(scan_suid_binaries(extra_dirs=suid_scan_extra_dirs))`. Both are then written to disk: - `log_dir/listener-baseline.json` — JSON array, sorted. - `log_dir/suid-baseline.json` — JSON array, sorted. Consequence: **on daemon start the baseline is whatever is running / present at that moment.** Anything already listening or already SUID at start is, by definition, baseline and will not alert. The stored files are for the one-shot CLI path, not reloaded by the running daemon. - **`load_baselines()`** reads those two files (`[]` on any error). It is used **only by `cli check`** (`cli.py`), never by `run()`. ### 1.5.2 Arming (the grace window) Inside `sweep(force_suid=False)`: ``` armed = (now - start_time) >= baseline_grace # default 10s gate = armed or force_suid state = SystemState( listener_baseline = self.listener_baseline if gate else None, suid_baseline = self.suid_baseline, # always passed suid_binaries = suid_binaries if gate else None, persist_since = last_persist_scan if gate else None, ) alerts = list(detectors.run_all(state, cfg)) last_persist_scan = now # advance AFTER the sweep ``` So during the first `baseline_grace` seconds the baseline-diff detectors are inert (their inputs are `None`), suppressing the startup burst. `suid_baseline` is always populated but only consumed when `suid_binaries` is present. `cli check` passes `force_suid=True`: it bypasses the grace gate **and** computes `suid_binaries` synchronously via `scan_suid_binaries` (rather than reading the async result), so a single-shot check gets a complete SUID view. ### 1.5.3 SUID scan cadence and baseline refresh The live SUID list is produced off the loop thread: - `_maybe_scan_suid(now)`: skip if a scan thread is alive or `(now - last_suid_scan) < suid_scan_interval` (default `60`s); else record `last_suid_scan = now` and spawn `_scan_suid`. - `_scan_suid()`: `self._suid_current = scan_suid_binaries(...)`. Then, every `suid_refresh` (default `3600`s), fold the current scan back into the baseline (`suid_baseline = set(result)`, re-save `suid-baseline.json`) so legitimately installed SUID binaries stop alerting after a while. `sweep()` reads `self._suid_current`, which is `None` until the first async scan completes — so `new_suid` is a no-op for the first cycle or two after start even once armed. A port using synchronous-on-cadence scanning must reproduce the same observable rule: **no `new_suid` alerts until a scan result exists**, and periodic baseline refresh on the `suid_refresh` interval. ### 1.5.4 persist_since `last_persist_scan` starts at `start_time` and is set to `now` at the **end** of every `sweep()`. The `persistence` detector alerts on watched files whose `mtime > persist_since`, i.e. modified since the previous sweep. Passing the *previous* sweep's timestamp (advanced afterward) is the contract. ### 1.5.5 first_seen store `detectors/first_seen.py` owns `log_dir/first-seen.json`, independent of the listener/SUID baselines. Format (schema `enodia.first_seen.v1`): ```json { "schema": "enodia.first_seen.v1", "initialized": false, "public_destinations": { "": ["ip:port", ...] }, "listener_ports": { "": ["port", ...] } } ``` Behavior each time the detector runs (recall: only when armed): 1. Load the store (missing/corrupt → the default above). 2. For each `ESTAB` socket to a **public** peer (`netutil.is_public_ip`) with a port: if `comm`'s `public_destinations` set lacks `ip:port`, add it (list kept sorted); and **if `initialized`**, emit SID `100074` `first_public_destination` MEDIUM, key `firstdest::`, classtype `network-rarity`. 3. For each `LISTEN`/`UNCONN` socket of kind `tcp`/`udp`/`""` with a numeric non-zero port: if `comm`'s `listener_ports` set lacks it, add it (sorted numerically); and **if `initialized`**, emit SID `100075` `first_listener_port` MEDIUM, key `firstlisten::`, classtype `network-rarity`. 4. Set `initialized = true` and save (atomic: write `*.json.tmp`, `os.replace`, `chmod 0640` best-effort). Net effect: the **first armed run on a fresh store learns silently** (`initialized` was false) and thereafter alerts once per genuinely new `comm→destination` / `comm→port`. Because `initialized` persists to disk, a daemon **restart** does not re-enter the silent-learning phase — unlike the listener/SUID baselines, which are rebuilt every start. This asymmetry is deliberate and must be preserved. --- ## 1.6 Dedup, cooldown, and reconciliation chokepoint `fresh_alerts(alerts, now, live_fps=None)` is the single gate every alert passes (loop and async): 1. **Reconciliation first** — if any alerts, load `ReconcileStore` and `filter_alerts(alerts, live_fps or {})`, dropping operator-acknowledged drift so an acked item never even consumes a cooldown slot. `live_fps` carries current fingerprints for content kinds (e.g. `{"fim": {...}}`); identity kinds need none. (Detailed in §8.) 2. **Cooldown** — under `_cooldown_lock`, for each surviving alert: emit it and stamp `cooldowns[key] = now` only if `(now - cooldowns.get(key, 0)) >= cooldown` (default `60`s). Same `key` inside the window is suppressed. Order matters: reconcile before cooldown. --- ## 1.7 Timing knobs (config.py defaults) | Key | Default | Governs | |-----|---------|---------| | `sample_interval` | `4.0` s | loop period | | `cooldown` | `60` s | per-key alert dedup window | | `baseline_grace` | `10` s | arming delay before baseline-diff detectors + background scans | | `suid_scan_interval` | `60` s | SUID rescan cadence | | `suid_refresh` | `3600` s | fold current SUID scan into baseline | | `suid_scan_extra_dirs` | writable mounts tuple | extra (off-device) SUID walk roots | | `first_seen_enabled` | `true` | first_seen on/off | | `log_dir` | `/var/log/enodia-sentinel` | store + evidence root; `ENODIA_LOG_DIR` override | Store paths derive from `log_dir`: `listener-baseline.json`, `suid-baseline.json`, `first-seen.json`, `fim-baseline.json`, `events.log`, `sentinel.pid`. --- ## 1.8 Port parity notes (Go 2.0) What the Go agent must reproduce to be behavior-equivalent for this subsystem: - **Registry order and gating** (§1.3), including the shared arming gate that silences `first_seen` while unarmed. - **Baseline provenance** (§1.5.1): baseline captured *at start*, not loaded, in the daemon path; loaded in the one-shot path. - **Time-based arming** (§1.5.2), not first-sweep arming. - **SUID cadence + refresh + "no alert until a scan exists"** (§1.5.3) — the Go design uses synchronous-on-cadence scanning, which is allowed as long as the observable rules hold. - **`persist_since` = previous sweep time**, advanced after the sweep (§1.5.4). - **`first-seen.json` format, learn-once semantics, and restart asymmetry** (§1.5.5); the file format is byte-compatible so either agent can read a store the other wrote. - **Reconcile-before-cooldown** ordering and 60s per-key cooldown (§1.6). Current Go status (as of this writing): poll detectors are ported and pass the parity fixture, but baselines are still **injected from fixtures** — the store package, live SUID walk, and the daemon-equivalent lifecycle above are the work this document specifies.