docs(behavior): add behavioral spec for sweep + baseline lifecycle

Start a per-subsystem behavioral specification under docs/behavior/ that
documents the current Python agent's observable behavior at reimplementation
fidelity. It is the parity contract for the Go 2.0 rewrite (Active Migration
Track), complementing the product-oriented docs/SPECIFICATION.md.

Section 1 covers system-state capture, the daemon sweep loop, alert dedup, the
detector registry gating, and the baseline lifecycle — including three
behaviors the port must match exactly: baselines are rebuilt (not loaded) at
daemon start, arming is time-based via baseline_grace, and first_seen's
initialized flag persists across restarts while listener/SUID baselines
re-learn every start.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D36ffwoga57opftxzJVHPW

docs/behavior/01-sweep-and-state.md

@@ -0,0 +1,355 @@
+<!-- SPDX-License-Identifier: GPL-3.0-or-later -->
+# 1. Sweep, System State, and Baseline Lifecycle
+
+Scope: how the daemon gathers one view of the host, runs poll detectors over
+it, and manages the baselines that the baseline-diff detectors
+(`first_seen`, `new_listener`, `new_suid`, `persistence`) depend on. This is
+the substrate every other detection subsystem runs on top of.
+
+Source: `system.py`, `daemon.py`, `alert.py`, `detectors/__init__.py`,
+`config.py`.
+
+---
+
+## 1.1 The alert value
+
+`alert.py`. Every detection is an immutable `Alert`:
+
+| Field | Type | Meaning |
+|-------|------|---------|
+| `severity` | `Severity` enum | `MEDIUM=1`, `HIGH=2`, `CRITICAL=3`; `str()` yields the name |
+| `signature` | str | human rule name, e.g. `new_listener` |
+| `key` | str | **stable dedup identity**, e.g. `lis:31337/nc`; drives cooldown |
+| `detail` | str | one-line human description |
+| `pids` | tuple[int, …] | processes the snapshot should deep-dive (default empty) |
+| `sid` | int | stable Snort-style signature id (default `0`) |
+| `classtype` | str | category (default `uncategorized`) |
+
+`Alert.to_dict()` is the serialization contract — key order:
+`sid, severity, signature, classtype, key, detail, pids` (with `severity` as
+the string name and `pids` as a list). A port must preserve `key` values
+exactly: cooldown dedup is keyed on them, so a differing key changes
+re-fire behavior.
+
+---
+
+## 1.2 System state capture
+
+`SystemState` (`system.py`) is **one sweep's cached view**, built once and
+shared by all detectors. It is fully injectable: constructing it with
+`processes=`/`sockets=` bypasses live collection (how tests and the parity
+harness avoid `/proc` and `ss`).
+
+### 1.2.1 Processes
+
+`SystemState.processes` enumerates `/proc`, keeping entries whose name
+`isdigit()`, each wrapped in a `Process(pid)`. Every `Process` field is a
+`cached_property` read lazily from `/proc/<pid>/…` and **fail-open** (any
+`OSError` → empty value), because a process can exit mid-sweep:
+
+| Field | Source | Notes |
+|-------|--------|-------|
+| `comm` | `comm` | stripped |
+| `cmdline` | `cmdline` | NUL→space, stripped |
+| `exe` | readlink `exe` | `""` if unreadable |
+| `cwd` | readlink `cwd` | |
+| `environ` | `environ` | NUL-split `k=v` → dict |
+| `fd_targets` | `fd/` | dict `fd(str) → link target`, fds sorted numerically |
+| `memory_maps` | `maps` | see parser below |
+| `status` | `status` | `key → value` (value stripped) |
+| `ppid` | `status[PPid]` | `0` on parse failure |
+| `uid` | `status[Uid]` field 0 | `0` if absent |
+
+`stdio_socket_inode()` returns the socket inode if fd 0, 1, or 2 links to
+`socket:[…]` — the reverse-shell tell — else `None`.
+
+`parse_memory_maps(text)` splits each line `maxsplit=5`; needs ≥5 fields;
+address `start-end` parsed as hex (skip line if no `-` or non-hex); `path` is
+field 5 when present else `""`. Emits `MemoryMap(start, end, perms, path)`.
+**The port must preserve `maxsplit=5` semantics** so pathnames containing
+spaces stay intact.
+
+### 1.2.2 Sockets
+
+`SystemState.sockets` runs `ss -H` **once per protocol family**, in this order,
+tagging each result with `kind`:
+
+```
+-tanep tcp   -uanep udp   -wanep raw   -Sanep sctp   -danep dccp
+-0anep packet   -Manep mptcp   --tipc -anep tipc   --xdp -anep xdp
+--vsock -anep vsock
+```
+
+Each `ss` call is fail-open (`OSError`/`SubprocessError`/10s timeout → `""`),
+so an unsupported family drops only its own rows. `_parse_ss` per line:
+
+- Split on whitespace; need ≥5 fields.
+- If `field[0].lower()` is a known netid (`tcp udp raw sctp dccp mptcp p_raw
+  p_dgr packet tipc xdp vsock`) **and** ≥6 fields: `state, local, peer =
+  fields[1], fields[4], fields[5]`; otherwise `fields[0], fields[3], fields[4]`.
+  (Handles both `ss` layouts — with and without a leading netid column.)
+- `inode` from regex `\bino:(\d+)` or `None`.
+- `comm`, `pid` from regex `users:\(\("([^"]+)",pid=(\d+),fd=\d+` or `""`/`None`.
+
+A `None` pid is **meaningful** and must round-trip: it serializes as `?` in
+prose and as a distinct key component. `Socket` fields: `state, local, peer,
+inode, comm, pid, kind`.
+
+Derived views: `net_peer_by_inode` (`inode → "local -> peer"`, network sockets
+only), `listening_sockets()` (`state == "LISTEN"`), `established_sockets()`
+(`state == "ESTAB"`).
+
+`listener_keys()` maps current listeners to identity keys `port/comm` where
+`port = local.rpartition(":")[2]` and `comm` defaults to `?`. This is the
+identity used by `new_listener` and by the listener baseline.
+
+### 1.2.3 SUID scan (expensive, gated)
+
+`scan_suid_binaries(root="/", extra_dirs=())` (`system.py`):
+
+1. Walk `root` **staying on its device** (the `find -xdev` equivalent) via an
+   explicit `os.scandir` stack; collect regular files whose mode has
+   `S_ISUID | S_ISGID`.
+2. **Also** walk each `extra_dirs` entry that is a directory, **ignoring the
+   device check** — these are the small writable mounts (`/tmp`, `/dev/shm`, …)
+   that are usually separate tmpfs yet are exactly where a hostile SUID binary
+   is dropped.
+3. Return the **sorted union**.
+
+Per-entry errors are skipped (fail-open). The walk is expensive, which is why
+the daemon gates its cadence (§1.5).
+
+---
+
+## 1.3 Detector registry and gating
+
+`detectors/__init__.py`. Detectors are pure `detect(state, cfg) -> Iterable
+[Alert]`. `REGISTRY` order **is** the order signatures appear in a snapshot and
+must be preserved by the port:
+
+```
+reverse_shell, ld_preload, deleted_exe, input_snooper, credential_access,
+stealth_network, memory_obfuscation, egress,
+first_seen*, new_listener*, persistence*, new_suid*†
+```
+
+`*` = `needs_baseline=True`; `†` `new_suid` is also `expensive=True`.
+
+`run_all(state, cfg, include=None)` yields, per registry entry, when:
+
+1. `cfg.enabled(name)` is true, and
+2. `include` is `None` or contains the name, and
+3. if `needs_baseline`: **`state.listener_baseline is not None`**.
+
+Condition 3 is the arming gate: `listener_baseline is None` skips **all four**
+baseline-diff detectors at once — including `first_seen`, even though
+`first_seen` never reads `listener_baseline`. `first_seen`'s own gate is this
+shared flag, not its store.
+
+---
+
+## 1.4 The daemon sweep loop
+
+`daemon.py:run()` sequence:
+
+1. `mkdir log_dir`; write `sentinel.pid`; append a start line to `events.log`.
+2. **`build_baselines()`** — see §1.5.1. Note: `run()` *always builds*; it does
+   not load.
+3. `load_fim_baseline()`; `snapshot.prune()`; start eBPF exec + syscall
+   monitors (each logs enabled/disabled to `events.log`; failure degrades to
+   `None`, never fatal).
+4. Loop until stopped, once per `sample_interval` (default `4.0`s, via
+   `self._stop.wait(...)` so stop is prompt):
+   - `write_heartbeat()` (dead-man's switch, every iteration).
+   - **Only once `(now - start_time) >= baseline_grace`** (default `10`s): kick
+     the cadence-gated background scanners — SUID, FIM, package verify, pkgdb
+     anchor, pkgdb signature verify, rootcheck. Each runs off-thread and
+     self-limits by its own interval; results feed back through
+     `_on_exec_alert`.
+   - `alerts = sweep()` (§1.5).
+   - `fresh = fresh_alerts(alerts, now)` (§1.6).
+   - If any fresh: capture a snapshot **off the loop thread** (a slow snapshot
+     must never stall detection; the forensic `SystemState` is rebuilt fresh
+     inside that thread).
+   - Every 20 sweeps: `snapshot.prune()`.
+
+`stop()` sets the event and stops both eBPF monitors.
+
+Asynchronous producers (eBPF monitors, FIM/pkg/rootcheck threads) do **not**
+go through `sweep()`; they call `_on_exec_alert(alert, live_fps=None)`, which
+runs the same `fresh_alerts` dedup and then captures off-thread. So the
+cooldown and reconciliation chokepoint (§1.6) is shared by loop and async
+paths.
+
+---
+
+## 1.5 Baseline lifecycle
+
+### 1.5.1 Listener and SUID baselines
+
+Held on the daemon as `set[str]`: `listener_baseline`, `suid_baseline`.
+
+- **`build_baselines()`** (called by `run()` at start, and by `cli check`):
+  `listener_baseline = SystemState().listener_keys()` (a *fresh* live capture);
+  `suid_baseline = set(scan_suid_binaries(extra_dirs=suid_scan_extra_dirs))`.
+  Both are then written to disk:
+  - `log_dir/listener-baseline.json` — JSON array, sorted.
+  - `log_dir/suid-baseline.json` — JSON array, sorted.
+
+  Consequence: **on daemon start the baseline is whatever is running / present
+  at that moment.** Anything already listening or already SUID at start is, by
+  definition, baseline and will not alert. The stored files are for the
+  one-shot CLI path, not reloaded by the running daemon.
+
+- **`load_baselines()`** reads those two files (`[]` on any error). It is used
+  **only by `cli check`** (`cli.py`), never by `run()`.
+
+### 1.5.2 Arming (the grace window)
+
+Inside `sweep(force_suid=False)`:
+
+```
+armed = (now - start_time) >= baseline_grace         # default 10s
+gate  = armed or force_suid
+state = SystemState(
+    listener_baseline = self.listener_baseline if gate else None,
+    suid_baseline     = self.suid_baseline,          # always passed
+    suid_binaries     = suid_binaries if gate else None,
+    persist_since     = last_persist_scan if gate else None,
+)
+alerts = list(detectors.run_all(state, cfg))
+last_persist_scan = now                              # advance AFTER the sweep
+```
+
+So during the first `baseline_grace` seconds the baseline-diff detectors are
+inert (their inputs are `None`), suppressing the startup burst. `suid_baseline`
+is always populated but only consumed when `suid_binaries` is present.
+
+`cli check` passes `force_suid=True`: it bypasses the grace gate **and**
+computes `suid_binaries` synchronously via `scan_suid_binaries` (rather than
+reading the async result), so a single-shot check gets a complete SUID view.
+
+### 1.5.3 SUID scan cadence and baseline refresh
+
+The live SUID list is produced off the loop thread:
+
+- `_maybe_scan_suid(now)`: skip if a scan thread is alive or
+  `(now - last_suid_scan) < suid_scan_interval` (default `60`s); else record
+  `last_suid_scan = now` and spawn `_scan_suid`.
+- `_scan_suid()`: `self._suid_current = scan_suid_binaries(...)`. Then, every
+  `suid_refresh` (default `3600`s), fold the current scan back into the
+  baseline (`suid_baseline = set(result)`, re-save
+  `suid-baseline.json`) so legitimately installed SUID binaries stop alerting
+  after a while.
+
+`sweep()` reads `self._suid_current`, which is `None` until the first async
+scan completes — so `new_suid` is a no-op for the first cycle or two after
+start even once armed. A port using synchronous-on-cadence scanning must
+reproduce the same observable rule: **no `new_suid` alerts until a scan result
+exists**, and periodic baseline refresh on the `suid_refresh` interval.
+
+### 1.5.4 persist_since
+
+`last_persist_scan` starts at `start_time` and is set to `now` at the **end**
+of every `sweep()`. The `persistence` detector alerts on watched files whose
+`mtime > persist_since`, i.e. modified since the previous sweep. Passing the
+*previous* sweep's timestamp (advanced afterward) is the contract.
+
+### 1.5.5 first_seen store
+
+`detectors/first_seen.py` owns `log_dir/first-seen.json`, independent of the
+listener/SUID baselines. Format (schema `enodia.first_seen.v1`):
+
+```json
+{
+  "schema": "enodia.first_seen.v1",
+  "initialized": false,
+  "public_destinations": { "<comm>": ["ip:port", ...] },
+  "listener_ports":      { "<comm>": ["port", ...] }
+}
+```
+
+Behavior each time the detector runs (recall: only when armed):
+
+1. Load the store (missing/corrupt → the default above).
+2. For each `ESTAB` socket to a **public** peer (`netutil.is_public_ip`) with a
+   port: if `comm`'s `public_destinations` set lacks `ip:port`, add it (list
+   kept sorted); and **if `initialized`**, emit SID `100074`
+   `first_public_destination` MEDIUM, key `firstdest:<comm>:<ip:port>`,
+   classtype `network-rarity`.
+3. For each `LISTEN`/`UNCONN` socket of kind `tcp`/`udp`/`""` with a numeric
+   non-zero port: if `comm`'s `listener_ports` set lacks it, add it (sorted
+   numerically); and **if `initialized`**, emit SID `100075`
+   `first_listener_port` MEDIUM, key `firstlisten:<comm>:<port>`, classtype
+   `network-rarity`.
+4. Set `initialized = true` and save (atomic: write `*.json.tmp`, `os.replace`,
+   `chmod 0640` best-effort).
+
+Net effect: the **first armed run on a fresh store learns silently**
+(`initialized` was false) and thereafter alerts once per genuinely new
+`comm→destination` / `comm→port`. Because `initialized` persists to disk, a
+daemon **restart** does not re-enter the silent-learning phase — unlike the
+listener/SUID baselines, which are rebuilt every start. This asymmetry is
+deliberate and must be preserved.
+
+---
+
+## 1.6 Dedup, cooldown, and reconciliation chokepoint
+
+`fresh_alerts(alerts, now, live_fps=None)` is the single gate every alert
+passes (loop and async):
+
+1. **Reconciliation first** — if any alerts, load `ReconcileStore` and
+   `filter_alerts(alerts, live_fps or {})`, dropping operator-acknowledged
+   drift so an acked item never even consumes a cooldown slot. `live_fps`
+   carries current fingerprints for content kinds (e.g. `{"fim": {...}}`);
+   identity kinds need none. (Detailed in §8.)
+2. **Cooldown** — under `_cooldown_lock`, for each surviving alert: emit it and
+   stamp `cooldowns[key] = now` only if `(now - cooldowns.get(key, 0)) >=
+   cooldown` (default `60`s). Same `key` inside the window is suppressed.
+
+Order matters: reconcile before cooldown.
+
+---
+
+## 1.7 Timing knobs (config.py defaults)
+
+| Key | Default | Governs |
+|-----|---------|---------|
+| `sample_interval` | `4.0` s | loop period |
+| `cooldown` | `60` s | per-key alert dedup window |
+| `baseline_grace` | `10` s | arming delay before baseline-diff detectors + background scans |
+| `suid_scan_interval` | `60` s | SUID rescan cadence |
+| `suid_refresh` | `3600` s | fold current SUID scan into baseline |
+| `suid_scan_extra_dirs` | writable mounts tuple | extra (off-device) SUID walk roots |
+| `first_seen_enabled` | `true` | first_seen on/off |
+| `log_dir` | `/var/log/enodia-sentinel` | store + evidence root; `ENODIA_LOG_DIR` override |
+
+Store paths derive from `log_dir`: `listener-baseline.json`,
+`suid-baseline.json`, `first-seen.json`, `fim-baseline.json`, `events.log`,
+`sentinel.pid`.
+
+---
+
+## 1.8 Port parity notes (Go 2.0)
+
+What the Go agent must reproduce to be behavior-equivalent for this subsystem:
+
+- **Registry order and gating** (§1.3), including the shared arming gate that
+  silences `first_seen` while unarmed.
+- **Baseline provenance** (§1.5.1): baseline captured *at start*, not loaded, in
+  the daemon path; loaded in the one-shot path.
+- **Time-based arming** (§1.5.2), not first-sweep arming.
+- **SUID cadence + refresh + "no alert until a scan exists"** (§1.5.3) — the Go
+  design uses synchronous-on-cadence scanning, which is allowed as long as the
+  observable rules hold.
+- **`persist_since` = previous sweep time**, advanced after the sweep (§1.5.4).
+- **`first-seen.json` format, learn-once semantics, and restart asymmetry**
+  (§1.5.5); the file format is byte-compatible so either agent can read a store
+  the other wrote.
+- **Reconcile-before-cooldown** ordering and 60s per-key cooldown (§1.6).
+
+Current Go status (as of this writing): poll detectors are ported and pass the
+parity fixture, but baselines are still **injected from fixtures** — the store
+package, live SUID walk, and the daemon-equivalent lifecycle above are the
+work this document specifies.

docs/behavior/index.md

@@ -0,0 +1,51 @@
+<!-- SPDX-License-Identifier: GPL-3.0-or-later -->
+# Enodia Sentinel — Behavioral Specification
+
+This directory documents the **observable behavior of the current Python
+production agent** at reimplementation fidelity: the inputs each subsystem
+reads, the algorithm it applies, the outputs it produces (SIDs, severities,
+dedup keys, JSON schemas), the files it writes, and its edge-case / fail-open
+rules.
+
+## Why this exists
+
+Python is the production agent and the **parity oracle** for the Go rewrite
+that will ship as Enodia Sentinel **2.0** (see `docs/ROADMAP.md`, Active
+Migration Track). "Match the oracle" was previously tribal knowledge encoded
+only in the parity fixture and the source itself. These documents make the
+contract reviewable: a Go port of any subsystem is correct when it reproduces
+the behavior described here.
+
+This is deliberately distinct from `docs/SPECIFICATION.md`, which defines the
+*product* (what Sentinel is, design principles, acceptance criteria). This set
+defines *mechanism* (what the code does, precisely enough to re-derive).
+
+## Conventions
+
+- **Source anchors** are written as `module.py:function` — the authority is
+  always the code, and these documents must be updated when the code changes.
+- **SIDs / severities / keys** quoted here are the stable contract; a port that
+  emits a different value for the same condition is a parity break.
+- "Fail-open" means a collection or read error degrades to an empty/absent
+  value and never raises — detection continues on partial data.
+- Defaults quoted are the `config.py` dataclass defaults as of package version
+  `0.7.0`; operators may override them.
+
+## Sections
+
+Written in port-priority order. Status reflects which documents exist.
+
+| # | File | Covers | Status |
+|---|------|--------|--------|
+| 1 | [`01-sweep-and-state.md`](01-sweep-and-state.md) | `system.py` capture, `daemon.py` sweep loop + baseline lifecycle, `alert.py`, detector registry, `config.py` timing knobs | **written** |
+| 2 | `02-poll-detectors.md` | the twelve `detectors/*` — per-detector input → match → output contract | planned |
+| 3 | `03-event-detection.md` | `event.py`, exec/syscall/host rule engines, `ruleops.py`, `sids.py` | planned |
+| 4 | `04-integrity.md` | `fim.py`, `pkgdb.py`, `selfprotect.py`, `assurance.py` | planned |
+| 5 | `05-rootcheck.md` | `rootcheck.py` anti-rootkit cross-view checks | planned |
+| 6 | `06-posture.md` | `posture.py` advisory host-hygiene checks | planned |
+| 7 | `07-investigation.md` | `snapshot.py`, `incident.py`, `correlation.py`, `enrich.py`, `triage.py` | planned |
+| 8 | `08-response.md` | `respond.py` plans, `reconcile.py` acknowledged-drift store | planned |
+| 9 | `09-interfaces.md` | `cli.py`, `web.py`, `tui.py` operator-visible contracts | planned |
+
+Shared helpers (`netutil.py`, `provenance.py`) are documented inline in whatever
+section first depends on them.