Add baseline reconciliation design spec
Covers the v0.9 roadmap item: accept legitimate FIM/package/listener/SUID drift with a mandatory reason and audited fingerprint-bound ack store. Approach A selected: ReconcileStore with re-alert-on-fingerprint-change, optional TTL, and stale tracking. Filter applied at daemon chokepoint, CLI fim check, and dashboard API. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Luna
parent
69798b26e4
commit
3d2047fde2
1 changed files with 238 additions and 0 deletions
|
|
@ -0,0 +1,238 @@
|
|||
# Baseline Reconciliation Design
|
||||
|
||||
**Date:** 2026-06-19
|
||||
**Status:** Approved for implementation
|
||||
**Roadmap:** v0.9 — "Add baseline reconciliation: accept legitimate FIM/package/listener/SUID changes with a recorded reason."
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
The daemon re-alerts on every sweep for legitimate drift — a config file the operator edited, a new service started after boot, a SUID binary installed by a package manager. Today the only remedies are:
|
||||
|
||||
- Rebuild baselines (`enodia-sentinel baseline` / `fim-update`), which silently folds the change in with no record of why.
|
||||
- Add permanent config allowlists (`listener_allow_ports`, etc.), which carry no per-item reason or audit trail.
|
||||
|
||||
Neither gives re-alert-on-change semantics: once drift is absorbed it stays absorbed, even if the file is edited a second time.
|
||||
|
||||
---
|
||||
|
||||
## Goals
|
||||
|
||||
1. Let operators accept specific, identified drift items with a mandatory reason.
|
||||
2. Suppress alerts for those items only while the live state still matches the accepted fingerprint.
|
||||
3. Re-alert and mark the acknowledgement stale when the item changes again or its TTL expires.
|
||||
4. Keep a full audit trail (who, when, why, what fingerprint) in a single queryable store.
|
||||
5. Leave existing baseline files and detector code untouched.
|
||||
|
||||
---
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Auto-accepting drift without operator input.
|
||||
- Modifying config files on behalf of the operator (that belongs to the FP-suppression bullet).
|
||||
- Covering eBPF-only signatures that have no baseline concept (reverse shell, memory obfuscation, etc.).
|
||||
|
||||
---
|
||||
|
||||
## Data Model
|
||||
|
||||
**Store file:** `log_dir/reconciliation.json` (overridable via `reconcile_store` config key).
|
||||
Written atomically: `reconciliation.json.tmp` → rename. Loaded with a file-mtime cache — the daemon never re-reads unless the file changed.
|
||||
|
||||
### Record schema
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": "fim",
|
||||
"target": "/etc/hosts",
|
||||
"fingerprint": {"sha256": "abc123", "mode": 33188, "uid": 0, "gid": 0},
|
||||
"reason": "added internal host entry",
|
||||
"actor": "luna",
|
||||
"accepted_at": "2026-06-19T14:00:00+00:00",
|
||||
"expires_at": "2026-06-26T14:00:00+00:00",
|
||||
"status": "ok"
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Notes |
|
||||
|---|---|---|
|
||||
| `kind` | string | `fim`, `pkgfile`, `listener`, `suid` |
|
||||
| `target` | string | Path or `port/comm` key |
|
||||
| `fingerprint` | object | Kind-specific; see below |
|
||||
| `reason` | string | Mandatory, operator-supplied |
|
||||
| `actor` | string | `os.getlogin()` / `$USER` / `"unknown"` |
|
||||
| `accepted_at` | ISO-8601 UTC | Set at accept time |
|
||||
| `expires_at` | ISO-8601 UTC or absent | Absent = no TTL |
|
||||
| `status` | string | `"ok"` or `"stale"` |
|
||||
|
||||
### Fingerprints by kind
|
||||
|
||||
| Kind | Fingerprint fields | Ack goes stale when… |
|
||||
|---|---|---|
|
||||
| `fim` | `{sha256, mode, uid, gid}` | Any field differs from the live FIM scan entry |
|
||||
| `pkgfile` | `{reasons: ["checksum mismatch", ...]}` | The set of pacman-reported flags for that path changes |
|
||||
| `listener` | `{key: "8080/nginx"}` | TTL expires or operator revokes (a different port/comm is a distinct ack, not a stale one) |
|
||||
| `suid` | `{path: "/usr/bin/sudo"}` | TTL expires or operator revokes |
|
||||
|
||||
For `fim` and `pkgfile`, content changes trigger stale automatically. For `listener` and `suid`, the fingerprint is the identity key itself — a different port/comm or path produces a new unrelated alert rather than staleing an existing ack. TTL and explicit revoke are the only staleing mechanisms for those two kinds.
|
||||
|
||||
### Stale transitions
|
||||
|
||||
The filter transitions `ok → stale` in-memory when:
|
||||
- Fingerprint check fails (live state diverged).
|
||||
- Current time is past `expires_at`.
|
||||
|
||||
The stale flag is written back to the store lazily — on the next `baseline list`, `accept`, or `revoke` call — to avoid disk writes on every daemon sweep.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### New module: `enodia_sentinel/reconcile.py`
|
||||
|
||||
Owns all store I/O, fingerprint building, and filter logic. No other module is aware of ack semantics.
|
||||
|
||||
```
|
||||
ReconcileStore
|
||||
.load(cfg) -> ReconcileStore # mtime-cached
|
||||
.accept(kind, target, fp, reason, actor, expires_at)
|
||||
.revoke(kind, target)
|
||||
.list(stale_only=False) -> list[Record]
|
||||
.filter_alerts(alerts, live_fps) -> list[Alert]
|
||||
.flush_stale() # write pending stale transitions
|
||||
|
||||
build_fingerprint(kind, target, live_data) -> dict
|
||||
```
|
||||
|
||||
`filter_alerts` is a method on `ReconcileStore`; called as `store.filter_alerts(alerts, live_fps)`. It is pure given the already-loaded store and a live fingerprint dict — easy to unit test without filesystem access.
|
||||
|
||||
### Integration points (three, minimal)
|
||||
|
||||
**1. Daemon sweep and eBPF path — `daemon.py:fresh_alerts()`**
|
||||
|
||||
```python
|
||||
store = ReconcileStore.load(self.cfg)
|
||||
live_fps = _build_live_fps(state) # pulls from already-computed state
|
||||
alerts = store.filter_alerts(alerts, live_fps)
|
||||
```
|
||||
|
||||
`_build_live_fps` is a small helper that assembles the fingerprint dict from `state.fim_baseline`, `state.listener_baseline`, `state.suid_baseline`, and the last pkgfile verify results already held on `self`.
|
||||
|
||||
**2. CLI read views — `cli.py`**
|
||||
|
||||
`fim check` wraps its diff alerts through `filter_alerts` before printing. The three new `baseline` subcommands call `ReconcileStore` methods directly.
|
||||
|
||||
**3. Dashboard APIs — `web.py`**
|
||||
|
||||
`/api/integrity` and alert lists wrap outbound alert lists through `filter_alerts`. No new endpoints needed; acknowledged items simply don't appear (or appear annotated — see `baseline list`).
|
||||
|
||||
### Config
|
||||
|
||||
```python
|
||||
reconcile_store: str = "" # empty = log_dir/reconciliation.json
|
||||
```
|
||||
|
||||
Added to `config.py` alongside other `log_dir`-derived paths. Also added to `config/enodia-sentinel.toml` in the paths/logging block (not the posture block) with a comment.
|
||||
|
||||
### File layout
|
||||
|
||||
```
|
||||
enodia_sentinel/reconcile.py new
|
||||
tests/test_reconcile.py new
|
||||
enodia_sentinel/config.py +1 field
|
||||
enodia_sentinel/daemon.py filter wired into fresh_alerts()
|
||||
enodia_sentinel/cli.py baseline accept/revoke/list subcommands
|
||||
enodia_sentinel/web.py filter on /api/integrity alert lists
|
||||
docs/COMMAND_REFERENCE.md new baseline accept/revoke/list section
|
||||
docs/ROADMAP.md mark v0.9 reconciliation bullet ✅
|
||||
config/enodia-sentinel.toml +reconcile_store knob
|
||||
```
|
||||
|
||||
No changes to `fim.py`, detector modules, or existing baseline JSON files.
|
||||
|
||||
---
|
||||
|
||||
## CLI
|
||||
|
||||
### `baseline accept`
|
||||
|
||||
```
|
||||
enodia-sentinel baseline accept <kind> <target> --reason "..." [--expires <duration>]
|
||||
```
|
||||
|
||||
- `<kind>`: `fim` | `pkgfile` | `listener` | `suid`
|
||||
- `<target>`: path or `port/comm` key
|
||||
- `--reason`: mandatory; hard error to omit
|
||||
- `--expires`: optional; accepts `Nd`, `Nh`, `Nm` (days/hours/minutes)
|
||||
|
||||
**Guards:**
|
||||
- If the target has no live data (path not in current FIM scan, listener not currently seen, etc.), prints a warning and requires `--force` to proceed. Prevents accepting something that can't be fingerprinted.
|
||||
- If an `ok` ack already exists for this target, updates it (new fingerprint, reason, timestamp) and prints a notice.
|
||||
|
||||
### `baseline revoke`
|
||||
|
||||
```
|
||||
enodia-sentinel baseline revoke <kind> <target>
|
||||
```
|
||||
|
||||
Removes the record from the store. Prints "not found" and exits `1` if the target isn't present.
|
||||
|
||||
### `baseline list`
|
||||
|
||||
```
|
||||
enodia-sentinel baseline list [--stale] [--json]
|
||||
```
|
||||
|
||||
Text output:
|
||||
|
||||
```
|
||||
KIND TARGET STATUS ACCEPTED EXPIRES REASON
|
||||
fim /etc/hosts ok 2026-06-19 14:00 2026-06-26 added internal host
|
||||
listener 8080/nginx ok 2026-06-19 15:00 - nginx added
|
||||
fim /etc/cron.d/backup stale 2026-06-18 10:00 - scheduled job
|
||||
```
|
||||
|
||||
`--stale` filters to stale/expired records only. Exit code `1` if any stale acks exist (scriptable check).
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Scenario | Behaviour |
|
||||
|---|---|
|
||||
| Store file missing | Treated as empty store; no acks applied |
|
||||
| Store file corrupt (bad JSON) | Fails closed: no acks applied, daemon logs warning, continues alerting normally |
|
||||
| Atomic write failure | Old file untouched; error reported to operator |
|
||||
| `accept` with no live fingerprint data | Warning printed; requires `--force` |
|
||||
| `revoke` on missing target | Prints "not found", exits `1` |
|
||||
| `expires` parse error | Hard error with usage hint |
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
**`tests/test_reconcile.py`** (new):
|
||||
|
||||
- `build_fingerprint` for each kind — correct dict shape and field names
|
||||
- `filter_alerts`: ack matches live fingerprint → alert suppressed
|
||||
- `filter_alerts`: fingerprint mismatch → alert passes through, ack marked stale in-memory
|
||||
- `filter_alerts`: expired TTL → alert passes through
|
||||
- `ReconcileStore.load` on missing file → empty store, no exception
|
||||
- `ReconcileStore.load` on corrupt JSON → empty store, no exception
|
||||
- `baseline accept` CLI round-trip: accept a FIM path → store written → `filter_alerts` suppresses it
|
||||
- `baseline list --stale` exits `1` when stale acks present, `0` when none
|
||||
- `baseline revoke` removes record; second revoke exits `1`
|
||||
|
||||
**Existing test updates:**
|
||||
|
||||
- `tests/test_daemon.py` (or integration): `fresh_alerts` with a matching ack → alert dropped; with mismatched fingerprint → alert passes
|
||||
- `tests/test_fim.py`: `fim check` CLI output suppresses acked paths
|
||||
|
||||
---
|
||||
|
||||
## Docs to update
|
||||
|
||||
- `docs/COMMAND_REFERENCE.md` — new `baseline accept/revoke/list` section
|
||||
- `docs/ROADMAP.md` — mark v0.9 reconciliation bullet ✅
|
||||
- `config/enodia-sentinel.toml` — `reconcile_store` knob with comment
|
||||
Loading…
Add table
Add a link
Reference in a new issue