Re-architect Sentinel as a zero-dependency Python package
Port the bash prototype to a structured, testable Python codebase while
preserving the same control loop and the seven detection signatures. The bash
implementation stays in src/ as the regression oracle.
Highlights:
- enodia_sentinel/ package (stdlib only — no runtime deps):
- detectors/ : one pure function per signature, detect(state, cfg)->Alerts
- system.py : SystemState — one cached /proc + ss snapshot per sweep,
fully injectable so detectors are unit-testable
- daemon.py : sweep loop, in-process cooldown dedup, threaded snapshot
capture, and a SUID filesystem scan moved OFF the loop
thread onto a slow background cadence
- snapshot.py: forensic text + JSON sidecar with per-signature IR guidance
- config.py : dataclass config via TOML + env overrides
- netutil.py : public-IP / CIDR logic via stdlib ipaddress
- tests/ : 25 stdlib-unittest cases (no root, no /proc, no ss needed)
- TOML config, launcher wrapper, Makefile (pip-free install), hardened
systemd unit (env-resolved ExecStart), updated PKGBUILD, rewritten README
Performance: per-sweep cost ~200 ms (shared cached state); the multi-second
SUID walk no longer blocks detection. scandir-based walk replaces os.walk.
Verified on Arch: all 7 detectors fire on red-team drills (reverse_shell,
ld_preload, deleted_exe, new_listener, new_suid confirmed live end-to-end),
no false positives on a clean sweep, 25/25 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Luna
parent
45f8acb24a
commit
28d67a1360
28 changed files with 1783 additions and 133 deletions
224
README.md
224
README.md
|
|
@ -3,62 +3,72 @@
|
|||
A host intrusion-detection daemon for Linux. It continuously runs a set of
|
||||
detectors over live system state — processes, sockets, file descriptors, the
|
||||
SUID inventory, and sensitive files — and writes a detailed forensic snapshot
|
||||
with incident-response guidance the moment a known attack signature appears.
|
||||
(text **and** JSON) with incident-response guidance the moment a known attack
|
||||
signature appears.
|
||||
|
||||
Think of it as the security counterpart to a performance watchdog: instead of
|
||||
"I/O pressure spiked, here's the kernel state," it's *"a shell just wired itself
|
||||
to a socket — here's the process tree, the peer, and what to do about it."*
|
||||
|
||||
> **Design lineage.** Sentinel reuses the proven control loop of a performance
|
||||
> anomaly-capture daemon (sample → threshold/trigger → rich snapshot → classify
|
||||
> → notify/retain) and swaps the inputs and signatures from *performance* to
|
||||
> *security*. The forensic-snapshot-on-trigger pattern is the same; the
|
||||
> detectors and the response guidance are new.
|
||||
> **Two implementations, on purpose.** The project began as a bash prototype
|
||||
> (`src/sentinel.sh`, kept as the regression **oracle**) and was re-architected
|
||||
> into a zero-dependency Python package with a unit-test suite, structured
|
||||
> detectors, and JSON output. The bash version and the Python version share one
|
||||
> red-team harness, so every signature is exercised against both.
|
||||
|
||||
## Why these detectors
|
||||
|
||||
Every detector keys on a behavior that is **cheap to observe** and **expensive
|
||||
for an attacker to avoid** — the high-signal, low-false-positive heuristics that
|
||||
real EDRs are built on:
|
||||
for an attacker to avoid** — the high-signal, low-false-positive heuristics real
|
||||
EDRs are built on:
|
||||
|
||||
| Signature | What it catches | Why it's hard to evade |
|
||||
|---|---|---|
|
||||
| `reverse_shell` | A shell/interpreter with a **socket on fd 0/1/2** | Interactive shells get a pty, not a socket — a socket on stdio is almost always `nc -e` / `bash -i >& /dev/tcp/...` |
|
||||
| `ld_preload_global` | A non-empty `/etc/ld.so.preload` | Injecting into *every* process is the whole point of a userland rootkit; the file has to exist |
|
||||
| `ld_preload_proc` | `LD_PRELOAD` pointing into `/tmp`, `/dev/shm`, … | Hooking libc to hide files/creds needs the library somewhere writable |
|
||||
| `deleted_exe` | A process running from a **deleted / `memfd:`** binary | Fileless malware deletes its dropper but the kernel still names the inode `(deleted)` |
|
||||
| `new_listener` | A listening port absent from the startup baseline | Bind shells and backdoors have to listen somewhere |
|
||||
| `new_suid` | A new SUID/SGID binary (critical in writable dirs) | A SUID `/tmp` binary is a textbook privesc persistence trick |
|
||||
| `persistence` | Changes to cron, systemd units, `authorized_keys`, rc files | Persistence has to write *somewhere* that survives reboot |
|
||||
| `suspicious_egress` | An interpreter holding an outbound conn to a public IP | C2 beacons and exfil need to phone home |
|
||||
| `reverse_shell` | An interpreter with a **network socket on fd 0/1/2** | Interactive shells get a pty and daemons get unix sockets — a *network* socket on stdio is `nc -e` / `bash -i >& /dev/tcp/...` |
|
||||
| `ld_preload` | Non-empty `/etc/ld.so.preload`, or `LD_PRELOAD` into a writable dir | Injecting into processes needs the library to exist somewhere |
|
||||
| `deleted_exe` | A process running from a **deleted / `memfd:`** binary | Fileless malware deletes its dropper; the kernel still names the inode `(deleted)` |
|
||||
| `new_listener` | A listening port absent from the startup baseline | Bind shells/backdoors have to listen somewhere |
|
||||
| `new_suid` | A new SUID/SGID binary (critical in a writable dir) | A SUID `/tmp` binary is a textbook privesc trick |
|
||||
| `persistence` | Changes to cron, systemd units, `authorized_keys`, rc files | Persistence has to write somewhere that survives reboot |
|
||||
| `egress` | An interpreter with an established connection to a public IP | C2 beacons and exfil have to phone home |
|
||||
|
||||
## The control loop
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────┐
|
||||
│ every SAMPLE_INTERVAL seconds │
|
||||
│ │
|
||||
│ run_detectors() ──► alert lines │
|
||||
│ │ SEV signature detail │
|
||||
│ ▼ │
|
||||
│ cooldown dedup (per signature+key) │
|
||||
│ │ │
|
||||
│ ▼ (fresh alerts only) │
|
||||
│ capture_snapshot() │
|
||||
│ • flagged-pid deep dive (/proc) │
|
||||
│ • full process tree + sockets │
|
||||
│ • ld.so.preload, recent file mods │
|
||||
│ • logins, failed auth, kernel modules │
|
||||
│ • per-signature response guidance │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ events.log + alert-*.log + desktop notify │
|
||||
└────────────────────────────────────────────┘
|
||||
enodia_sentinel/
|
||||
├── cli.py run / check / baseline / list-detectors
|
||||
├── daemon.py sweep loop · cooldown dedup · backgrounded SUID scan
|
||||
├── system.py SystemState — one cached snapshot of /proc + ss per sweep
|
||||
├── snapshot.py forensic text+JSON capture · response guidance · retention
|
||||
├── config.py dataclass config (TOML + env overrides)
|
||||
├── netutil.py public-IP / CIDR logic (stdlib ipaddress)
|
||||
├── alert.py Alert / Severity
|
||||
└── detectors/ one module per signature, each a pure function:
|
||||
detect(state, cfg) -> Iterable[Alert]
|
||||
```
|
||||
|
||||
Output lives in `/var/log/enodia-sentinel/`:
|
||||
- `events.log` — one line per alert (timestamp, severity, signatures, file)
|
||||
- `alert-YYYYMMDD-HHMMSS.log` — the full forensic snapshot
|
||||
The loop is deliberately the same control flow as the bash prototype, but the
|
||||
state lives in real objects:
|
||||
|
||||
```
|
||||
every sample_interval seconds:
|
||||
state = SystemState() # /proc + ss gathered once, cached
|
||||
alerts = run_all(detectors, state, cfg)
|
||||
fresh = drop alerts still within cooldown
|
||||
if fresh:
|
||||
snapshot.capture(fresh) # on a worker thread
|
||||
```
|
||||
|
||||
Two design choices keep it fast and unobtrusive:
|
||||
|
||||
- **One `SystemState` per sweep.** Detectors read shared, cached `/proc`/`ss`
|
||||
data instead of each shelling out — a sweep costs ~200 ms regardless of how
|
||||
many detectors run.
|
||||
- **The filesystem-wide SUID scan runs off the loop thread** on a slow cadence,
|
||||
so the multi-second walk never stalls live detection.
|
||||
|
||||
Everything in `SystemState` is **injectable**, which is what makes the detectors
|
||||
unit-testable without root or a live system (see `tests/`).
|
||||
|
||||
## Quick start
|
||||
|
||||
|
|
@ -72,103 +82,111 @@ sudo tail -f /var/log/enodia-sentinel/events.log
|
|||
sentinel-redteam # safe, self-cleaning attack simulations
|
||||
```
|
||||
|
||||
You'll watch the drills trip `reverse_shell`, `ld_preload_proc`, `deleted_exe`,
|
||||
`new_listener`, and `new_suid` in real time, each producing a snapshot with
|
||||
response guidance.
|
||||
You'll watch the drills trip `reverse_shell`, `ld_preload`, `deleted_exe`,
|
||||
`new_listener`, and `new_suid` in real time, each producing a `.log` + `.json`
|
||||
snapshot with response guidance.
|
||||
|
||||
### Without installing (try it in place)
|
||||
Output lives in `/var/log/enodia-sentinel/`:
|
||||
- `events.log` — one line per alert
|
||||
- `alert-YYYYMMDD-HHMMSS.log` — human-readable forensic snapshot
|
||||
- `alert-YYYYMMDD-HHMMSS.json` — same data, structured (SIEM-ready)
|
||||
|
||||
### Without installing
|
||||
|
||||
```bash
|
||||
sudo ./src/sentinel.sh --baseline # establish listener/SUID baselines
|
||||
sudo ./src/sentinel.sh --check # run every detector once, print findings
|
||||
make test # run the unit suite
|
||||
python3 -m enodia_sentinel.cli baseline # establish baselines
|
||||
python3 -m enodia_sentinel.cli check # run every detector once, print findings
|
||||
```
|
||||
|
||||
No pip, no virtualenv, no dependencies — it's stdlib-only and installs as a
|
||||
plain package directory plus a launcher wrapper.
|
||||
|
||||
## The red-team harness
|
||||
|
||||
`sentinel-redteam` is the demo and the regression test in one. It simulates each
|
||||
threat with **safe, clearly-labeled stand-ins** (everything tagged
|
||||
`enodia-drill`, auto-cleaned on exit):
|
||||
|
||||
- **reverse_shell** — a `bash` whose stdio is a TCP socket to a *local* listener
|
||||
(no traffic leaves the box)
|
||||
- **ld_preload** — a process with `LD_PRELOAD=/tmp/...` (an empty file; never
|
||||
actually loaded, and it does **not** touch the system-wide `/etc/ld.so.preload`)
|
||||
- **deleted_exe** — a copy of `/bin/sleep` run from `/tmp`, then deleted
|
||||
- **new_listener** — an ephemeral `nc -l` on loopback
|
||||
- **new_suid** — a `chmod 4755` copy of `/bin/true` in `/tmp`
|
||||
- **persistence** — writes to a sandbox `authorized_keys` (only fires if you opt
|
||||
the sandbox into `WATCH_PERSISTENCE`; it refuses to touch your real dotfiles)
|
||||
`sentinel-redteam` is the demo and the integration test in one. It simulates
|
||||
each threat with **safe, clearly-labeled stand-ins** (everything tagged
|
||||
`enodia-drill`, auto-cleaned on exit), using a local Python TCP listener so no
|
||||
traffic ever leaves the host:
|
||||
|
||||
```bash
|
||||
sentinel-redteam --list # list drills
|
||||
sentinel-redteam reverse_shell new_suid # run specific ones
|
||||
HOLD=30 sentinel-redteam # keep artifacts alive 30s
|
||||
sentinel-redteam --list # list drills
|
||||
sentinel-redteam reverse_shell new_suid # run specific ones
|
||||
HOLD=30 sentinel-redteam # keep artifacts alive 30s
|
||||
```
|
||||
|
||||
It never touches your real dotfiles or `/etc/ld.so.preload`; the LD_PRELOAD
|
||||
drill only sets the env var on a throwaway process.
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
make test # 25 unit tests, stdlib unittest, no deps
|
||||
```
|
||||
|
||||
Detectors are pure functions over an injectable `SystemState`, so tests build
|
||||
fake processes/sockets and assert on the alerts — no root, no `/proc`, no `ss`:
|
||||
|
||||
```python
|
||||
proc = FakeProc(pid=100, comm="bash", _stdio_inode=999)
|
||||
sock = Socket("ESTAB", "127.0.0.1:55", "9.9.9.9:443", 999, "bash", 100)
|
||||
state = SystemState(processes=[proc], sockets=[sock])
|
||||
assert list(reverse_shell.detect(state, Config()))[0].signature == "reverse_shell"
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Edit `/etc/enodia-sentinel.conf`, then `sudo systemctl restart
|
||||
enodia-sentinel.service`. Key options:
|
||||
Edit `/etc/enodia-sentinel.toml`, then `sudo systemctl restart
|
||||
enodia-sentinel.service`. Every key is optional. Highlights:
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
| Key | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `SAMPLE_INTERVAL` | 4 | seconds between detector sweeps |
|
||||
| `COOLDOWN` | 60 | min seconds before re-alerting the same signature |
|
||||
| `DET_*` | 1 | per-detector on/off switches |
|
||||
| `INTERPRETERS` | bash sh … | process names treated as shells for rsh/egress |
|
||||
| `EGRESS_ALLOW_CIDRS` | "" | trusted public ranges that won't trip egress |
|
||||
| `LISTENER_ALLOW_PORTS` | "" | ports that never trip `new_listener` |
|
||||
| `SUID_HOT_DIRS` | /tmp /dev/shm … | dirs where SUID = CRITICAL |
|
||||
| `WATCH_PERSISTENCE` | cron/systemd/… | files watched for tampering |
|
||||
| `CAPTURE_EXECVE_BPFTRACE` | 0 | add a bpftrace execve trace to snapshots |
|
||||
| `MAX_SNAPSHOTS` / `MAX_SNAPSHOT_AGE_DAYS` | 300 / 60 | retention |
|
||||
| `NOTIFY_USERS` | "" | desktop notify-send targets |
|
||||
|
||||
## Tuning out false positives
|
||||
|
||||
- A long-running daemon legitimately running a deleted exe after a package
|
||||
upgrade won't alert — `deleted_exe` only fires for `/tmp`, `/dev/shm`,
|
||||
`/run`, `/var/tmp`, or `memfd:` paths.
|
||||
- Your own admin shells over SSH won't trip `reverse_shell` — they're on a pty.
|
||||
- Browsers and chat apps that interpreters open to public IPs *can* trip
|
||||
`suspicious_egress` if you add them to `INTERPRETERS`; the default list is
|
||||
deliberately scripting-focused. Use `EGRESS_ALLOW_CIDRS` for known-good ranges.
|
||||
| `sample_interval` | 4 | seconds between sweeps |
|
||||
| `cooldown` | 60 | min seconds before re-alerting a signature |
|
||||
| `detectors` | all 7 | the enabled detector list |
|
||||
| `interpreters` | bash sh … | process names treated as shells |
|
||||
| `egress_allow_cidrs` | [] | trusted public ranges (won't trip egress) |
|
||||
| `suid_hot_dirs` | /tmp … | dirs where a SUID binary is CRITICAL |
|
||||
| `suid_scan_extra_dirs` | /tmp … | writable mounts always scanned (tmpfs-safe) |
|
||||
| `capture_execve_bpftrace` | false | add a bpftrace execve trace to snapshots |
|
||||
| `notify_users` | [] | desktop notify-send targets |
|
||||
|
||||
## Security model
|
||||
|
||||
Sentinel runs as root because it must read every process's `/proc`, the full
|
||||
socket table, and root-owned files like `authorized_keys`. The systemd unit
|
||||
constrains that power hard: `ProtectSystem=strict` with the log dir as the only
|
||||
constrains that power: `ProtectSystem=strict` with the log dir as the only
|
||||
writable path, `ProtectHome=read-only`, `NoNewPrivileges`,
|
||||
`MemoryDenyWriteExecute`, and a minimal capability set
|
||||
`MemoryDenyWriteExecute`, `RestrictNamespaces`, and a minimal capability set
|
||||
(`CAP_SYS_PTRACE`, `CAP_DAC_READ_SEARCH`). It only ever **reads** the system and
|
||||
**writes** to its own log directory.
|
||||
|
||||
## Roadmap — from polling to eBPF
|
||||
|
||||
This v0 is **poll-based**: it sweeps `/proc` and `ss` every few seconds. That's
|
||||
robust, dependency-light, and catches anything that lingers — but it can miss
|
||||
sub-second processes, and a sweep is observable. The planned evolution:
|
||||
This is **poll-based**: it sweeps `/proc` and `ss` every few seconds. Robust,
|
||||
dependency-light, and catches anything that lingers — but it can miss sub-second
|
||||
processes. The planned evolution:
|
||||
|
||||
1. **bpftrace tracepoints (now, optional)** — `CAPTURE_EXECVE_BPFTRACE=1` adds a
|
||||
live `execve` trace to each snapshot. The gentle on-ramp to kernel tracing.
|
||||
2. **Event-driven execve/connect detection** — replace the polling gap with
|
||||
`bpftrace` probes on `sys_enter_execve`, `security_bprm_check`, and
|
||||
`tcp_connect`, streamed to the daemon so short-lived reverse shells can't
|
||||
slip between sweeps.
|
||||
3. **A libbpf + CO-RE agent (Go or Rust userland)** — the real EDR core: LSM
|
||||
hooks, ring-buffer event streaming, per-process lineage tracking, and
|
||||
tamper-resistance. This is the production-grade rewrite the polling daemon
|
||||
prototypes the detection logic for.
|
||||
1. **bpftrace tracepoints (now, optional)** — `capture_execve_bpftrace = true`
|
||||
adds a live `execve` trace to each snapshot. The gentle on-ramp to kernel
|
||||
tracing.
|
||||
2. **Event-driven detection** — `bpftrace`/BCC probes on `sys_enter_execve`,
|
||||
`security_bprm_check`, and `tcp_connect`, streamed to the daemon so a
|
||||
short-lived reverse shell can't slip between sweeps.
|
||||
3. **A libbpf + CO-RE agent (Go or Rust userland)** — the production EDR core:
|
||||
LSM hooks, ring-buffer event streaming, per-process lineage, tamper
|
||||
resistance.
|
||||
|
||||
The polling daemon isn't throwaway — it's the **oracle**: every signature here
|
||||
is a test case the eBPF agent must reproduce, and `sentinel-redteam` is the
|
||||
shared regression suite for both.
|
||||
shared regression suite.
|
||||
|
||||
## Project status
|
||||
|
||||
v0.1 — working poll-based detection across 7 signatures, forensic snapshots,
|
||||
red-team harness, systemd packaging. Built and tested on Arch Linux.
|
||||
v0.2 — Python re-architecture of the bash prototype: 7 detectors, text+JSON
|
||||
forensic snapshots, backgrounded SUID scanning, 25-test unit suite, red-team
|
||||
harness, hardened systemd unit, Arch packaging. Zero runtime dependencies.
|
||||
Built and tested on Arch Linux.
|
||||
|
||||
## License
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue