Enodia Sentinel

A Linux host security platform: intrusion detection, file/package integrity, anti-rootkit cross-checks, tamper-evidence, forensic snapshots, push alerts, and a read-only operator console.

The current daemon continuously runs detectors over live system state — processes, sockets, file descriptors, the SUID inventory, sensitive files, the package database, and rootkit-hiding artifacts — then writes a detailed forensic snapshot (text and JSON) with incident-response guidance the moment a known attack signature appears.

Think of it as the security counterpart to a performance watchdog: instead of "I/O pressure spiked, here's the kernel state," it's "a shell just wired itself to a socket — here's the process tree, the peer, what changed on disk, whether the package database was tampered with, and what to do next."

Product direction

Enodia is becoming more than an IDS. The local detector is the first layer of a broader host security platform:

Function	Current capability	Direction
Detect	Poll detectors + eBPF exec rules	More event sources and correlation
Verify	FIM, package DB anchor, signed-package checks	External anchors and signed evidence
Investigate	Text/JSON snapshots, dashboard, triage	Incident timelines and evidence export
Respond	Human guidance today	Dry-run response plans and audited containment
Assure	Heartbeat, watchdog, self-integrity, rootcheck	Fleet health and attestation-ready anchors

Project docs:

• Specification — product model, current scope, target platform shape, data model, and acceptance criteria.
• Roadmap — phased work from local sensor to incident, response, fleet, and assurance layers.
• Operations guide — install checks, health checks, alert workflow, baseline hygiene, and evidence export.
• IR runbooks — confirm/preserve/contain/recover playbooks per alert: reverse shells, persistence, trojaned binaries, rootkits, tampering.

Two implementations, on purpose. The project began as a bash prototype (src/sentinel.sh, kept as the regression oracle) and was re-architected into a zero-dependency Python package with a unit-test suite, structured detectors, and JSON output. The bash version and the Python version share one red-team harness, so every signature is exercised against both.

Why these detectors

Every detector keys on a behavior that is cheap to observe and expensive for an attacker to avoid — the high-signal, low-false-positive heuristics real EDRs are built on:

Signature	What it catches	Why it's hard to evade
reverse_shell	An interpreter with a network socket on fd 0/1/2	Interactive shells get a pty and daemons get unix sockets — a network socket on stdio is nc -e / bash -i >& /dev/tcp/...
ld_preload	Non-empty /etc/ld.so.preload, or LD_PRELOAD into a writable dir	Injecting into processes needs the library to exist somewhere
deleted_exe	A process running from a deleted / memfd: binary	Fileless malware deletes its dropper; the kernel still names the inode (deleted)
new_listener	A listening port absent from the startup baseline	Bind shells/backdoors have to listen somewhere
new_suid	A new SUID/SGID binary (critical in a writable dir)	A SUID /tmp binary is a textbook privesc trick
persistence	Changes to cron, systemd units, authorized_keys, rc files	Persistence has to write somewhere that survives reboot
egress	An interpreter with an established connection to a public IP	C2 beacons and exfil have to phone home

Every detection carries a stable sid and a classtype (à la Snort/Suricata), so it can be referenced, tuned, and tracked across revisions.

Event-driven detection (eBPF + a Snort-style rule engine)

Polling has a blind spot: a process that runs and exits between two sweeps is invisible to it. The event layer closes that gap. An eBPF probe (loaded with bcc) fires on every execve and hands each event to a declarative rule engine — the host-event analogue of Snort matching packets:

# a rule is data, not code — sid, msg, classtype, severity + conditions
sid        = 100002
msg        = "Reverse-shell command pattern in execve arguments"
severity   = "CRITICAL"
classtype  = "c2-reverse-shell"
argv_regex = "/dev/(tcp|udp)/| -i\\b| -e\\b| pty\\.spawn"

Shipped rules cover fileless execution from world-writable dirs (sid 100001), reverse-shell argv patterns (100002), web/DB services spawning a shell — webshell/RCE (100003), and curl|sh-style ingress tool transfer (100004). Operators add their own via exec_rules_file without touching code.

The layer is fail-safe: if bcc/root/BTF aren't available it logs the reason and the daemon runs poll-only — a broken probe can never take detection down. Lineage: the rule-driven engine + SIDs come from Snort; the host-IDS framing (and the queued FIM / hidden-process checks) from OSSEC.

Architecture

enodia_sentinel/
├── cli.py            run / check / baseline / list-detectors
├── daemon.py         sweep loop · cooldown dedup · backgrounded SUID scan
├── system.py         SystemState — one cached snapshot of /proc + ss per sweep
├── snapshot.py       forensic text+JSON capture · response guidance · retention
├── config.py         dataclass config (TOML + env overrides)
├── netutil.py        public-IP / CIDR logic (stdlib ipaddress)
├── alert.py          Alert / Severity (with Snort-style sid + classtype)
├── web.py            read-only dashboard: stdlib http server + JSON API + auth
├── static/           the self-contained dashboard SPA
├── fim.py            file integrity monitoring (hash baseline + pacman verify)
├── pkgdb.py          package-DB integrity + signed-package verification
├── rootcheck.py      anti-rootkit cross-view (hidden procs/modules/ports)
├── selfprotect.py    self-integrity footprint + dead-man's-switch heartbeat
├── triage.py         false-positive classification
├── provenance.py     package-ownership lookups (pacman/dpkg/rpm)
├── detectors/        poll detectors — one module per signature, each a pure
│                       function: detect(state, cfg) -> Iterable[Alert]
├── notify/           outbound push — ntfy / Pushover / webhook backends
└── events/           event-driven layer (eBPF)
    ├── bcc_source.py   real eBPF execve probe loaded via bcc
    ├── exec_event.py   the ExecEvent type
    ├── rules.py        Snort-style ExecRule engine + default rules
    └── monitor.py      runs the probe on a thread, routes events → rules

Two complementary detection paths feed one Alert → snapshot pipeline:

• Poll — every few seconds, sweep /proc/ss (catches anything lingering).
• Event — eBPF fires on every execve, matched against the rule engine (catches processes that exit between sweeps).

The loop is deliberately the same control flow as the bash prototype, but the state lives in real objects:

every sample_interval seconds:
    state  = SystemState()                 # /proc + ss gathered once, cached
    alerts = run_all(detectors, state, cfg)
    fresh  = drop alerts still within cooldown
    if fresh:
        snapshot.capture(fresh)            # on a worker thread

Two design choices keep it fast and unobtrusive:

• One SystemState per sweep. Detectors read shared, cached /proc/ss data instead of each shelling out — a sweep costs ~200 ms regardless of how many detectors run.
• The filesystem-wide SUID scan runs off the loop thread on a slow cadence, so the multi-second walk never stalls live detection.

Everything in SystemState is injectable, which is what makes the detectors unit-testable without root or a live system (see tests/).

Quick start

sudo make install
sudo make enable          # start + enable the systemd service

# prove it works — in one terminal:
sudo tail -f /var/log/enodia-sentinel/events.log
# in another:
sentinel-redteam          # safe, self-cleaning attack simulations

You'll watch the drills trip the core poll signatures in real time, each producing a .log + .json snapshot with response guidance.

Output lives in /var/log/enodia-sentinel/:

• events.log — one line per alert
• alert-YYYYMMDD-HHMMSS.log — human-readable forensic snapshot
• alert-YYYYMMDD-HHMMSS.json — same data, structured (SIEM-ready)

Without installing

make test                                   # run the unit suite
python3 -m enodia_sentinel.cli baseline     # establish baselines
python3 -m enodia_sentinel.cli check        # run every detector once, print findings

No pip, no virtualenv, no dependencies — it's stdlib-only and installs as a plain package directory plus a launcher wrapper.

The red-team harness

sentinel-redteam is the demo and the integration test in one. It simulates each threat with safe, clearly-labeled stand-ins (everything tagged enodia-drill, auto-cleaned on exit), using a local Python TCP listener so no traffic ever leaves the host:

sentinel-redteam --list                  # list drills
sentinel-redteam reverse_shell new_suid  # run specific ones
HOLD=30 sentinel-redteam                  # keep artifacts alive 30s

It never touches your real dotfiles or /etc/ld.so.preload; the LD_PRELOAD drill only sets the env var on a throwaway process.

Testing

make test          # stdlib unittest suite, no runtime deps

Detectors are pure functions over an injectable SystemState, so tests build fake processes/sockets and assert on the alerts — no root, no /proc, no ss:

proc  = FakeProc(pid=100, comm="bash", _stdio_inode=999)
sock  = Socket("ESTAB", "127.0.0.1:55", "9.9.9.9:443", 999, "bash", 100)
state = SystemState(processes=[proc], sockets=[sock])
assert list(reverse_shell.detect(state, Config()))[0].signature == "reverse_shell"

Configuration

Edit /etc/enodia-sentinel.toml, then sudo systemctl restart enodia-sentinel.service. Every key is optional. Highlights:

Key	Default	Purpose
sample_interval	4	seconds between sweeps
cooldown	60	min seconds before re-alerting a signature
detectors	all 7	the enabled detector list
interpreters	bash sh …	process names treated as shells
egress_allow_cidrs	[]	trusted public ranges (won't trip egress)
suid_hot_dirs	/tmp …	dirs where a SUID binary is CRITICAL
suid_scan_extra_dirs	/tmp …	writable mounts always scanned (tmpfs-safe)
capture_execve_bpftrace	false	add a bpftrace execve trace to snapshots
notify_users	[]	desktop notify-send targets
pkgdb_pkgverify	false	verify on-disk files against signed cache packages
pkgdb_pkgverify_sample	40	packages verified per pass (rotates over time)
rootcheck_enabled	true	run the anti-rootkit cross-view sweep
rootcheck_interval	300	seconds between cross-view sweeps

Web dashboard

A read-only console, served by the stdlib http.server (no Flask, no JS framework, no CDN — one self-contained page):

enodia-sentinel web                      # serves on the Tailscale IP by default
# or as a service:
sudo systemctl enable --now enodia-sentinel-web

• Bound to your Tailscale interface by default (auto-detected), so it's reachable from your phone/laptop on the tailnet but not the LAN or internet.
• Bearer-token auth (constant-time check); the token is auto-generated and saved on first run and printed in the startup line. Open http://<tailscale-ip>:8787/?token=….
• Read-only: severity cards, a live alert list, and the full forensic snapshot per alert. No actions, no writes — minimal attack surface for sensitive data. JSON API at /api/status, /api/alerts, /api/alerts/<id>, /api/events.

Phone push notifications

When an alert at/above notify_min_severity fires, Sentinel pushes to whichever backends you've configured (all via stdlib urllib, no SDKs):

Backend	Enable by setting	Notes
ntfy	notify_ntfy_url + notify_ntfy_topic	open-source, self-hostable, free apps
Pushover	notify_pushover_token + _user	polished, reliable
Webhook	notify_webhook_url	generic JSON POST (Discord/Slack/your own)

Severity maps to each service's priority (a CRITICAL is an urgent ntfy push / a high-priority Pushover). Sends happen on worker threads and swallow their own errors — a flaky notifier never stalls detection.

notify_min_severity = "HIGH"
notify_ntfy_url   = "https://ntfy.sh"
notify_ntfy_topic = "enodia-7Hq2x"      # keep this secret — it's the access control

Tamper-evidence & self-protection

An attacker's first move against a sensor is to disable or blind it — and on a box where they have root, any purely-local defense is ultimately defeatable (they share your privileges). Sentinel doesn't pretend otherwise. The goal is to make tampering evident by anchoring trust where the attacker has less control, and to make going silent loud.

Package-DB integrity. pacman -Qkk trusts the local package database — so a root attacker can modify /usr/bin/ssh and rewrite its stored checksum, and verification passes. Sentinel guards the DB itself: a legitimate change only happens during a logged transaction, so it anchors a fingerprint of /var/lib/pacman/local (refreshed only by the pacman hook) and cross-checks pacman.log. A DB change with no corresponding transaction is flagged pkgdb_tamper (CRITICAL, sid 100021) — directly catching the "overwrite the hashes" attack.

Self-integrity. Sentinel's own binaries, config, systemd units, and pacman hook are always in the FIM watch set, so tampering with the watchdog trips the watchdog.

Dead-man's switch. The daemon writes a heartbeat every loop; the dashboard exposes its age. Run an external watcher on another host (over Tailscale) and silence becomes the alarm:

# on a SEPARATE machine — alerts you if the sensor or the whole box goes dark
enodia-sentinel watchdog --url http://100.x.x.x:8787 --token <T> --max-age 120

Hardening (the layers beyond on-box detection):

1. Immutability — chattr +i Sentinel's binaries, config, baselines, and the pacman hook so even root must visibly clear the flag first.
2. ✅ Cryptographic anchor (done) — verify on-disk files against the signed package in the cache rather than the mutable DB; a maintainer's PGP signature is a root of trust the attacker doesn't hold. See Signed-package verification below.
3. External anchor (next) — mirror the DB/FIM fingerprints off-box so a local attacker can't refresh the anchor to cover their tracks.

On "anti-rootkit": the robust version isn't stealth (fragile, and a genuine dual-use rootkit technique) — it's tamper-evidence. Don't hide; be un-hideable. The durable defenses move the trust anchor below or outside the attacker: signatures, external monitors, and ultimately the kernel (the eBPF roadmap) rather than userland the attacker can rewrite.

Signed-package verification (the independent anchor)

Layer 1 (above) catches an out-of-band DB edit. But a root attacker can do the full job: modify /usr/bin/sshd, rewrite its hash in the local DB, and run fim-update to re-anchor — defeating both pacman -Qkk and the DB-fingerprint check, because every reference they're checked against is one they can rewrite.

The one reference they can't forge is the distro's signing key. Packages in the cache are signed, and each carries a .MTREE manifest of per-file SHA-256 hashes. Layer 2 extracts that manifest from the cached package and compares the on-disk files to it — a reference independent of the local DB:

enodia-sentinel pkgdb-verify              # verify a rotating sample of packages
enodia-sentinel pkgdb-verify --sample 200 # verify more per run

A divergence is pkg_signature_mismatch (CRITICAL, sid 100027) — a trojaned binary that a rewritten checksum DB would have hidden. It also flags a SigLevel downgrade in pacman.conf (pacman_siglevel_disabled, CRITICAL, sid 100026), since disabling signature checking is how an attacker would slip an unsigned package past the anchor in the first place.

Verifying every package each pass is expensive (untar + hash every file), so it runs on its own slow cadence (pkgdb_pkgverify_interval) and checks a rotating sample (pkgdb_pkgverify_sample) per pass — over enough passes the whole installed set is covered, and any single mismatch fires immediately. It's off by default (needs the package cache populated); enable with pkgdb_pkgverify = true.

Anti-rootkit (cross-view detection)

A rootkit hides by lying to one view of the system — but the technique that hides it is also how you catch it: ask the same question two different ways and compare the answers. A discrepancy is the hiding artifact.

Cross-check	Hidden thing it surfaces	sid
kill(pid, 0) for every PID vs the /proc listing	a process the kernel schedules but /proc omits	100022
/sys/module (initstate=live) vs /proc/modules	a loaded LKM hidden from the module list	100023
/proc/net/tcp vs ss	a listening port a hooked ss won't report	100024
/sys/class/net/*/flags	an interface in promiscuous mode (a sniffer)	100025

enodia-sentinel rootcheck     # one-shot cross-view scan

The daemon runs it on a slow background cadence (rootcheck_interval) and routes any finding into the normal alert/snapshot/push pipeline. Honest limits: this runs in user space, so it reliably catches userland (LD_PRELOAD) rootkits and the common /proc-hiding LKMs, but a kernel rootkit that hooks every path consistently can still evade it. It raises the bar and catches the common cases; it is not a guarantee against a bespoke ring-0 implant — pair it with the off-box dead-man's switch. (Lineage: OSSEC's rootcheck / chkrootkit's cross-view idea.)

File integrity monitoring (Tripwire-style)

Detects tampering with binaries and critical configs by content hash, so it catches a malicious swap even when the timestamp is preserved (the weakness of mtime-based checks). Two engines, split by who owns the file:

• Package verification — pacman -Qkk checks package-owned binaries against the distro's own signed checksums. No baseline to maintain, and it's implicitly current because the package DB updates on every pacman -Syu. A trojaned /usr/bin/ssh surfaces as a checksum mismatch (CRITICAL, sid 100020).
• Hash baseline — a SHA-256 baseline of the files pacman doesn't track (/usr/local, /etc configs, systemd units, SSH keys). A pacman PostTransaction hook runs fim-update after every upgrade, so legitimate package changes never alert — no manual tripwire --update ritual.

enodia-sentinel fim-baseline          # establish the baseline
enodia-sentinel fim-check             # report changes vs baseline
enodia-sentinel fim-check --packages  # also verify package-owned files

The baseline is only refreshed by fim-update / the pacman hook — a flagged change stays flagged until you acknowledge it, exactly like Tripwire. The daemon runs the scan on a slow background cadence (fim_scan_interval) and routes any change into the normal alert/snapshot/push pipeline (fim_modified sid 100017, fim_added 100018, fim_removed 100019).

False positives & triage

A host IDS that cries wolf gets ignored, so Sentinel ships explicit tooling to separate benign noise from real findings — built on provenance: a binary shipped by your package manager (pacman/dpkg/rpm) is overwhelmingly likely to be legitimate (the same idea behind OSSEC's rootcheck and AIDE).

enodia-sentinel triage     # classify captured alerts, suggest allowlist entries

12 distinct detections — 11 likely false-positive, 1 to review.

[FP  ] new_listener   x99  listener binary is package-owned (qbittorrent)
[FP  ] new_listener   x1   loopback-only listener (not externally reachable)
[REVIEW] new_listener x1   unrecognized listener *:1740 (?)
...
To suppress the false positives, add to your config:
  # listener_allow_comms += "qbittorrent"

Triage is deliberately conservative: reverse_shell, egress, and the eBPF exec rules are always flagged review (provenance can't clear a network shell), and any listener it can't attribute to a process is reviewed rather than cleared. Suppression is never automatic — you choose what to allowlist.

Knobs to quiet known-good activity:

Config	Effect
listener_allow_comms	never alert on listeners owned by these apps
listener_allow_ports	never alert on these ports
suppress_package_owned_listeners	drop new_listener when the binary is package-owned (best single knob for a desktop/seedbox)
egress_allow_cidrs	trusted public ranges for the egress detector
suid_hot_dirs / exec_rules_file	tune SUID criticality / add custom exec rules

Security model

Sentinel runs as root because it must read every process's /proc, the full socket table, and root-owned files like authorized_keys. The systemd unit constrains that power: ProtectSystem=strict with the log dir as the only writable path, ProtectHome=read-only, NoNewPrivileges, MemoryDenyWriteExecute, RestrictNamespaces, and a minimal capability set (CAP_SYS_PTRACE, CAP_DAC_READ_SEARCH). It only ever reads the system and writes to its own log directory.

Enabling the eBPF monitor

The event layer needs python-bpfcc and privileges the hardened unit deliberately withholds (bcc JIT-compiles its programs, so it needs write+exec memory and CAP_BPF/CAP_PERFMON/CAP_SYS_ADMIN). Under the default unit the monitor simply fails closed and the daemon runs poll-only. To turn it on:

sudo pacman -S python-bpfcc
sudo install -Dm644 systemd/enodia-sentinel-ebpf.conf \
    /etc/systemd/system/enodia-sentinel.service.d/ebpf.conf
sudo systemctl daemon-reload && sudo systemctl restart enodia-sentinel
# confirm:  grep 'eBPF exec monitor' /var/log/enodia-sentinel/events.log

The drop-in relaxes MemoryDenyWriteExecute and widens the capability set — a conscious tradeoff documented in the file itself.

Roadmap

The short version: Enodia grows from local HIDS into a host security platform: incident grouping, posture checks, response planning, richer eBPF telemetry, fleet health, external anchors, and eventually attestation-ready assurance.

See docs/ROADMAP.md for the release tracks and phased plan. The polling daemon isn't throwaway — it's the oracle: every signature is a test case the event layer must reproduce, and sentinel-redteam is the shared regression suite for both.

Project status

v0.7 — closes the tamper-evidence loop with the independent anchor: signed-package verification (compares on-disk files to the .MTREE in the signed cache package, surviving a rewritten checksum DB) plus a SigLevel-downgrade check, and an anti-rootkit cross-view layer (hidden processes, modules, ports, and promiscuous interfaces — each caught by asking the same question two ways and diffing the answers).

v0.6 — adds tamper-evidence: out-of-band package-DB integrity (catches rewritten checksums), self-integrity of Sentinel's own footprint, and a dead-man's-switch heartbeat with an external watchdog.

v0.5 — adds file integrity monitoring (SHA-256 baseline + pacman -Qkk verification, auto-refreshed by a pacman hook) and false-positive triage via package-ownership provenance.

v0.4 — adds a read-only web dashboard (stdlib server, Tailscale-bound, token-auth) and phone push (ntfy / Pushover / webhook), both zero-dependency.

v0.3 — adds the event-driven eBPF layer: a real bcc execve probe feeding a Snort-style declarative rule engine (4 default rules), stable signature IDs + classtypes on every detection, fail-safe degradation to poll-only, and an opt-in hardening drop-in. Inspired by Snort (rule engine, SIDs) and OSSEC (HIDS framing; FIM + hidden-process checks are next).

v0.2 — Python re-architecture of the bash prototype: 7 detectors, text+JSON forensic snapshots, backgrounded SUID scanning, 25-test unit suite, red-team harness, hardened systemd unit, Arch packaging. Zero runtime dependencies. Built and tested on Arch Linux.

License

GPL-3.0-or-later — see LICENSE.