164 lines
5.3 KiB
Markdown
164 lines
5.3 KiB
Markdown
# Enodia Sentinel Operations Guide
|
|
|
|
This guide turns the current toolset into repeatable operator workflows. It is
|
|
deliberately practical: what to run, what to expect, and what to check next.
|
|
|
|
## First Install Checklist
|
|
|
|
1. Install and enable the service.
|
|
|
|
```bash
|
|
sudo make install
|
|
sudo make enable
|
|
```
|
|
|
|
2. Build the listener and SUID baselines.
|
|
|
|
```bash
|
|
sudo enodia-sentinel baseline
|
|
```
|
|
|
|
3. Build the FIM baseline.
|
|
|
|
```bash
|
|
sudo enodia-sentinel fim-baseline
|
|
```
|
|
|
|
4. Run a one-shot check.
|
|
|
|
```bash
|
|
sudo enodia-sentinel check
|
|
sudo enodia-sentinel rootcheck
|
|
sudo enodia-sentinel pkgdb-check
|
|
```
|
|
|
|
5. Start the dashboard if desired.
|
|
|
|
```bash
|
|
sudo systemctl enable --now enodia-sentinel-web
|
|
```
|
|
|
|
Open the `https://...` URL from the service log. The default certificate is
|
|
self-signed, so add a browser exception or configure `web_tls_cert` and
|
|
`web_tls_key` with your own local/private CA certificate.
|
|
|
|
6. Configure one off-box notification or watchdog path. A local-only alerting
|
|
path is not enough if the whole host goes silent.
|
|
|
|
## Routine Health Checks
|
|
|
|
Run these after upgrades, config edits, or suspicious behavior:
|
|
|
|
```bash
|
|
enodia-sentinel check
|
|
enodia-sentinel fim-check
|
|
enodia-sentinel pkgdb-check
|
|
enodia-sentinel pkgdb-verify --sample 100
|
|
enodia-sentinel rootcheck
|
|
enodia-sentinel posture check
|
|
enodia-sentinel triage
|
|
```
|
|
|
|
Expected healthy output:
|
|
|
|
- `check`: no alerts, or only expected tuned findings.
|
|
- `fim-check`: no changes against baseline after legitimate updates have been
|
|
acknowledged.
|
|
- `pkgdb-check`: package DB consistent with anchor.
|
|
- `pkgdb-verify`: sampled files match signed cache packages.
|
|
- `rootcheck`: no hidden processes, modules, ports, or sniffers.
|
|
- `posture check`: no SSH/sudo/PATH/permission/signature hygiene findings, or
|
|
only ones you have consciously accepted (e.g. password auth on a host that
|
|
needs it).
|
|
|
|
## Alert Workflow
|
|
|
|
When Sentinel fires:
|
|
|
|
1. Open the newest JSON and text snapshots in `/var/log/enodia-sentinel`.
|
|
2. Identify the signature, `sid`, affected PIDs, executable paths, remote peers,
|
|
and parent process.
|
|
3. Run `enodia-sentinel triage` to separate known benign listener noise from
|
|
findings that need review.
|
|
4. Build a read-only response plan from the grouped incident:
|
|
|
|
```bash
|
|
enodia-sentinel incident list
|
|
enodia-sentinel respond plan <incident-id>
|
|
```
|
|
|
|
Review the plan before acting; Sentinel does not execute containment
|
|
commands.
|
|
5. If the alert involves a binary, run:
|
|
|
|
```bash
|
|
enodia-sentinel fim-check --packages
|
|
enodia-sentinel pkgdb-verify --sample 200
|
|
```
|
|
|
|
6. If the alert involves hiding or tampering, run:
|
|
|
|
```bash
|
|
enodia-sentinel rootcheck
|
|
enodia-sentinel pkgdb-check
|
|
```
|
|
|
|
7. Preserve evidence before changing state:
|
|
|
|
```bash
|
|
cp -a /var/log/enodia-sentinel /tmp/enodia-sentinel-evidence
|
|
```
|
|
|
|
8. Contain manually for now, following the matching playbook in
|
|
[RUNBOOKS.md](RUNBOOKS.md). The response plan gives reviewed commands, but
|
|
applying them remains an explicit operator action.
|
|
|
|
## Common Findings
|
|
|
|
| Finding | First checks |
|
|
|---|---|
|
|
| `reverse_shell` | Inspect parent process, fd table, remote peer, shell argv, and user. |
|
|
| `egress` | Confirm interpreter, destination IP, command line, and parent process. |
|
|
| `new_listener` | Identify owning process and whether the binary is package-owned. |
|
|
| `new_suid` | Check path, owner, package provenance, and whether it lives in a writable directory. |
|
|
| `persistence` | Diff the changed file and identify the modifying package or process if possible. |
|
|
| `fim_modified` | Verify package ownership, package checksum, and whether the change followed an upgrade. |
|
|
| `pkgdb_tamper` | Treat as high-confidence tampering until a legitimate package transaction explains it. |
|
|
| `pkg_signature_mismatch` | Treat as a potentially trojaned package-owned file. |
|
|
| `hidden_*` rootcheck alerts | Compare live tools, preserve evidence, and consider offline analysis. |
|
|
|
|
## Baseline Hygiene
|
|
|
|
Baselines are useful only if they represent a known-good state.
|
|
|
|
- Build baselines after installing Sentinel on a clean host.
|
|
- Rebuild listener/SUID baselines after intentionally adding long-running
|
|
services or privileged binaries.
|
|
- Use `fim-update` only after confirming changes are legitimate. Package hooks
|
|
already run it for normal package transactions.
|
|
- Keep a copy of important anchors off-box when possible.
|
|
|
|
## Suggested Hardening
|
|
|
|
- Keep the dashboard bound to Tailscale or loopback, not a public interface.
|
|
- Set `web_token` explicitly for fully read-only service operation.
|
|
- Keep HTTPS enabled. The built-in self-signed certificate is acceptable for a
|
|
single host after you add an exception; use a private/local CA for cleaner
|
|
browser trust.
|
|
- Enable at least one push backend.
|
|
- Run `watchdog` from another machine.
|
|
- Consider `chattr +i` for Sentinel binaries, systemd units, config, baselines,
|
|
and the pacman hook after setup.
|
|
- Keep the package cache populated if using signed-package verification.
|
|
- Do not disable package signature verification.
|
|
|
|
## Evidence Export
|
|
|
|
Until the incident command set exists, an evidence bundle is simply:
|
|
|
|
```bash
|
|
tar -C /var/log -czf /tmp/enodia-sentinel-evidence.tgz enodia-sentinel
|
|
```
|
|
|
|
That bundle contains events, snapshots, baselines, heartbeat state, package DB
|
|
anchor, and dashboard-readable JSON.
|