# Enodia Sentinel Operations Guide This guide turns the current toolset into repeatable operator workflows. It is deliberately practical: what to run, what to expect, and what to check next. ## First Install Checklist 1. Install and enable the service. ```bash sudo make install sudo make enable ``` 2. Build the listener and SUID baselines. ```bash sudo enodia-sentinel baseline ``` 3. Build the FIM baseline. ```bash sudo enodia-sentinel fim-baseline ``` 4. Run a one-shot check. ```bash sudo enodia-sentinel check sudo enodia-sentinel rootcheck sudo enodia-sentinel pkgdb-check ``` 5. Start the dashboard if desired. ```bash sudo systemctl enable --now enodia-sentinel-web ``` Open the `https://...` URL from the service log. The default certificate is self-signed, so add a browser exception or configure `web_tls_cert` and `web_tls_key` with your own local/private CA certificate. 6. Configure one off-box notification or watchdog path. A local-only alerting path is not enough if the whole host goes silent. ## Routine Health Checks Run these after upgrades, config edits, or suspicious behavior: ```bash enodia-sentinel check enodia-sentinel fim-check enodia-sentinel pkgdb-check enodia-sentinel pkgdb-verify --sample 100 enodia-sentinel rootcheck enodia-sentinel posture check enodia-sentinel triage ``` Expected healthy output: - `check`: no alerts, or only expected tuned findings. - `fim-check`: no changes against baseline after legitimate updates have been acknowledged. - `pkgdb-check`: package DB consistent with anchor. - `pkgdb-verify`: sampled files match signed cache packages. - `rootcheck`: no hidden processes, processes hidden from `ps`, hidden modules/ports/raw/special-protocol sockets, sniffers, known rootkit modules, raw ICMP/SCTP-style channels, or unexplained kernel/module taint. - `posture check`: no SSH/sudo/PATH/permission/signature hygiene findings, or only ones you have consciously accepted (e.g. password auth on a host that needs it). The HTTPS dashboard shows the same posture findings under the Posture tab and summarizes FIM/package anchors plus heartbeat freshness under the Integrity tab for quick remote review. For SSH/tmux operators, `enodia-sentinel tui` provides a terminal dashboard over the same local read-only management state: status, recent alerts, incidents, incident timelines, posture findings, integrity/watchdog state, event-rule metadata, event tail, and dry-run response-plan previews. Newer operators can follow the built-in workflow hints: Status -> Incidents -> Timeline -> Response Plans -> Integrity. Press `n`/`b` to step through that workflow, or use `:` for command mode and `Tab` to complete TUI commands. On desktop hosts the optional tray applet (`enodia-sentinel-tray`, `[tray]` extra) wraps the same workflow: it drives the same `systemctl` units operators use directly, so daemon start/stop/restart prompts for polkit authorization as usual. The applet is purely a convenience frontend — it is not required for the daemon to run and adds nothing to the core agent. A windowed desktop GUI (`enodia-sentinel gui` / `enodia-sentinel-gui`) is also available using only stdlib `tkinter`. It shows the same read-only tabs as the TUI plus daemon-control buttons, and like the tray applet it is optional and does not change the core agent's zero-dependency design. A richer Qt6 frontend (`enodia-sentinel gui-qt` / `enodia-sentinel-gui-qt`, `[qt]` extra with `PySide6`) provides the same data with a native look, resizable tables, and a response-plan detail pane. ## Alert Workflow When Sentinel fires: 1. Open the newest JSON and text snapshots in `/var/log/enodia-sentinel`. 2. Identify the signature, `sid`, affected PIDs, executable paths, remote peers, and parent process. 3. Run `enodia-sentinel triage` to separate known benign listener noise from findings that need review. 4. Build a read-only response plan from the grouped incident: ```bash enodia-sentinel incident list enodia-sentinel respond plan ``` Review the plan before acting; Sentinel does not execute containment commands. CLI-generated plans are saved under `/var/log/enodia-sentinel/response-plans/` and indexed in `/var/log/enodia-sentinel/response-audit.log` for later review. To rehearse a saved plan and record that review without executing anything: ```bash enodia-sentinel respond apply --dry-run ``` 5. If the alert involves a package-owned binary, review the package recovery actions in the response plan and run the restore sequence only after the owning package and package source are trusted: ```bash pacman -Qo /path/to/file enodia-sentinel pkgdb-verify --sample 200 pacman -S --overwrite '*' enodia-sentinel fim-check --packages enodia-sentinel fim-update ``` 6. If the alert involves hiding or tampering, run: ```bash enodia-sentinel rootcheck enodia-sentinel pkgdb-check ``` 7. Preserve evidence before changing state: ```bash cp -a /var/log/enodia-sentinel /tmp/enodia-sentinel-evidence ``` 8. Contain manually for now, following the matching playbook in [RUNBOOKS.md](RUNBOOKS.md). The response plan gives reviewed commands, but applying them remains an explicit operator action. ## Common Findings | Finding | First checks | |---|---| | `reverse_shell` | Inspect parent process, fd table, remote peer, shell argv, and user. | | `egress` | Confirm interpreter, destination IP, command line, and parent process. | | `new_listener` | Identify owning process and whether the binary is package-owned. | | `new_suid` | Check path, owner, package provenance, and whether it lives in a writable directory. | | `persistence` | Diff the changed file and identify the modifying package or process if possible. | | `fim_modified` | Verify package ownership, package checksum, and whether the change followed an upgrade. | | `pkgdb_tamper` | Treat as high-confidence tampering until a legitimate package transaction explains it. | | `pkg_signature_mismatch` | Treat as a potentially trojaned package-owned file. | | `rootkit_*` / `kernel_tainted` alerts | Compare live tools, review module provenance, preserve evidence, and consider offline analysis. | ## Baseline Hygiene Baselines are useful only if they represent a known-good state. - Build baselines after installing Sentinel on a clean host. - Rebuild listener/SUID baselines after intentionally adding long-running services or privileged binaries. - Use `fim-update` only after confirming changes are legitimate. Package hooks already run it for normal package transactions. - Keep a copy of important anchors off-box when possible. - Prefer `baseline accept --reason "..."` over rebuilding a whole baseline when only one item changed legitimately (an edited config, a new service port, an installed SUID helper). Unlike a rebuild, the acceptance is per-item, carries a recorded reason, is fingerprint-bound (FIM/package items re-alert if the file changes *again*), and can be time-boxed with `--expires`. Review outstanding acceptances with `baseline list`; it exits non-zero and marks rows `stale` when an accepted item has drifted again or its TTL lapsed. ## Suggested Hardening - Keep the dashboard bound to Tailscale or loopback, not a public interface. - Set `web_token` explicitly for fully read-only service operation. - Keep HTTPS enabled. The built-in self-signed certificate is acceptable for a single host after you add an exception; use a private/local CA for cleaner browser trust. - Enable at least one push backend. - Run `watchdog` from another machine. - Consider `chattr +i` for Sentinel binaries, systemd units, config, baselines, and the pacman hook after setup. - Keep the package cache populated if using signed-package verification. - Do not disable package signature verification. ## Evidence Export Until the incident command set exists, an evidence bundle is simply: ```bash tar -C /var/log -czf /tmp/enodia-sentinel-evidence.tgz enodia-sentinel ``` That bundle contains events, snapshots, baselines, heartbeat state, package DB anchor, and dashboard-readable JSON.