# Enodia Sentinel Operations Guide

This guide turns the current toolset into repeatable operator workflows. It is
deliberately practical: what to run, what to expect, and what to check next.

## First Install Checklist

1. Install and enable the service.

   ```bash
   sudo make install
   sudo make enable
   ```

2. Build the listener and SUID baselines.

   ```bash
   sudo enodia-sentinel baseline
   ```

3. Build the FIM baseline.

   ```bash
   sudo enodia-sentinel fim-baseline
   ```

4. Run a one-shot check.

   ```bash
   sudo enodia-sentinel check
   sudo enodia-sentinel rootcheck
   sudo enodia-sentinel pkgdb-check
   ```

5. Start the dashboard if desired.

   ```bash
   sudo systemctl enable --now enodia-sentinel-web
   ```

   Open the `https://...` URL from the service log. The default certificate is
   self-signed, so add a browser exception or configure `web_tls_cert` and
   `web_tls_key` with your own local/private CA certificate.

6. Configure one off-box notification or watchdog path. A local-only alerting
   path is not enough if the whole host goes silent.

## Routine Health Checks

Run these after upgrades, config edits, or suspicious behavior:

```bash
enodia-sentinel check
enodia-sentinel fim-check
enodia-sentinel pkgdb-check
enodia-sentinel pkgdb-verify --sample 100
enodia-sentinel rootcheck
enodia-sentinel posture check
enodia-sentinel triage
```

Expected healthy output:

- `check`: no alerts, or only expected tuned findings.
- `fim-check`: no changes against baseline after legitimate updates have been
  acknowledged.
- `pkgdb-check`: package DB consistent with anchor.
- `pkgdb-verify`: sampled files match signed cache packages.
- `rootcheck`: no hidden processes, modules, ports, or sniffers.
- `posture check`: no SSH/sudo/PATH/permission/signature hygiene findings, or
  only ones you have consciously accepted (e.g. password auth on a host that
  needs it).

## Alert Workflow

When Sentinel fires:

1. Open the newest JSON and text snapshots in `/var/log/enodia-sentinel`.
2. Identify the signature, `sid`, affected PIDs, executable paths, remote peers,
   and parent process.
3. Run `enodia-sentinel triage` to separate known benign listener noise from
   findings that need review.
4. Build a read-only response plan from the grouped incident:

   ```bash
   enodia-sentinel incident list
   enodia-sentinel respond plan <incident-id>
   ```

   Review the plan before acting; Sentinel does not execute containment
   commands.
5. If the alert involves a binary, run:

   ```bash
   enodia-sentinel fim-check --packages
   enodia-sentinel pkgdb-verify --sample 200
   ```

6. If the alert involves hiding or tampering, run:

   ```bash
   enodia-sentinel rootcheck
   enodia-sentinel pkgdb-check
   ```

7. Preserve evidence before changing state:

   ```bash
   cp -a /var/log/enodia-sentinel /tmp/enodia-sentinel-evidence
   ```

8. Contain manually for now, following the matching playbook in
   [RUNBOOKS.md](RUNBOOKS.md). The response plan gives reviewed commands, but
   applying them remains an explicit operator action.

## Common Findings

| Finding | First checks |
|---|---|
| `reverse_shell` | Inspect parent process, fd table, remote peer, shell argv, and user. |
| `egress` | Confirm interpreter, destination IP, command line, and parent process. |
| `new_listener` | Identify owning process and whether the binary is package-owned. |
| `new_suid` | Check path, owner, package provenance, and whether it lives in a writable directory. |
| `persistence` | Diff the changed file and identify the modifying package or process if possible. |
| `fim_modified` | Verify package ownership, package checksum, and whether the change followed an upgrade. |
| `pkgdb_tamper` | Treat as high-confidence tampering until a legitimate package transaction explains it. |
| `pkg_signature_mismatch` | Treat as a potentially trojaned package-owned file. |
| `hidden_*` rootcheck alerts | Compare live tools, preserve evidence, and consider offline analysis. |

## Baseline Hygiene

Baselines are useful only if they represent a known-good state.

- Build baselines after installing Sentinel on a clean host.
- Rebuild listener/SUID baselines after intentionally adding long-running
  services or privileged binaries.
- Use `fim-update` only after confirming changes are legitimate. Package hooks
  already run it for normal package transactions.
- Keep a copy of important anchors off-box when possible.

## Suggested Hardening

- Keep the dashboard bound to Tailscale or loopback, not a public interface.
- Set `web_token` explicitly for fully read-only service operation.
- Keep HTTPS enabled. The built-in self-signed certificate is acceptable for a
  single host after you add an exception; use a private/local CA for cleaner
  browser trust.
- Enable at least one push backend.
- Run `watchdog` from another machine.
- Consider `chattr +i` for Sentinel binaries, systemd units, config, baselines,
  and the pacman hook after setup.
- Keep the package cache populated if using signed-package verification.
- Do not disable package signature verification.

## Evidence Export

Until the incident command set exists, an evidence bundle is simply:

```bash
tar -C /var/log -czf /tmp/enodia-sentinel-evidence.tgz enodia-sentinel
```

That bundle contains events, snapshots, baselines, heartbeat state, package DB
anchor, and dashboard-readable JSON.