feat(go): advance validation sidecar toward production
This commit is contained in:
Luna
parent
6a06eba255
commit
f85c2e831a
92 changed files with 8881 additions and 91 deletions
276
docs/behavior/02-poll-detectors.md
Normal file
276
docs/behavior/02-poll-detectors.md
Normal file
|
|
@ -0,0 +1,276 @@
|
|||
<!-- SPDX-License-Identifier: GPL-3.0-or-later -->
|
||||
# 2. Poll Detectors
|
||||
|
||||
Scope: the per-detector behavioral contract for the twelve pure detectors in
|
||||
`detectors/`. Each is a `detect(state, cfg) -> Iterable[Alert]` over the
|
||||
`SystemState` built in §1. This section documents, per detector, the inputs it
|
||||
reads, the match algorithm, the exact alert it emits, and its allow-list /
|
||||
fail-open rules. The registry order and the baseline-arming gate are in §1.3 /
|
||||
§1.5; the alert fields and dedup keying are in §1.1.
|
||||
|
||||
Every detector is fail-open by inheritance: it reads already-captured
|
||||
`SystemState` fields, and missing `/proc`/`ss` data is an empty value, never an
|
||||
exception.
|
||||
|
||||
## 2.0 Signature summary
|
||||
|
||||
The stable contract. A port emitting a different SID, severity, classtype, or
|
||||
key for the same condition is a parity break.
|
||||
|
||||
| SID | signature | severity | classtype | key |
|
||||
|-----|-----------|----------|-----------|-----|
|
||||
| 100010 | `reverse_shell` | CRITICAL | `c2-reverse-shell` | `rsh:<pid>` |
|
||||
| 100011 | `ld_preload` | CRITICAL | `rootkit-preload` | `ldp:global`, `ldp:<pid>` |
|
||||
| 100012 | `deleted_exe` | CRITICAL | `fileless-execution` | `del:<pid>` |
|
||||
| 100013 | `new_listener` | HIGH | `backdoor-listener` | `lis:<port>/<comm>` |
|
||||
| 100014 | `new_suid` | CRITICAL / HIGH | `privilege-escalation` | `suid:<path>` |
|
||||
| 100015 | `persistence` | HIGH | `persistence` | `persist:<path>` |
|
||||
| 100016 | `egress` | HIGH | `c2-exfil` | `egr:<pid>:<host>` |
|
||||
| 100032 | `input_snooper` | HIGH | `credential-keylogging` | `input:<pid>:<target>` |
|
||||
| 100033 | `credential_access` | CRITICAL | `credential-theft` | `cred:<pid>:<target>` |
|
||||
| 100036 | `stealth_network` | MEDIUM / HIGH | `covert-channel` | `stealthnet:<kind>:<pid>:<local>:<peer>` |
|
||||
| 100039 | `memory_obfuscation` | HIGH / CRITICAL | `memory-obfuscation` | `mem:memory_obfuscation:<pid>:<lo>-<hi>` |
|
||||
| 100048 | `process_hiding_library` | CRITICAL | `process-hiding` | `mem:process_hiding_library:<pid>:<lo>-<hi>` |
|
||||
| 100049 | `process_injection_library` | HIGH | `process-injection` | `mem:process_injection_library:<pid>:<lo>-<hi>` |
|
||||
| 100074 | `first_public_destination` | MEDIUM | `network-rarity` | `firstdest:<comm>:<ip:port>` |
|
||||
| 100075 | `first_listener_port` | MEDIUM | `network-rarity` | `firstlisten:<comm>:<port>` |
|
||||
|
||||
`<lo>-<hi>` are the map bounds formatted as lowercase hex (`{start:x}-{end:x}`).
|
||||
|
||||
Allow-list config keys and their defaults live in `config.py`; this section
|
||||
names each key where used. The Go port mirrors the same defaults (see
|
||||
`go-agent/internal/config`), so the values are asserted by config tests, not
|
||||
repeated here.
|
||||
|
||||
---
|
||||
|
||||
## 2.1 Shared address helpers (`netutil.py`)
|
||||
|
||||
Used by `egress`, `stealth_network`, and `first_seen`.
|
||||
|
||||
- `split_host_port(endpoint)` — splits an `ss` endpoint. If it starts with `[`,
|
||||
partition on `]` (IPv6, e.g. `[::1]:22` → `::1`, `22`); else `rpartition(":")`
|
||||
(e.g. `10.0.0.2%eth0:443` → `10.0.0.2%eth0`, `443`). Returns `(host, port)`
|
||||
strings.
|
||||
- `parse_addr(addr)` — strips `[]`, drops an IPv6 `%zone`, returns an
|
||||
`ipaddress` object or `None`.
|
||||
- `is_public_ip(addr)` — `True` only if `parse_addr(...).is_global`: excludes
|
||||
loopback, RFC1918, link-local, CGNAT (100.64/10), and other non-global
|
||||
ranges. Unparseable → `False`.
|
||||
- `ip_in_cidrs(addr, cidrs)` — `True` if `addr` is inside any CIDR (the operator
|
||||
trust list); version-matched; bad CIDRs skipped.
|
||||
|
||||
## 2.2 Package provenance (`provenance.py`)
|
||||
|
||||
Used by `new_listener` (§2.11) as an optional suppressor.
|
||||
|
||||
`package_owner(path)` returns the owning package string or `None`, using the
|
||||
first available of `pacman -Qo` / `dpkg -S` / `rpm -qf` (5s timeout each, cached
|
||||
via `lru_cache`; `(deleted)` suffix stripped first). `is_package_owned(path) =
|
||||
package_owner(path) is not None`. **Contract:** ownership *ranks/explains*, it
|
||||
never proves safety — it may suppress a `new_listener` alert but is never used
|
||||
to silently drop anything else.
|
||||
|
||||
---
|
||||
|
||||
## 2.3 `reverse_shell` — SID 100010, CRITICAL
|
||||
|
||||
Interpreter process with a **network** socket on its stdio.
|
||||
|
||||
- Input: `state.processes`, `state.net_peer_by_inode`.
|
||||
- Match, per process: `proc.comm in cfg.interpreters`; `proc.stdio_socket_inode()`
|
||||
is not `None` (fd 0/1/2 links to `socket:[…]`); and that inode appears in the
|
||||
network socket table (`net_peer_by_inode`). The last condition is what
|
||||
excludes benign unix sockets / pipes and keeps false positives near zero.
|
||||
- Emit: key `rsh:<pid>`, `pids=(pid,)`, detail
|
||||
`pid=… comm=… stdio=net-socket peer=[local -> peer] cmd=[<cmdline first 90>]`.
|
||||
- Config: `interpreters` allow-*set of shell/scripting comms* (membership means
|
||||
"watch", not "trust").
|
||||
|
||||
## 2.4 `ld_preload` — SID 100011, CRITICAL
|
||||
|
||||
Two independent signatures, same SID:
|
||||
|
||||
1. **Global** — `/etc/ld.so.preload` exists and is non-empty (fail-open read).
|
||||
Key `ldp:global`, no pids, detail includes the file contents (newlines →
|
||||
spaces).
|
||||
2. **Per-process** — a process whose `environ["LD_PRELOAD"]` starts with a
|
||||
suspicious prefix: `("/tmp/", "/dev/shm/", "/var/tmp/", "/run/user/", "./")`.
|
||||
Key `ldp:<pid>`, `pids=(pid,)`.
|
||||
|
||||
Note the global check reads `/etc/ld.so.preload` directly (not the injected
|
||||
`state`), so a port must read it live inside the detector or supply it on state.
|
||||
|
||||
## 2.5 `deleted_exe` — SID 100012, CRITICAL
|
||||
|
||||
Process running from a deleted or memfd-backed binary in an attacker-controlled
|
||||
location.
|
||||
|
||||
- Match: `proc.exe` is fileless — `"memfd:" in exe`, **or** `"(deleted)" in exe`
|
||||
**and** `exe` starts with `("/tmp/", "/dev/shm/", "/var/tmp/", "/run/")`.
|
||||
Normal-path deleted exes (a daemon still running after a package upgrade) are
|
||||
ignored on purpose.
|
||||
- Emit: key `del:<pid>`, `pids=(pid,)`, detail `pid=… comm=… exe=[…]`.
|
||||
|
||||
## 2.6 `input_snooper` — SID 100032, HIGH
|
||||
|
||||
Process holding a keyboard/HID event device open (keylogger tell).
|
||||
|
||||
- Match, per process: skip if `comm in cfg.input_snooper_allow_comms`; then scan
|
||||
`fd_targets` for a target starting with `("/dev/input/event", "/dev/uinput",
|
||||
"/dev/hidraw")`. **First** matching fd emits and then `break` (at most one
|
||||
alert per process).
|
||||
- Emit: key `input:<pid>:<target>`, `pids=(pid,)`, detail
|
||||
`pid=… comm=… fd=… has input device open: <target>`.
|
||||
- Config: `input_snooper_allow_comms` (display servers, compositors, remappers).
|
||||
|
||||
## 2.7 `credential_access` — SID 100033, CRITICAL
|
||||
|
||||
Live process with a sensitive credential file open.
|
||||
|
||||
- Match, per process: skip if `comm in cfg.credential_access_allow_comms`; then
|
||||
per fd target (with a trailing ` (deleted)` stripped), classify via
|
||||
`_sensitive_kind`, first hit emits then `break`. Classification precedence:
|
||||
1. exact file in `{/etc/shadow, /etc/gshadow, /etc/security/opasswd}` →
|
||||
"system credential database";
|
||||
2. matches `cfg.credential_access_extra_paths` (entry ending `/` = prefix;
|
||||
else exact or `entry/` prefix) → "configured credential path";
|
||||
3. path contains `/.ssh/` and basename starts `id_` or ends `.pem`/`.key` →
|
||||
"private SSH key";
|
||||
4. path under `/etc/NetworkManager/system-connections/` → "NetworkManager
|
||||
secret profile";
|
||||
5. basename in `{logins.json, key4.db, "Login Data", Cookies}` **and** path
|
||||
(lowercased) contains a browser marker (`/.mozilla/`, `/firefox/`,
|
||||
`/chromium/`, `/google-chrome/`, `/chrome/`, `/brave`, `/vivaldi/`,
|
||||
`/edge/`) → "browser credential store".
|
||||
|
||||
Non-absolute paths are never sensitive.
|
||||
- Emit: key `cred:<pid>:<target>`, `pids=(pid,)`, detail
|
||||
`pid=… comm=… fd=… has <kind> open: <target>`.
|
||||
- Config: `credential_access_allow_comms`, `credential_access_extra_paths`.
|
||||
|
||||
## 2.8 `stealth_network` — SID 100036, MEDIUM or HIGH
|
||||
|
||||
Sockets in families attackers use to slide around TCP/UDP checks.
|
||||
|
||||
- Watched kinds: `{raw, sctp, dccp, packet, mptcp, tipc, xdp, vsock}` (matched
|
||||
on `sock.kind.lower()`).
|
||||
- Skip if: kind in `cfg.stealth_network_allow_kinds`; or `sock.comm` in
|
||||
`cfg.stealth_network_allow_comms`; or `sock.state` is non-empty and **not** in
|
||||
`{ESTAB, LISTEN, UNCONN, CONNECTED, SYN-SENT, SYN-RECV}`.
|
||||
- Severity: HIGH if kind in `{raw, packet, xdp}`; else HIGH if `state == LISTEN`
|
||||
or the peer is a public IP outside the egress trust list; else MEDIUM.
|
||||
- Emit: key `stealthnet:<kind>:<pid>:<local>:<peer>`, `pids=(pid,)` only when
|
||||
pid is known, detail `<kind> socket state=… local=… peer=… comm=<comm or ?>
|
||||
pid=<pid or ?>`.
|
||||
|
||||
## 2.9 `egress` — SID 100016, HIGH
|
||||
|
||||
Interpreter holding an outbound connection to a public IP.
|
||||
|
||||
- Match, per `established_sockets()`: `s.comm in cfg.interpreters`; peer host is
|
||||
public (`is_public_ip`); peer not in `cfg.egress_allow_cidrs`.
|
||||
- Emit: key `egr:<pid>:<host>`, `pids=(pid,)` when pid known, detail
|
||||
`pid=… comm=… -> <host>:<port> (interpreter to public IP)`.
|
||||
|
||||
## 2.10 `memory_obfuscation` — SIDs 100039 / 100048 / 100049
|
||||
|
||||
Inspects `/proc/<pid>/maps` shapes (never reads process memory). Per process,
|
||||
each map is classified by `_indicator`; **at most one alert per signature per
|
||||
process** (a per-process `seen` set). Precedence within a single map:
|
||||
|
||||
1. If `path` matches `cfg.memory_obfuscation_allow_paths` (prefix) → ignore.
|
||||
2. `path` basename contains a hide hint (`libhide`, `libprocesshide`,
|
||||
`process_hide`, `proc_hide`, `rootkit_hide`) → **`process_hiding_library`**,
|
||||
SID 100048, CRITICAL, classtype `process-hiding`.
|
||||
3. executable **and** library from a writable/runtime path (`/tmp/`, `/dev/shm/`,
|
||||
`/var/tmp/`, `/run/user/` with a `.so`/`.so.`/`.dylib` in the name) →
|
||||
**`process_injection_library`**, SID 100049, HIGH, classtype
|
||||
`process-injection`.
|
||||
4. executable **and** path contains `memfd:` or `(deleted)` →
|
||||
**`memory_obfuscation`**, SID 100039, CRITICAL.
|
||||
5. executable **and** writable (RWX) → `memory_obfuscation`, HIGH.
|
||||
6. executable **and** anonymous (`""`, `[heap]`, `[stack]`, `[anon…`,
|
||||
`[stack:…`) → `memory_obfuscation`, HIGH.
|
||||
|
||||
The `memory_obfuscation` signature (only) is additionally suppressed when
|
||||
`comm in cfg.memory_obfuscation_allow_comms` (JITs: java, node, browsers,
|
||||
dotnet, qemu, wine, wasmtime). The library signatures are **not** suppressed by
|
||||
that allow-list.
|
||||
|
||||
- Emit: key `mem:<signature>:<pid>:<lo>-<hi>`, `pids=(pid,)`, detail
|
||||
`pid=… comm=… <indicator detail> range=<lo>-<hi>`.
|
||||
|
||||
## 2.11 `new_listener` — SID 100013, HIGH
|
||||
|
||||
New listening socket absent from the startup baseline (see §1.5.1). No-op when
|
||||
`state.listener_baseline is None` (unarmed).
|
||||
|
||||
- Match, per `LISTEN` socket: identity `key = port/comm` where
|
||||
`port = local.rpartition(":")[2]` and `comm` defaults `?`. Skip if key in
|
||||
`state.listener_baseline`, or `port in cfg.listener_allow_ports`, or
|
||||
`comm in cfg.listener_allow_comms`.
|
||||
- Optional provenance gate: if `cfg.suppress_package_owned_listeners` and the
|
||||
owning process's `exe` `is_package_owned`, skip.
|
||||
- Emit: key `lis:<port>/<comm>`, `pids=(pid,)` when known, detail
|
||||
`new listening socket <local> by comm=<comm>`.
|
||||
|
||||
## 2.12 `new_suid` — SID 100014, HIGH or CRITICAL
|
||||
|
||||
SUID/SGID binary not in the baseline. No-op when `state.suid_binaries is None`
|
||||
**or** `state.suid_baseline is None` (see §1.5.3 — no alerts until an async scan
|
||||
result exists).
|
||||
|
||||
- Match, per path in `state.suid_binaries` not in `state.suid_baseline`:
|
||||
`hot = any(path.startswith(d.rstrip("/") + "/") for d in cfg.suid_hot_dirs)`.
|
||||
- Severity: CRITICAL if hot (writable dir), else HIGH.
|
||||
- Emit: key `suid:<path>`, no pids, detail
|
||||
`SUID/SGID binary in writable dir: <path>` (hot) or `new SUID/SGID binary:
|
||||
<path>`.
|
||||
|
||||
## 2.13 `persistence` — SID 100015, HIGH
|
||||
|
||||
Watched persistence-relevant file modified since the previous sweep. No-op when
|
||||
`state.persist_since is None` (unarmed).
|
||||
|
||||
- Input: walk `cfg.watch_persistence` (a dir → recurse `os.walk`, a file →
|
||||
itself). For each existing file, `mtime = os.lstat(path).st_mtime`
|
||||
(unreadable → skip).
|
||||
- Match: `mtime > state.persist_since` (the *previous* sweep's timestamp; §1.5.4).
|
||||
- Emit: key `persist:<path>`, no pids, detail `persistence file modified:
|
||||
<path>`.
|
||||
|
||||
## 2.14 `first_seen` — SIDs 100074 / 100075, MEDIUM
|
||||
|
||||
Network-rarity tracking with a self-owned persistent store. The store format,
|
||||
the `initialized` learn-once semantics, and the restart asymmetry are specified
|
||||
in §1.5.5 — not repeated here. Match summary:
|
||||
|
||||
- `first_public_destination` (100074): a `comm` reaches an `ESTAB` peer that is
|
||||
a public IP:port not previously recorded for it. Key
|
||||
`firstdest:<comm>:<ip:port>`, `pids=(pid,)` when known.
|
||||
- `first_listener_port` (100075): a `comm` opens a `LISTEN`/`UNCONN` tcp/udp
|
||||
socket on a numeric non-zero port not previously recorded for it. Key
|
||||
`firstlisten:<comm>:<port>`.
|
||||
|
||||
Both only alert when the store is already `initialized`; the recording of the
|
||||
new value happens regardless (so the alert fires once).
|
||||
|
||||
---
|
||||
|
||||
## 2.15 Port parity notes (Go port)
|
||||
|
||||
- The Go poll detectors for §2.3–§2.14 are **ported and pass the parity
|
||||
fixture** (`scripts/check-go-parity.py`), covering exact SID/severity/key/
|
||||
classtype/detail output. This document is the human-readable form of that
|
||||
fixture's contract.
|
||||
- `first_seen`/`new_listener`/`new_suid`/`persistence` receive injected baseline
|
||||
inputs in parity mode and use the opt-in live lifecycle described in §1 when
|
||||
`--state-dir` is supplied.
|
||||
- Watch two easily-missed rules when reviewing a port: the **first-match-then-
|
||||
break** behavior in `input_snooper`/`credential_access` (one alert per
|
||||
process), and the **per-signature `seen` dedup** in `memory_obfuscation` (one
|
||||
alert per signature per process, precedence-ordered).
|
||||
- `ld_preload`'s global check reads `/etc/ld.so.preload` **inside the detector**
|
||||
(or from an injected state field), not from the shared `SystemState` process
|
||||
view.
|
||||
Loading…
Add table
Add a link
Reference in a new issue