feat(go): chain retained snapshot artifacts

This commit is contained in:
Luna 2026-07-22 02:20:46 -07:00
parent fd2bbba0d7
commit 1ab4add205
No known key found for this signature in database
7 changed files with 278 additions and 16 deletions

View file

@ -1,8 +1,8 @@
# Go Port Handoff # Go Port Handoff
Saved: 2026-07-22T02:03:00-07:00 Saved: 2026-07-22T02:10:00-07:00
Branch: `main` Branch: `main`
Base commit: `6d5a897` (`feat(go): persist Python-compatible incidents`) Base commit: `fd2bbba` (`feat(go): add bounded hot-path enrichment`)
Status: implemented and green, but uncommitted Status: implemented and green, but uncommitted
## Worktree warning ## Worktree warning
@ -10,8 +10,9 @@ Status: implemented and green, but uncommitted
The checkout is intentionally dirty and contains work from multiple related The checkout is intentionally dirty and contains work from multiple related
continuations. Do not reset, clean, or broadly restage it. continuations. Do not reset, clean, or broadly restage it.
- The validation-sidecar and incident-persistence tranches are committed as - The validation-sidecar, incident-persistence, and bounded-enrichment tranches
`f85c2e8` and `6d5a897`. The bounded-enrichment slice is uncommitted. are signed commits `f85c2e8`, `538d1d9`, and `fd2bbba`. The local-assurance
slice is uncommitted.
- At this checkpoint, `git status --short` has 20 entries with untracked - At this checkpoint, `git status --short` has 20 entries with untracked
directories collapsed. directories collapsed.
- The Python GUI files and tests are separate pre-existing work. Preserve them - The Python GUI files and tests are separate pre-existing work. Preserve them
@ -94,8 +95,12 @@ static binary.
context, lineage, remote-IP classification, and candidate paths entirely from context, lineage, remote-IP classification, and candidate paths entirely from
already-captured records. Alert/process/indicator caps bound attacker- already-captured records. Alert/process/indicator caps bound attacker-
controlled work on the emission path. controlled work on the emission path.
- Package ownership, executable hashes, file metadata, integrity anchors, - Snapshot log/JSON revisions now append synchronized private
assurance chaining, and notification fan-out are not yet ported. `enodia.hash_chain.v1` records. Artifact SHA-256 is streamed, previous-hash
lookup reads a bounded chain tail, and append completion is synced before the
alert event is emitted.
- Package ownership, executable enrichment hashes, file metadata, richer
integrity anchors, and notification fan-out are not yet ported.
- No live system service was installed or enabled during development. - No live system service was installed or enabled during development.
## Last green verification ## Last green verification
@ -140,9 +145,9 @@ the suite still exits successfully.
## Resume here ## Resume here
1. Move slow enrichment collectors (ownership, bounded hashing, file metadata, 1. Move slow enrichment collectors (ownership, bounded executable hashing, file
integrity anchors) behind a bounded asynchronous worker, then add local metadata, integrity anchors) behind a bounded asynchronous worker without
assurance chaining without destructive response behavior. destructive response behavior.
2. Connect the isolated Go snapshot/event state to explicit read-only 2. Connect the isolated Go snapshot/event state to explicit read-only
management consumers while keeping Python authoritative. management consumers while keeping Python authoritative.
3. Continue broader Phase 3 rule metadata/state parity before considering any 3. Continue broader Phase 3 rule metadata/state parity before considering any

View file

@ -96,8 +96,9 @@ sidecar also writes a private atomic `incidents.json` with the required
correlation evidence. Hot-path Go enrichment is bounded and no-I/O: process and correlation evidence. Hot-path Go enrichment is bounded and no-I/O: process and
parent context, lineage, remote classification, and candidate paths come from parent context, lineage, remote classification, and candidate paths come from
already-captured records. Python remains authoritative for ownership/hashes, already-captured records. Python remains authoritative for ownership/hashes,
file and integrity metadata, assurance chaining, notifications, and management file and richer integrity metadata, notifications, and management consumers.
consumers. Each snapshot text/JSON revision is SHA-256 hashed into a synchronized private
`hash-chain.jsonl` using the stable `enodia.hash_chain.v1` record schema.
Each successful sweep atomically refreshes Each successful sweep atomically refreshes
`/var/lib/enodia-sentinel-go/heartbeat` as a Unix timestamp with mode `0600`; `/var/lib/enodia-sentinel-go/heartbeat` as a Unix timestamp with mode `0600`;

View file

@ -52,8 +52,8 @@ Current scope:
- a shared fixture checked against the Python detector by - a shared fixture checked against the Python detector by
`scripts/check-go-parity.py`. `scripts/check-go-parity.py`.
It does not yet provide Python's rich enrichment, assurance chain, notification It does not yet provide Python's full enrichment, notification fan-out, or
fan-out, or management consumers for those snapshots, and it does not replace management consumers for those snapshots, and it does not replace
`enodia-sentinel`. The module requires Go 1.25 or newer; development is currently `enodia-sentinel`. The module requires Go 1.25 or newer; development is currently
verified with Go 1.26.5. Run a single terminal-visible stateless sweep: verified with Go 1.26.5. Run a single terminal-visible stateless sweep:
@ -102,7 +102,8 @@ a probe load failure is visible in status events and the polling loop remains
active. Installing the sidecar does not enable it or alter active. Installing the sidecar does not enable it or alter
`enodia-sentinel.service`. The unit reaches `active` only after baseline `enodia-sentinel.service`. The unit reaches `active` only after baseline
initialization, event-log and snapshot-store preflight, and both requested probe initialization, event-log and snapshot-store preflight, and both requested probe
load attempts finish. load attempts finish. Snapshot-store preflight includes the private assurance
chain.
Alert envelopes are also batched into private Alert envelopes are also batched into private
`alert-YYYYMMDD-HHMMSS.{json,log}` pairs under the same isolated directory. `alert-YYYYMMDD-HHMMSS.{json,log}` pairs under the same isolated directory.
@ -115,8 +116,9 @@ the configured `incident_tracking`, `incident_window`, and
and one snapshot name idempotently. Hot-path enrichment derives bounded and one snapshot name idempotently. Hot-path enrichment derives bounded
process/parent context, lineage, remote-IP classification, and candidate paths process/parent context, lineage, remote-IP classification, and candidate paths
from already-captured data without new I/O. Package ownership, hashes, file from already-captured data without new I/O. Package ownership, hashes, file
metadata, integrity anchors, assurance chaining, and notifications remain metadata, richer integrity anchors, and notifications remain future parity
future parity work. work. Snapshot text/JSON revisions append synchronized `enodia.hash_chain.v1`
records to a private local `hash-chain.jsonl`.
After every successfully emitted status record, the service atomically updates After every successfully emitted status record, the service atomically updates
`/var/lib/enodia-sentinel-go/heartbeat` with the current Unix timestamp (mode `/var/lib/enodia-sentinel-go/heartbeat` with the current Unix timestamp (mode

View file

@ -0,0 +1,179 @@
// SPDX-License-Identifier: GPL-3.0-or-later
// Package assurance appends Python-schema-compatible hash-chain records for
// local forensic artifacts. Artifact hashing streams from disk, and previous-
// record lookup reads only a bounded tail of the chain.
package assurance
import (
"bufio"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"sync"
"time"
)
const (
Schema = "enodia.hash_chain.v1"
ChainName = "hash-chain.jsonl"
maxTailBytes = 1 << 20
)
type Record struct {
Schema string `json:"schema"`
Time float64 `json:"time"`
Kind string `json:"kind"`
Path string `json:"path"`
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
PrevHash string `json:"prev_hash"`
ChainHash string `json:"chain_hash"`
}
type Chain struct {
Path string
mu sync.Mutex
}
// New preflights the private append-only chain file before service readiness.
func New(directory string) (*Chain, error) {
if directory == "" {
return nil, fmt.Errorf("assurance directory is required")
}
if err := os.MkdirAll(directory, 0o750); err != nil {
return nil, fmt.Errorf("create assurance directory: %w", err)
}
path := filepath.Join(directory, ChainName)
file, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o600)
if err != nil {
return nil, fmt.Errorf("preflight assurance chain: %w", err)
}
if err := file.Close(); err != nil {
return nil, fmt.Errorf("close assurance chain: %w", err)
}
if err := os.Chmod(path, 0o600); err != nil {
return nil, fmt.Errorf("protect assurance chain: %w", err)
}
return &Chain{Path: path}, nil
}
// Append hashes artifact and durably appends its chained evidence record.
func (c *Chain) Append(kind, artifact string, when time.Time) (Record, error) {
c.mu.Lock()
defer c.mu.Unlock()
digest, size, err := fileDigest(artifact)
if err != nil {
// Python records unavailable artifacts with empty evidence rather than
// dropping the chain entry.
digest, size = "", 0
}
previous, err := lastHash(c.Path)
if err != nil {
return Record{}, err
}
record := Record{
Schema: Schema, Time: float64(when.UnixNano()) / 1e9, Kind: kind,
Path: artifact, Size: size, SHA256: digest, PrevHash: previous,
}
payload, err := canonicalPayload(record)
if err != nil {
return Record{}, err
}
sum := sha256.Sum256(append(payload, []byte(previous)...))
record.ChainHash = hex.EncodeToString(sum[:])
raw, err := json.Marshal(record)
if err != nil {
return Record{}, fmt.Errorf("encode assurance record: %w", err)
}
file, err := os.OpenFile(c.Path, os.O_APPEND|os.O_WRONLY, 0o600)
if err != nil {
return Record{}, fmt.Errorf("open assurance chain: %w", err)
}
defer file.Close()
if _, err := file.Write(append(raw, '\n')); err != nil {
return Record{}, fmt.Errorf("append assurance chain: %w", err)
}
if err := file.Sync(); err != nil {
return Record{}, fmt.Errorf("sync assurance chain: %w", err)
}
return record, nil
}
func fileDigest(path string) (string, int64, error) {
file, err := os.Open(path)
if err != nil {
return "", 0, err
}
defer file.Close()
hash := sha256.New()
size, err := io.Copy(hash, file)
if err != nil {
return "", 0, err
}
return hex.EncodeToString(hash.Sum(nil)), size, nil
}
func lastHash(path string) (string, error) {
file, err := os.Open(path)
if err != nil {
return "", fmt.Errorf("open assurance tail: %w", err)
}
defer file.Close()
info, err := file.Stat()
if err != nil {
return "", fmt.Errorf("stat assurance chain: %w", err)
}
offset := info.Size() - maxTailBytes
if offset < 0 {
offset = 0
}
if _, err := file.Seek(offset, io.SeekStart); err != nil {
return "", fmt.Errorf("seek assurance tail: %w", err)
}
raw, err := io.ReadAll(io.LimitReader(file, maxTailBytes))
if err != nil {
return "", fmt.Errorf("read assurance tail: %w", err)
}
lines := strings.Split(strings.TrimSpace(string(raw)), "\n")
for index := len(lines) - 1; index >= 0; index-- {
var record Record
if json.Unmarshal([]byte(lines[index]), &record) == nil && record.ChainHash != "" {
return record.ChainHash, nil
}
}
return "", nil
}
func canonicalPayload(record Record) ([]byte, error) {
// encoding/json sorts string map keys, matching Python's sort_keys=True.
return json.Marshal(map[string]any{
"schema": record.Schema, "time": record.Time, "kind": record.Kind,
"path": record.Path, "size": record.Size, "sha256": record.SHA256,
"prev_hash": record.PrevHash,
})
}
// ReadAll decodes a chain for verification and read-only consumers.
func ReadAll(path string) ([]Record, error) {
file, err := os.Open(path)
if err != nil {
return nil, err
}
defer file.Close()
var records []Record
scanner := bufio.NewScanner(file)
for scanner.Scan() {
var record Record
if err := json.Unmarshal(scanner.Bytes(), &record); err != nil {
return nil, err
}
records = append(records, record)
}
return records, scanner.Err()
}

View file

@ -0,0 +1,55 @@
// SPDX-License-Identifier: GPL-3.0-or-later
package assurance
import (
"os"
"path/filepath"
"testing"
"time"
)
func TestAppendBuildsDurableLinkedRecords(t *testing.T) {
directory := t.TempDir()
chain, err := New(directory)
if err != nil {
t.Fatal(err)
}
firstPath := filepath.Join(directory, "first.json")
secondPath := filepath.Join(directory, "second.log")
os.WriteFile(firstPath, []byte("first"), 0o600)
os.WriteFile(secondPath, []byte("second"), 0o600)
first, err := chain.Append("snapshot-json", firstPath, time.Unix(1000, 0))
if err != nil {
t.Fatal(err)
}
second, err := chain.Append("snapshot-log", secondPath, time.Unix(1001, 0))
if err != nil {
t.Fatal(err)
}
if first.ChainHash == "" || second.PrevHash != first.ChainHash || second.ChainHash == first.ChainHash {
t.Fatalf("first=%#v second=%#v", first, second)
}
records, err := ReadAll(chain.Path)
if err != nil || len(records) != 2 {
t.Fatalf("records=%#v err=%v", records, err)
}
info, err := os.Stat(chain.Path)
if err != nil || info.Mode().Perm() != 0o600 {
t.Fatalf("chain mode info=%v err=%v", info, err)
}
}
func TestAppendUnavailableArtifactMatchesPythonFailOpenRecord(t *testing.T) {
chain, err := New(t.TempDir())
if err != nil {
t.Fatal(err)
}
record, err := chain.Append("missing", "/definitely/missing", time.Unix(1, 0))
if err != nil {
t.Fatal(err)
}
if record.Size != 0 || record.SHA256 != "" || record.ChainHash == "" {
t.Fatalf("record=%#v", record)
}
}

View file

@ -16,6 +16,7 @@ import (
"sync" "sync"
"time" "time"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/assurance"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/correlation" "codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/correlation"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/enrich" "codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/enrich"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/incident" "codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/incident"
@ -64,6 +65,7 @@ type Store struct {
ParentOf func(int) int ParentOf func(int) int
Incidents *incident.Store Incidents *incident.Store
LineageDepth int LineageDepth int
Assurance *assurance.Chain
mu sync.Mutex mu sync.Mutex
} }
@ -102,6 +104,10 @@ func New(dir, procRoot string, maxCount, maxAgeDays int) (*Store, error) {
store := &Store{Dir: dir, ProcRoot: procRoot, MaxCount: maxCount, MaxAgeDays: maxAgeDays, Now: time.Now, LineageDepth: 8} store := &Store{Dir: dir, ProcRoot: procRoot, MaxCount: maxCount, MaxAgeDays: maxAgeDays, Now: time.Now, LineageDepth: 8}
store.Collect = func(pid int) ProcessDetail { return collectProcess(procRoot, pid) } store.Collect = func(pid int) ProcessDetail { return collectProcess(procRoot, pid) }
store.ParentOf = func(pid int) int { return readParent(procRoot, pid) } store.ParentOf = func(pid int) int { return readParent(procRoot, pid) }
store.Assurance, err = assurance.New(dir)
if err != nil {
return nil, err
}
if err := store.Prune(); err != nil { if err := store.Prune(); err != nil {
return nil, err return nil, err
} }
@ -183,6 +189,14 @@ func (s *Store) CaptureEvent(event map[string]any) (string, error) {
if err := writeAtomic(base+".log", []byte(formatText(report))); err != nil { if err := writeAtomic(base+".log", []byte(formatText(report))); err != nil {
return "", err return "", err
} }
if s.Assurance != nil {
if _, err := s.Assurance.Append("snapshot-log", base+".log", captured); err != nil {
return "", err
}
if _, err := s.Assurance.Append("snapshot-json", jsonPath, captured); err != nil {
return "", err
}
}
if err := s.pruneLocked(); err != nil { if err := s.pruneLocked(); err != nil {
return "", err return "", err
} }

View file

@ -10,6 +10,7 @@ import (
"testing" "testing"
"time" "time"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/assurance"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/model" "codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/model"
) )
@ -61,6 +62,11 @@ func TestCaptureEventWritesCompatiblePairAndMergesSameSecond(t *testing.T) {
t.Fatalf("%s mode=%#o", path, info.Mode().Perm()) t.Fatalf("%s mode=%#o", path, info.Mode().Perm())
} }
} }
records, err := assurance.ReadAll(filepath.Join(store.Dir, assurance.ChainName))
if err != nil || len(records) != 4 || records[0].Kind != "snapshot-log" ||
records[1].Kind != "snapshot-json" || records[1].PrevHash != records[0].ChainHash {
t.Fatalf("assurance records=%#v err=%v", records, err)
}
stats, err := store.SnapshotStats() stats, err := store.SnapshotStats()
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)