feat(go): chain retained snapshot artifacts

This commit is contained in:
Luna 2026-07-22 02:20:46 -07:00
parent fd2bbba0d7
commit 1ab4add205
No known key found for this signature in database
7 changed files with 278 additions and 16 deletions

View file

@ -1,8 +1,8 @@
# Go Port Handoff
Saved: 2026-07-22T02:03:00-07:00
Saved: 2026-07-22T02:10:00-07:00
Branch: `main`
Base commit: `6d5a897` (`feat(go): persist Python-compatible incidents`)
Base commit: `fd2bbba` (`feat(go): add bounded hot-path enrichment`)
Status: implemented and green, but uncommitted
## Worktree warning
@ -10,8 +10,9 @@ Status: implemented and green, but uncommitted
The checkout is intentionally dirty and contains work from multiple related
continuations. Do not reset, clean, or broadly restage it.
- The validation-sidecar and incident-persistence tranches are committed as
`f85c2e8` and `6d5a897`. The bounded-enrichment slice is uncommitted.
- The validation-sidecar, incident-persistence, and bounded-enrichment tranches
are signed commits `f85c2e8`, `538d1d9`, and `fd2bbba`. The local-assurance
slice is uncommitted.
- At this checkpoint, `git status --short` has 20 entries with untracked
directories collapsed.
- The Python GUI files and tests are separate pre-existing work. Preserve them
@ -94,8 +95,12 @@ static binary.
context, lineage, remote-IP classification, and candidate paths entirely from
already-captured records. Alert/process/indicator caps bound attacker-
controlled work on the emission path.
- Package ownership, executable hashes, file metadata, integrity anchors,
assurance chaining, and notification fan-out are not yet ported.
- Snapshot log/JSON revisions now append synchronized private
`enodia.hash_chain.v1` records. Artifact SHA-256 is streamed, previous-hash
lookup reads a bounded chain tail, and append completion is synced before the
alert event is emitted.
- Package ownership, executable enrichment hashes, file metadata, richer
integrity anchors, and notification fan-out are not yet ported.
- No live system service was installed or enabled during development.
## Last green verification
@ -140,9 +145,9 @@ the suite still exits successfully.
## Resume here
1. Move slow enrichment collectors (ownership, bounded hashing, file metadata,
integrity anchors) behind a bounded asynchronous worker, then add local
assurance chaining without destructive response behavior.
1. Move slow enrichment collectors (ownership, bounded executable hashing, file
metadata, integrity anchors) behind a bounded asynchronous worker without
destructive response behavior.
2. Connect the isolated Go snapshot/event state to explicit read-only
management consumers while keeping Python authoritative.
3. Continue broader Phase 3 rule metadata/state parity before considering any

View file

@ -96,8 +96,9 @@ sidecar also writes a private atomic `incidents.json` with the required
correlation evidence. Hot-path Go enrichment is bounded and no-I/O: process and
parent context, lineage, remote classification, and candidate paths come from
already-captured records. Python remains authoritative for ownership/hashes,
file and integrity metadata, assurance chaining, notifications, and management
consumers.
file and richer integrity metadata, notifications, and management consumers.
Each snapshot text/JSON revision is SHA-256 hashed into a synchronized private
`hash-chain.jsonl` using the stable `enodia.hash_chain.v1` record schema.
Each successful sweep atomically refreshes
`/var/lib/enodia-sentinel-go/heartbeat` as a Unix timestamp with mode `0600`;

View file

@ -52,8 +52,8 @@ Current scope:
- a shared fixture checked against the Python detector by
`scripts/check-go-parity.py`.
It does not yet provide Python's rich enrichment, assurance chain, notification
fan-out, or management consumers for those snapshots, and it does not replace
It does not yet provide Python's full enrichment, notification fan-out, or
management consumers for those snapshots, and it does not replace
`enodia-sentinel`. The module requires Go 1.25 or newer; development is currently
verified with Go 1.26.5. Run a single terminal-visible stateless sweep:
@ -102,7 +102,8 @@ a probe load failure is visible in status events and the polling loop remains
active. Installing the sidecar does not enable it or alter
`enodia-sentinel.service`. The unit reaches `active` only after baseline
initialization, event-log and snapshot-store preflight, and both requested probe
load attempts finish.
load attempts finish. Snapshot-store preflight includes the private assurance
chain.
Alert envelopes are also batched into private
`alert-YYYYMMDD-HHMMSS.{json,log}` pairs under the same isolated directory.
@ -115,8 +116,9 @@ the configured `incident_tracking`, `incident_window`, and
and one snapshot name idempotently. Hot-path enrichment derives bounded
process/parent context, lineage, remote-IP classification, and candidate paths
from already-captured data without new I/O. Package ownership, hashes, file
metadata, integrity anchors, assurance chaining, and notifications remain
future parity work.
metadata, richer integrity anchors, and notifications remain future parity
work. Snapshot text/JSON revisions append synchronized `enodia.hash_chain.v1`
records to a private local `hash-chain.jsonl`.
After every successfully emitted status record, the service atomically updates
`/var/lib/enodia-sentinel-go/heartbeat` with the current Unix timestamp (mode

View file

@ -0,0 +1,179 @@
// SPDX-License-Identifier: GPL-3.0-or-later
// Package assurance appends Python-schema-compatible hash-chain records for
// local forensic artifacts. Artifact hashing streams from disk, and previous-
// record lookup reads only a bounded tail of the chain.
package assurance
import (
"bufio"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"sync"
"time"
)
const (
Schema = "enodia.hash_chain.v1"
ChainName = "hash-chain.jsonl"
maxTailBytes = 1 << 20
)
type Record struct {
Schema string `json:"schema"`
Time float64 `json:"time"`
Kind string `json:"kind"`
Path string `json:"path"`
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
PrevHash string `json:"prev_hash"`
ChainHash string `json:"chain_hash"`
}
type Chain struct {
Path string
mu sync.Mutex
}
// New preflights the private append-only chain file before service readiness.
func New(directory string) (*Chain, error) {
if directory == "" {
return nil, fmt.Errorf("assurance directory is required")
}
if err := os.MkdirAll(directory, 0o750); err != nil {
return nil, fmt.Errorf("create assurance directory: %w", err)
}
path := filepath.Join(directory, ChainName)
file, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o600)
if err != nil {
return nil, fmt.Errorf("preflight assurance chain: %w", err)
}
if err := file.Close(); err != nil {
return nil, fmt.Errorf("close assurance chain: %w", err)
}
if err := os.Chmod(path, 0o600); err != nil {
return nil, fmt.Errorf("protect assurance chain: %w", err)
}
return &Chain{Path: path}, nil
}
// Append hashes artifact and durably appends its chained evidence record.
func (c *Chain) Append(kind, artifact string, when time.Time) (Record, error) {
c.mu.Lock()
defer c.mu.Unlock()
digest, size, err := fileDigest(artifact)
if err != nil {
// Python records unavailable artifacts with empty evidence rather than
// dropping the chain entry.
digest, size = "", 0
}
previous, err := lastHash(c.Path)
if err != nil {
return Record{}, err
}
record := Record{
Schema: Schema, Time: float64(when.UnixNano()) / 1e9, Kind: kind,
Path: artifact, Size: size, SHA256: digest, PrevHash: previous,
}
payload, err := canonicalPayload(record)
if err != nil {
return Record{}, err
}
sum := sha256.Sum256(append(payload, []byte(previous)...))
record.ChainHash = hex.EncodeToString(sum[:])
raw, err := json.Marshal(record)
if err != nil {
return Record{}, fmt.Errorf("encode assurance record: %w", err)
}
file, err := os.OpenFile(c.Path, os.O_APPEND|os.O_WRONLY, 0o600)
if err != nil {
return Record{}, fmt.Errorf("open assurance chain: %w", err)
}
defer file.Close()
if _, err := file.Write(append(raw, '\n')); err != nil {
return Record{}, fmt.Errorf("append assurance chain: %w", err)
}
if err := file.Sync(); err != nil {
return Record{}, fmt.Errorf("sync assurance chain: %w", err)
}
return record, nil
}
func fileDigest(path string) (string, int64, error) {
file, err := os.Open(path)
if err != nil {
return "", 0, err
}
defer file.Close()
hash := sha256.New()
size, err := io.Copy(hash, file)
if err != nil {
return "", 0, err
}
return hex.EncodeToString(hash.Sum(nil)), size, nil
}
func lastHash(path string) (string, error) {
file, err := os.Open(path)
if err != nil {
return "", fmt.Errorf("open assurance tail: %w", err)
}
defer file.Close()
info, err := file.Stat()
if err != nil {
return "", fmt.Errorf("stat assurance chain: %w", err)
}
offset := info.Size() - maxTailBytes
if offset < 0 {
offset = 0
}
if _, err := file.Seek(offset, io.SeekStart); err != nil {
return "", fmt.Errorf("seek assurance tail: %w", err)
}
raw, err := io.ReadAll(io.LimitReader(file, maxTailBytes))
if err != nil {
return "", fmt.Errorf("read assurance tail: %w", err)
}
lines := strings.Split(strings.TrimSpace(string(raw)), "\n")
for index := len(lines) - 1; index >= 0; index-- {
var record Record
if json.Unmarshal([]byte(lines[index]), &record) == nil && record.ChainHash != "" {
return record.ChainHash, nil
}
}
return "", nil
}
func canonicalPayload(record Record) ([]byte, error) {
// encoding/json sorts string map keys, matching Python's sort_keys=True.
return json.Marshal(map[string]any{
"schema": record.Schema, "time": record.Time, "kind": record.Kind,
"path": record.Path, "size": record.Size, "sha256": record.SHA256,
"prev_hash": record.PrevHash,
})
}
// ReadAll decodes a chain for verification and read-only consumers.
func ReadAll(path string) ([]Record, error) {
file, err := os.Open(path)
if err != nil {
return nil, err
}
defer file.Close()
var records []Record
scanner := bufio.NewScanner(file)
for scanner.Scan() {
var record Record
if err := json.Unmarshal(scanner.Bytes(), &record); err != nil {
return nil, err
}
records = append(records, record)
}
return records, scanner.Err()
}

View file

@ -0,0 +1,55 @@
// SPDX-License-Identifier: GPL-3.0-or-later
package assurance
import (
"os"
"path/filepath"
"testing"
"time"
)
func TestAppendBuildsDurableLinkedRecords(t *testing.T) {
directory := t.TempDir()
chain, err := New(directory)
if err != nil {
t.Fatal(err)
}
firstPath := filepath.Join(directory, "first.json")
secondPath := filepath.Join(directory, "second.log")
os.WriteFile(firstPath, []byte("first"), 0o600)
os.WriteFile(secondPath, []byte("second"), 0o600)
first, err := chain.Append("snapshot-json", firstPath, time.Unix(1000, 0))
if err != nil {
t.Fatal(err)
}
second, err := chain.Append("snapshot-log", secondPath, time.Unix(1001, 0))
if err != nil {
t.Fatal(err)
}
if first.ChainHash == "" || second.PrevHash != first.ChainHash || second.ChainHash == first.ChainHash {
t.Fatalf("first=%#v second=%#v", first, second)
}
records, err := ReadAll(chain.Path)
if err != nil || len(records) != 2 {
t.Fatalf("records=%#v err=%v", records, err)
}
info, err := os.Stat(chain.Path)
if err != nil || info.Mode().Perm() != 0o600 {
t.Fatalf("chain mode info=%v err=%v", info, err)
}
}
func TestAppendUnavailableArtifactMatchesPythonFailOpenRecord(t *testing.T) {
chain, err := New(t.TempDir())
if err != nil {
t.Fatal(err)
}
record, err := chain.Append("missing", "/definitely/missing", time.Unix(1, 0))
if err != nil {
t.Fatal(err)
}
if record.Size != 0 || record.SHA256 != "" || record.ChainHash == "" {
t.Fatalf("record=%#v", record)
}
}

View file

@ -16,6 +16,7 @@ import (
"sync"
"time"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/assurance"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/correlation"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/enrich"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/incident"
@ -64,6 +65,7 @@ type Store struct {
ParentOf func(int) int
Incidents *incident.Store
LineageDepth int
Assurance *assurance.Chain
mu sync.Mutex
}
@ -102,6 +104,10 @@ func New(dir, procRoot string, maxCount, maxAgeDays int) (*Store, error) {
store := &Store{Dir: dir, ProcRoot: procRoot, MaxCount: maxCount, MaxAgeDays: maxAgeDays, Now: time.Now, LineageDepth: 8}
store.Collect = func(pid int) ProcessDetail { return collectProcess(procRoot, pid) }
store.ParentOf = func(pid int) int { return readParent(procRoot, pid) }
store.Assurance, err = assurance.New(dir)
if err != nil {
return nil, err
}
if err := store.Prune(); err != nil {
return nil, err
}
@ -183,6 +189,14 @@ func (s *Store) CaptureEvent(event map[string]any) (string, error) {
if err := writeAtomic(base+".log", []byte(formatText(report))); err != nil {
return "", err
}
if s.Assurance != nil {
if _, err := s.Assurance.Append("snapshot-log", base+".log", captured); err != nil {
return "", err
}
if _, err := s.Assurance.Append("snapshot-json", jsonPath, captured); err != nil {
return "", err
}
}
if err := s.pruneLocked(); err != nil {
return "", err
}

View file

@ -10,6 +10,7 @@ import (
"testing"
"time"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/assurance"
"codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/model"
)
@ -61,6 +62,11 @@ func TestCaptureEventWritesCompatiblePairAndMergesSameSecond(t *testing.T) {
t.Fatalf("%s mode=%#o", path, info.Mode().Perm())
}
}
records, err := assurance.ReadAll(filepath.Join(store.Dir, assurance.ChainName))
if err != nil || len(records) != 4 || records[0].Kind != "snapshot-log" ||
records[1].Kind != "snapshot-json" || records[1].PrevHash != records[0].ChainHash {
t.Fatalf("assurance records=%#v err=%v", records, err)
}
stats, err := store.SnapshotStats()
if err != nil {
t.Fatal(err)