diff --git a/docs/GO_PORT_HANDOFF.md b/docs/GO_PORT_HANDOFF.md index 28dc648..71ad581 100644 --- a/docs/GO_PORT_HANDOFF.md +++ b/docs/GO_PORT_HANDOFF.md @@ -1,8 +1,8 @@ # Go Port Handoff -Saved: 2026-07-22T02:03:00-07:00 +Saved: 2026-07-22T02:10:00-07:00 Branch: `main` -Base commit: `6d5a897` (`feat(go): persist Python-compatible incidents`) +Base commit: `fd2bbba` (`feat(go): add bounded hot-path enrichment`) Status: implemented and green, but uncommitted ## Worktree warning @@ -10,8 +10,9 @@ Status: implemented and green, but uncommitted The checkout is intentionally dirty and contains work from multiple related continuations. Do not reset, clean, or broadly restage it. -- The validation-sidecar and incident-persistence tranches are committed as - `f85c2e8` and `6d5a897`. The bounded-enrichment slice is uncommitted. +- The validation-sidecar, incident-persistence, and bounded-enrichment tranches + are signed commits `f85c2e8`, `538d1d9`, and `fd2bbba`. The local-assurance + slice is uncommitted. - At this checkpoint, `git status --short` has 20 entries with untracked directories collapsed. - The Python GUI files and tests are separate pre-existing work. Preserve them @@ -94,8 +95,12 @@ static binary. context, lineage, remote-IP classification, and candidate paths entirely from already-captured records. Alert/process/indicator caps bound attacker- controlled work on the emission path. -- Package ownership, executable hashes, file metadata, integrity anchors, - assurance chaining, and notification fan-out are not yet ported. +- Snapshot log/JSON revisions now append synchronized private + `enodia.hash_chain.v1` records. Artifact SHA-256 is streamed, previous-hash + lookup reads a bounded chain tail, and append completion is synced before the + alert event is emitted. +- Package ownership, executable enrichment hashes, file metadata, richer + integrity anchors, and notification fan-out are not yet ported. - No live system service was installed or enabled during development. ## Last green verification @@ -140,9 +145,9 @@ the suite still exits successfully. ## Resume here -1. Move slow enrichment collectors (ownership, bounded hashing, file metadata, - integrity anchors) behind a bounded asynchronous worker, then add local - assurance chaining without destructive response behavior. +1. Move slow enrichment collectors (ownership, bounded executable hashing, file + metadata, integrity anchors) behind a bounded asynchronous worker without + destructive response behavior. 2. Connect the isolated Go snapshot/event state to explicit read-only management consumers while keeping Python authoritative. 3. Continue broader Phase 3 rule metadata/state parity before considering any diff --git a/docs/PACKAGING.md b/docs/PACKAGING.md index b16e941..6bed7a5 100644 --- a/docs/PACKAGING.md +++ b/docs/PACKAGING.md @@ -96,8 +96,9 @@ sidecar also writes a private atomic `incidents.json` with the required correlation evidence. Hot-path Go enrichment is bounded and no-I/O: process and parent context, lineage, remote classification, and candidate paths come from already-captured records. Python remains authoritative for ownership/hashes, -file and integrity metadata, assurance chaining, notifications, and management -consumers. +file and richer integrity metadata, notifications, and management consumers. +Each snapshot text/JSON revision is SHA-256 hashed into a synchronized private +`hash-chain.jsonl` using the stable `enodia.hash_chain.v1` record schema. Each successful sweep atomically refreshes `/var/lib/enodia-sentinel-go/heartbeat` as a Unix timestamp with mode `0600`; diff --git a/go-agent/README.md b/go-agent/README.md index 77a7b2b..665623b 100644 --- a/go-agent/README.md +++ b/go-agent/README.md @@ -52,8 +52,8 @@ Current scope: - a shared fixture checked against the Python detector by `scripts/check-go-parity.py`. -It does not yet provide Python's rich enrichment, assurance chain, notification -fan-out, or management consumers for those snapshots, and it does not replace +It does not yet provide Python's full enrichment, notification fan-out, or +management consumers for those snapshots, and it does not replace `enodia-sentinel`. The module requires Go 1.25 or newer; development is currently verified with Go 1.26.5. Run a single terminal-visible stateless sweep: @@ -102,7 +102,8 @@ a probe load failure is visible in status events and the polling loop remains active. Installing the sidecar does not enable it or alter `enodia-sentinel.service`. The unit reaches `active` only after baseline initialization, event-log and snapshot-store preflight, and both requested probe -load attempts finish. +load attempts finish. Snapshot-store preflight includes the private assurance +chain. Alert envelopes are also batched into private `alert-YYYYMMDD-HHMMSS.{json,log}` pairs under the same isolated directory. @@ -115,8 +116,9 @@ the configured `incident_tracking`, `incident_window`, and and one snapshot name idempotently. Hot-path enrichment derives bounded process/parent context, lineage, remote-IP classification, and candidate paths from already-captured data without new I/O. Package ownership, hashes, file -metadata, integrity anchors, assurance chaining, and notifications remain -future parity work. +metadata, richer integrity anchors, and notifications remain future parity +work. Snapshot text/JSON revisions append synchronized `enodia.hash_chain.v1` +records to a private local `hash-chain.jsonl`. After every successfully emitted status record, the service atomically updates `/var/lib/enodia-sentinel-go/heartbeat` with the current Unix timestamp (mode diff --git a/go-agent/internal/assurance/chain.go b/go-agent/internal/assurance/chain.go new file mode 100644 index 0000000..1da7e37 --- /dev/null +++ b/go-agent/internal/assurance/chain.go @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: GPL-3.0-or-later + +// Package assurance appends Python-schema-compatible hash-chain records for +// local forensic artifacts. Artifact hashing streams from disk, and previous- +// record lookup reads only a bounded tail of the chain. +package assurance + +import ( + "bufio" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "fmt" + "io" + "os" + "path/filepath" + "strings" + "sync" + "time" +) + +const ( + Schema = "enodia.hash_chain.v1" + ChainName = "hash-chain.jsonl" + maxTailBytes = 1 << 20 +) + +type Record struct { + Schema string `json:"schema"` + Time float64 `json:"time"` + Kind string `json:"kind"` + Path string `json:"path"` + Size int64 `json:"size"` + SHA256 string `json:"sha256"` + PrevHash string `json:"prev_hash"` + ChainHash string `json:"chain_hash"` +} + +type Chain struct { + Path string + mu sync.Mutex +} + +// New preflights the private append-only chain file before service readiness. +func New(directory string) (*Chain, error) { + if directory == "" { + return nil, fmt.Errorf("assurance directory is required") + } + if err := os.MkdirAll(directory, 0o750); err != nil { + return nil, fmt.Errorf("create assurance directory: %w", err) + } + path := filepath.Join(directory, ChainName) + file, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o600) + if err != nil { + return nil, fmt.Errorf("preflight assurance chain: %w", err) + } + if err := file.Close(); err != nil { + return nil, fmt.Errorf("close assurance chain: %w", err) + } + if err := os.Chmod(path, 0o600); err != nil { + return nil, fmt.Errorf("protect assurance chain: %w", err) + } + return &Chain{Path: path}, nil +} + +// Append hashes artifact and durably appends its chained evidence record. +func (c *Chain) Append(kind, artifact string, when time.Time) (Record, error) { + c.mu.Lock() + defer c.mu.Unlock() + digest, size, err := fileDigest(artifact) + if err != nil { + // Python records unavailable artifacts with empty evidence rather than + // dropping the chain entry. + digest, size = "", 0 + } + previous, err := lastHash(c.Path) + if err != nil { + return Record{}, err + } + record := Record{ + Schema: Schema, Time: float64(when.UnixNano()) / 1e9, Kind: kind, + Path: artifact, Size: size, SHA256: digest, PrevHash: previous, + } + payload, err := canonicalPayload(record) + if err != nil { + return Record{}, err + } + sum := sha256.Sum256(append(payload, []byte(previous)...)) + record.ChainHash = hex.EncodeToString(sum[:]) + raw, err := json.Marshal(record) + if err != nil { + return Record{}, fmt.Errorf("encode assurance record: %w", err) + } + file, err := os.OpenFile(c.Path, os.O_APPEND|os.O_WRONLY, 0o600) + if err != nil { + return Record{}, fmt.Errorf("open assurance chain: %w", err) + } + defer file.Close() + if _, err := file.Write(append(raw, '\n')); err != nil { + return Record{}, fmt.Errorf("append assurance chain: %w", err) + } + if err := file.Sync(); err != nil { + return Record{}, fmt.Errorf("sync assurance chain: %w", err) + } + return record, nil +} + +func fileDigest(path string) (string, int64, error) { + file, err := os.Open(path) + if err != nil { + return "", 0, err + } + defer file.Close() + hash := sha256.New() + size, err := io.Copy(hash, file) + if err != nil { + return "", 0, err + } + return hex.EncodeToString(hash.Sum(nil)), size, nil +} + +func lastHash(path string) (string, error) { + file, err := os.Open(path) + if err != nil { + return "", fmt.Errorf("open assurance tail: %w", err) + } + defer file.Close() + info, err := file.Stat() + if err != nil { + return "", fmt.Errorf("stat assurance chain: %w", err) + } + offset := info.Size() - maxTailBytes + if offset < 0 { + offset = 0 + } + if _, err := file.Seek(offset, io.SeekStart); err != nil { + return "", fmt.Errorf("seek assurance tail: %w", err) + } + raw, err := io.ReadAll(io.LimitReader(file, maxTailBytes)) + if err != nil { + return "", fmt.Errorf("read assurance tail: %w", err) + } + lines := strings.Split(strings.TrimSpace(string(raw)), "\n") + for index := len(lines) - 1; index >= 0; index-- { + var record Record + if json.Unmarshal([]byte(lines[index]), &record) == nil && record.ChainHash != "" { + return record.ChainHash, nil + } + } + return "", nil +} + +func canonicalPayload(record Record) ([]byte, error) { + // encoding/json sorts string map keys, matching Python's sort_keys=True. + return json.Marshal(map[string]any{ + "schema": record.Schema, "time": record.Time, "kind": record.Kind, + "path": record.Path, "size": record.Size, "sha256": record.SHA256, + "prev_hash": record.PrevHash, + }) +} + +// ReadAll decodes a chain for verification and read-only consumers. +func ReadAll(path string) ([]Record, error) { + file, err := os.Open(path) + if err != nil { + return nil, err + } + defer file.Close() + var records []Record + scanner := bufio.NewScanner(file) + for scanner.Scan() { + var record Record + if err := json.Unmarshal(scanner.Bytes(), &record); err != nil { + return nil, err + } + records = append(records, record) + } + return records, scanner.Err() +} diff --git a/go-agent/internal/assurance/chain_test.go b/go-agent/internal/assurance/chain_test.go new file mode 100644 index 0000000..6328816 --- /dev/null +++ b/go-agent/internal/assurance/chain_test.go @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-3.0-or-later + +package assurance + +import ( + "os" + "path/filepath" + "testing" + "time" +) + +func TestAppendBuildsDurableLinkedRecords(t *testing.T) { + directory := t.TempDir() + chain, err := New(directory) + if err != nil { + t.Fatal(err) + } + firstPath := filepath.Join(directory, "first.json") + secondPath := filepath.Join(directory, "second.log") + os.WriteFile(firstPath, []byte("first"), 0o600) + os.WriteFile(secondPath, []byte("second"), 0o600) + first, err := chain.Append("snapshot-json", firstPath, time.Unix(1000, 0)) + if err != nil { + t.Fatal(err) + } + second, err := chain.Append("snapshot-log", secondPath, time.Unix(1001, 0)) + if err != nil { + t.Fatal(err) + } + if first.ChainHash == "" || second.PrevHash != first.ChainHash || second.ChainHash == first.ChainHash { + t.Fatalf("first=%#v second=%#v", first, second) + } + records, err := ReadAll(chain.Path) + if err != nil || len(records) != 2 { + t.Fatalf("records=%#v err=%v", records, err) + } + info, err := os.Stat(chain.Path) + if err != nil || info.Mode().Perm() != 0o600 { + t.Fatalf("chain mode info=%v err=%v", info, err) + } +} + +func TestAppendUnavailableArtifactMatchesPythonFailOpenRecord(t *testing.T) { + chain, err := New(t.TempDir()) + if err != nil { + t.Fatal(err) + } + record, err := chain.Append("missing", "/definitely/missing", time.Unix(1, 0)) + if err != nil { + t.Fatal(err) + } + if record.Size != 0 || record.SHA256 != "" || record.ChainHash == "" { + t.Fatalf("record=%#v", record) + } +} diff --git a/go-agent/internal/snapshot/store.go b/go-agent/internal/snapshot/store.go index 530384f..92f3241 100644 --- a/go-agent/internal/snapshot/store.go +++ b/go-agent/internal/snapshot/store.go @@ -16,6 +16,7 @@ import ( "sync" "time" + "codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/assurance" "codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/correlation" "codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/enrich" "codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/incident" @@ -64,6 +65,7 @@ type Store struct { ParentOf func(int) int Incidents *incident.Store LineageDepth int + Assurance *assurance.Chain mu sync.Mutex } @@ -102,6 +104,10 @@ func New(dir, procRoot string, maxCount, maxAgeDays int) (*Store, error) { store := &Store{Dir: dir, ProcRoot: procRoot, MaxCount: maxCount, MaxAgeDays: maxAgeDays, Now: time.Now, LineageDepth: 8} store.Collect = func(pid int) ProcessDetail { return collectProcess(procRoot, pid) } store.ParentOf = func(pid int) int { return readParent(procRoot, pid) } + store.Assurance, err = assurance.New(dir) + if err != nil { + return nil, err + } if err := store.Prune(); err != nil { return nil, err } @@ -183,6 +189,14 @@ func (s *Store) CaptureEvent(event map[string]any) (string, error) { if err := writeAtomic(base+".log", []byte(formatText(report))); err != nil { return "", err } + if s.Assurance != nil { + if _, err := s.Assurance.Append("snapshot-log", base+".log", captured); err != nil { + return "", err + } + if _, err := s.Assurance.Append("snapshot-json", jsonPath, captured); err != nil { + return "", err + } + } if err := s.pruneLocked(); err != nil { return "", err } diff --git a/go-agent/internal/snapshot/store_test.go b/go-agent/internal/snapshot/store_test.go index 9baa139..905905e 100644 --- a/go-agent/internal/snapshot/store_test.go +++ b/go-agent/internal/snapshot/store_test.go @@ -10,6 +10,7 @@ import ( "testing" "time" + "codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/assurance" "codeberg.org/anassaeneroi/enodia-sentinal/go-agent/internal/model" ) @@ -61,6 +62,11 @@ func TestCaptureEventWritesCompatiblePairAndMergesSameSecond(t *testing.T) { t.Fatalf("%s mode=%#o", path, info.Mode().Perm()) } } + records, err := assurance.ReadAll(filepath.Join(store.Dir, assurance.ChainName)) + if err != nil || len(records) != 4 || records[0].Kind != "snapshot-log" || + records[1].Kind != "snapshot-json" || records[1].PrevHash != records[0].ChainHash { + t.Fatalf("assurance records=%#v err=%v", records, err) + } stats, err := store.SnapshotStats() if err != nil { t.Fatal(err)