Skip to content

CRDT State

Two daemons, one infrastructure. CRDTs are how s.a.u avoids split-brain without a central coordinator.

Important

When multiple daemon instances or concurrent goroutines mutate remediation state, CloudSlash needs a way to converge without a central lock. This page covers the Last-Writer-Wins Map CRDT implemented in pkg/engine/sau/crdt/: how it merges, rejects stale updates, and guarantees eventual consistency.


Network Partitions

and CAP

Running CloudSlash across multiple CI/CD runner nodes creates unavoidable network partitions. If Runner A mutates node 1 while Runner B mutates node 2, routing both mutations through a centralized SQL store risks transaction lock failure: the fundamental CAP Theorem constraint.

The answer is to drop centralized locking entirely and track internal state with a Last-Writer-Wins (LWW) Map CRDT.

Mathematical Formalization

A CRDT guarantees convergence mathematically. The merge(X, Y) operation must satisfy three properties:

  1. Commutativity: \(X \cup Y = Y \cup X\). Order doesn't matter.
  2. Associativity: \((X \cup Y) \cup Z = X \cup (Y \cup Z)\). Grouping doesn't matter.
  3. Idempotency: \(X \cup X = X\). Applying the same update twice has no effect.

Go Implementation

Each mutating event is indexed against local physical clocks.

// pkg/engine/sau/crdt.go
type LWWRegister struct {
    Value     interface{}
    Timestamp int64
}

type LWWMap struct {
    Data map[string]LWWRegister
    Mu   sync.RWMutex
}

func (m *LWWMap) Merge(remoteMap map[string]LWWRegister) {
    m.Mu.Lock()
    defer m.Mu.Unlock()

    for key, remoteReg := range remoteMap {
        localReg, exists := m.Data[key]
        if !exists || remoteReg.Timestamp > localReg.Timestamp {
            m.Data[key] = remoteReg
        }
        // In exact timestamp collisions, arbitrarily prefer remote to satisfy commutativity
        if exists && remoteReg.Timestamp == localReg.Timestamp {
            // Compare bytes deterministically if utilizing vectors
            m.Data[key] = remoteReg
        }
    }
}

Saga Resumption

If a daemon crashes with exit status 137 (OOM kill) mid-saga, the LWW Map guarantees exact state continuity on restart.

The restarted daemon reads its persisted CRDT payload. Because steps n=1 and n=2 were logged with monotonic timestamps, the map mathematically reflects exactly what happened before the kernel fault. The daemon picks up at n=3 or executes the required compensation sequence.