Compounding Intelligence

Agentic Memory vs RGI Judgment Memory

Two architectures for teaching agents to remember decisions. One stores traces and retrieves by similarity. The other learns a compressed prototype that scores in O(1) — no retrieval, no LLM call.

pip install graph-attention-engine  # ~844 params, Apache 2.0

What should an agent remember — and how?

AGENTIC MEMORY trace 1 trace 2 trace 3 … trace n k-NN search (scan all) 1 LLM call (synthesize) O(n) time · O(n) space No drift protection RGI JUDGMENT MEMORY centroid tensor (~844 params) L2 distance (single computation) Conservation gate O(1) time · O(1) space Zero LLM calls · Drift-protected

What is Judgment Memory?

  • One learned prototype (centroid) per decision cell
  • Each verified outcome shifts the nearest centroid: c += η·(outcome − c)
  • Criteria that predict quality get higher attention weights — criteria that don't get suppressed
  • ~844 parameters · No LLM in the loop · Fully inspectable

Head-to-Head — 2,000 Decisions

Same decision stream. 4 categories, 8 factors, 4 actions. JM scores before it learns. k-NN: distance-weighted retrieval, k=5, 400-chunk cap, FIFO eviction.

Rolling accuracy (window 50) — click a voiceover to start

Judgment is geometry — not a metaphor

1 Arrive 8 factors extracted: amount · vendor history · PO match approval tier · days outstanding line-item count · currency risk prior exceptions 2 Score L2 distances: Auto-Approve: 0.034 Escalate: 0.71 · Hold: 1.24 P(auto-approve) = 0.97 1 computation · 0 LLM calls 3 Learn Conservation gate: disp 0.034 < thresh 0.5 → PASS Full η = 0.08 Centroid shifts 0.0017 toward this invoice BLACK BOX Neural network says “approve” Why? Backprop through 10M params Not auditable in production Cannot undo a wrong decision RGI OPEN GEOMETRY L2 distance = 0.034 to Auto-Approve 8 factor values visible Conservation gate visible Identify the cell. Trace it. Undo if wrong. vs
THE TRADEOFF AGENTIC MEMORY WINS Inspect which traces influenced each decision JUDGMENT MEMORY WINS O(1) · Zero LLM · Conservation gate <1ms vs ~340ms · Drift-protected vs EVERY QUARTER, EVERY CFO TEAM “Is 3.76% margin the right number — or is it more likely 2.95%?” The senior analyst who answers draws on years of pattern recognition. If they leave, the next person guesses. JUDGMENT MEMORY CHANGES THIS 10,000 verified decisions → centroid tensor → new analyst inherits on day one The knowledge doesn't leave when the person does.

Full Production Classes — Not Pseudocode

Two complete implementations. Read them side by side.

class AgenticMemory:
    """k-NN retrieval memory. The pattern behind LangChain, Mem0,
    and most agentic frameworks."""

    def __init__(self, k=5, cap=200):
        self.store = []          # flat list, all categories mixed
        self.k = k               # neighbors to retrieve
        self.cap = cap           # max traces before eviction

    def score(self, factors, category='general'):
        """O(n*K) — scans entire store, every call."""
        if len(self.store) < self.k:
            return None, 0.0

        dists = [
            (euclidean(factors, t['factors']), t)
            for t in self.store
        ]
        dists.sort(key=lambda x: x[0])
        neighbors = dists[:self.k]

        votes = {}
        for d, t in neighbors:
            w = 1.0 / (d + 1e-8)
            votes[t['action']] = votes.get(t['action'], 0) + w

        best = max(votes, key=votes.get)
        return best, votes[best] / sum(votes.values())
        # → THEN: 1 LLM call to synthesize retrieved context

    def learn(self, factors, action, **kwargs):
        """Append. No compression. No gate. No quality signal."""
        self.store.append({
            'factors': factors,
            'action': action,
            'ts': now()
        })
        if len(self.store) > self.cap:
            self.store.pop(0)     # FIFO eviction
        # No drift protection
        # No outcome-quality handling
        # kwargs silently ignored
class JudgmentMemory:
    """Centroid learning. O(1) inference, conservation-gated updates."""

    def __init__(self, tau=0.1, drift_thresh=0.4):
        self.centroids = {}      # {category: {action: [mu]}}
        self.tau = tau            # softmax temperature
        self.drift_thresh = drift_thresh
        self.gate_log = []       # every drift event recorded

    def score(self, factors, category='general'):
        """O(A*K) — constant, category-scoped. Zero LLM calls."""
        if category not in self.centroids:
            return None, 0.0

        C = self.centroids[category]
        dists = {
            a: sum((f - c)**2 for f, c in zip(factors, mu))
            for a, mu in C.items()
        }
        exps = {a: exp(-d / self.tau) for a, d in dists.items()}
        Z = sum(exps.values())
        best = max(exps, key=exps.get)
        return best, exps[best] / Z
        # → Done. 0 LLM calls. <1ms.

    def learn(self, factors, action, category,
              confirmed=True, outcome=None):
        """Update centroid. Conservation gate checks drift."""
        if outcome == '__incorrect__':
            return {'skipped': True}  # uncertain → skip

        eta = 0.05 if confirmed else 0.01   # 5x differential
        mu = self.centroids[category][action]

        # Conservation gate
        disp = norm(factors - mu)
        if disp > self.drift_thresh:
            eta *= 0.5           # dampen outlier
            self.gate_log.append({'disp': disp, 'action': action})

        for i in range(len(mu)):
            mu[i] += eta * (factors[i] - mu[i])  # c += η·(x - c)

        return {'gated': disp > self.drift_thresh,
                'eta': eta, 'disp': disp}

Five things JudgmentMemory does that AgenticMemory cannot:

  • Category-scoped centroids — decisions don't bleed across domains
  • Conservation gate — outliers can't hijack learned judgment
  • Differentiated learning rates — confirmed experience weighs 5× more than overrides
  • Outcome-quality handling — uncertain decisions are skipped, not learned from
  • Gate log — every drift event recorded for audit

Structural Comparison

Dimension Agentic Memory Judgment Memory
Inference complexity O(n·K) — grows with store O(A·K) — constant
Memory footprint n traces × K factors each A·K floats per category
LLM calls at inference 1 (synthesis of retrieved context) 0
Drift protection None (FIFO eviction only) Conservation gate (3-layer)
Learning signal Append only Confirmed/override (η differentiated)
Latency (modeled) ~340ms (retrieval + LLM) <1ms
Category isolation Single shared store Scoped per category
Eval protocol No separation of train/eval Score-before-learn (strict online)

Why This Matters for Your Decision Systems

Ablation effect
+9.2 pp
JM present vs removed
95% CI
[+7.2, +11.3]
500 bootstrap resamples — excludes zero
Drift recovery
+14 pp
JM ahead of k-NN at N=1,150
Infrastructure
pip install
No GPUs · No LLM calls · Apache 2.0