What 5,535 DolphinBench Runs Taught Me About Local AI Memory

1. Why DolphinBench Breaks Toy Haystacks & My 5-Gate Clean-Room

Toy needle-in-a-haystack benchmarks hide a self-contained fact inside filler text and query the exact same nouns. Real workspace agents fail on stateful action execution under evolving rules. Released by Mem0 (mem0ai/dolphinbench), DolphinBench tests 600 live mock_mcp workspace tasks across 13,539 chronological sessions (1,500,265 o200k_base tokens) and three personas—Morgan (200 Founder/Ops tasks), Alex (200 Staff SWE tasks with 18 KB incident logs and deprecated flags), and Riley (200 Growth PM tasks)—plus 15 pure-deterministic counterfactuals. Passing requires retrieving standing policies that share zero nouns with today's terse prompt, resolving [Date: YYYY-MM-DD] temporal supersession, chaining list_* state lookups before mutating records, and withholding prohibited side effects.

Every run across my 9 × 615 = 5,535-trace local ablation suite and my 22 × 600 = 13,200-trace official openai/gpt-5.6-sol scoreboard is enforced by verify_zero_leakage.py across five gates:

1. Zero-LLM 0o444 SQLite Immutability: Ingests all 13,539 sessions in 153.4s with 0 LLM calls ($0.00), truncates WAL, applies chmod 0o444 + file:...?mode=ro URIs, and verifies pre/post SHA-256 digests across every 600-task run.
2. Static AST Clean-Room Scan: Verifies zero imports from grading rubrics, zero persona proper nouns, and zero test-ID branching.
3. Official 5-Method BenchmarkHarness Replay: Passes upstream python -m harness.export_submission --release upstream --check offline with 0 errors across all 22 archives.
4. Mechanical & Semantic Assertion Suite (1,920 YAML Assertions): Evaluates the exact same 1,310 deterministic mechanical checks (across 599 tasks) and 610 field_llm_judge semantic checks on every 600-task archive, alongside my zero-LLM inner-loop tool-argument gate.
5. Counterfactual Causality Gate (15 Tasks, memory_enabled=True vs. False): Confirms causal lift from retrieved memory (+53.3 pp on both 30B and 120B).

2. Axis A: Zero-LLM Hybrid Memory vs. Raw Cosine (+12.50 pp SWE)

Cloud memory layers run paid LLM extraction over every message during ingestion, whereas naive vector RAG embeds whole turns and fetches top-k cosine neighbors. Holding Stock gpt-oss:120b, 1024d snowflake-arctic-embed2, the 4,200-token budget, and judges fixed, swapping single-query cosine (VanillaSnowflakeHarness) for SovereignMemoryStore lifts overall pass rate from 48.50% (291/600) → 55.83% (335/600, +7.33 pp).

• Morgan Ties (65.0% Vanilla vs. 64.0% Sovereign): Short, self-contained executive notes (flight seating, wire caps) already rank cleanly under dense cosine similarity.
• Alex (32.0% → 44.5%, +12.50 pp) & Riley (48.5% → 59.0%, +10.50 pp) Jump: SWE and PM logs mix 14 KB pasted incident packets with terse channel rules. SovereignMemoryStore fixes three failure modes deterministically: (1) 256/32 Raw-RAG windows (arXiv:2610.30755) prevent long turns from diluting pooled vectors or starving token budgets; (2) Porter-FTS5 BM25 + 4-Way RRF (k = 60) anchors exact symbols like dedup-v2 and #release-ops; and (3) directive/transition regex priors + chronological turn_index packing keep later-dated policy overrides ordered last, backed by active search_persona_memory calls.

# sovereign_memory.py — Read-only 0o444 URI guard + 4-Way Hybrid RRF + Raw-RAG window projection
def retrieve_persona_briefing(path: Path, cache: FrozenCache, query: str, q_vec: list[float], rrf_k: int = 60):
    if (path.stat().st_mode & 0o222) != 0:
        raise PermissionError(f"Refusing benchmark retrieval on writable SQLite store: {path}")

    fts_expr = build_fts5_or_query(query)  # Quoted Porter-stemmed token OR-clause
    conn = sqlite3.connect(f"file:{path}?mode=ro", uri=True, timeout=10.0)
    try:
        top_turn_bm25 = [int(r[0]) for r in conn.execute(
            "SELECT rowid - 1 FROM turns_fts WHERE turns_fts MATCH ? ORDER BY bm25(turns_fts) ASC LIMIT 128",
            (fts_expr,),
        ).fetchall()]
        top_win_bm25 = [int(r[0]) for r in conn.execute(
            "SELECT rowid - 1 FROM windows_fts WHERE windows_fts MATCH ? ORDER BY bm25(windows_fts) ASC LIMIT 128",
            (fts_expr,),
        ).fetchall()]
    finally:
        conn.close()

    # 4-Way RRF: Turn-Dense (1.0) + Turn-BM25 (1.15) + Win-Dense (0.85) + Win-BM25 (0.95)
    turn_rrf: dict[int, float] = {}
    for rank, (t_idx, _) in enumerate(top_turn_dense):
        turn_rrf[t_idx] = turn_rrf.get(t_idx, 0.0) + 1.00 / (rrf_k + rank + 1)
    for rank, t_idx in enumerate(top_turn_bm25):
        turn_rrf[t_idx] = turn_rrf.get(t_idx, 0.0) + 1.15 / (rrf_k + rank + 1)
    for rank, (w_idx, _) in enumerate(top_win_dense):
        for t_idx in cache.windows[w_idx]["turn_indices"]:
            turn_rrf[t_idx] = turn_rrf.get(t_idx, 0.0) + 0.85 / (rrf_k + rank + 1)
    for rank, w_idx in enumerate(top_win_bm25):
        for t_idx in cache.windows[w_idx]["turn_indices"]:
            turn_rrf[t_idx] = turn_rrf.get(t_idx, 0.0) + 0.95 / (rrf_k + rank + 1)
    # Cross-encoder rerank + salience boost -> knapsack-pack 4,200 tokens in chronological order.

3. Axis B: Parameter Scale Phase Transition (8B → 30B MoE → 120B MoE)

With retrieval frozen, every model evaluated on SovereignMemoryStore receives the byte-identical 4,200-token turn-1 briefing across all 600 tasks, isolating parameter scale across my 6-run Phase-1 matrix (3,690 traces across Node-02 Dual RTX 2080 Ti 22 GB NVLink, Node-01 Dual RTX 3060 24 GB, and 96 GB Blackwell G4 burst):

Stock hermes3:latest (8B Q4_0) did not fail on JSON tool syntax—it emitted valid tool calls on 593/600 tasks (66.44% mechanical task gate), yet collapsed to 4.50% (0/15 pure deterministic) when binding multiple policy constraints simultaneously. Stepping up to local Qwen3-Coder-30B-A3B (only 3.3B active parameters on Dual RTX 2080 Ti at $0.00) crosses that reasoning threshold immediately, hitting 80.33% under hermes3 and a suite-high 88.81% (532/599) mechanical compliance.

4. Axis C: Wave-4 QLoRA+DPO, INV-STACK-10 2PC & Scale-Aware Grounded Harness v0.2.0

My most useful discovery came from Run 3's failure, where applying my Wave-3 SRE Code-Auditor adapter to gpt-oss:120b dropped generalist accuracy from 55.83% → 36.50%, alongside a hidden Ollama template bug on 8B:

• Wave-3 Auditor Role Bias (30.0% Side-Effect Withholding): Hardcoding a zero-trust SRE Auditor SYSTEM prompt into the fused Modelfile plus bug-hunting SFT caused 15.6x over-calling of read-only search_persona_memory (701 vs. 45 calls) and 180/600 (30.0%) zero-mutation turns where the model asked for human approval instead of executing routine calendar/email tasks.
• Ollama's Silent Hermes-3 Go Template Bug ({{- else if .System }}): Inspecting ollama show --modelfile hermes3:latest explained why Stock 8B scored near zero—attaching MCP tools silently discarded the entire runtime SYSTEM prompt while duplicating <tool_call> branches on assistant turns:

{{- if .Tools }}
<|im_start|>system
You are a function calling AI model... <tools>{{ .Tools }}</tools><|im_end>
{{- else if .System }}   <-- BUG: .System is silently dropped whenever .Tools is non-empty!
<|im_start|>system
{{ .System }}<|im_end>
{{- end }}

4.1 Deterministic Tool-Loop Governance Without Retraining: krusch-stack INV-STACK-10 2PC Action Harness

Inspecting post-Wave-4 traces revealed mechanical execution pathologies orthogonal to both memory retrieval and adapter weights: 5–12x identical read spin-loops, duplicate mutating MCP writes, unresolved placeholder arguments, and premature zero-app stops. Subclassing BenchmarkHarness in clean-room sovereign_harness_krusch_stack.py over krusch-stack Tier 3A (POST /action-harness/{start,intercept,outcome,verify-stop,summary}, 68/68 Node + 6/6 Python tests) enforces SHA-256 canonical BLOCK_SPIN_LOOP / BLOCK_DUPLICATE_WRITE, pre-commit HOLD_FOR_REVISION staging, and single-user-turn-preserving role: "developer" CONTINUE_VERIFY recovery steps:

• Live 10-Task Diagnostic Failure Slice (Local Qwen3-Coder-30B, $0.00): Eliminated 100% of duplicate MCP calls (10 → 0), reduced zero-app chat-only stops -66.7% (3 → 1), and recovered +20.0 pp (0/10 → 2/10) end-to-end mechanical passes (riley/096 & morgan/058).
• Full-600 Local Dual-2080-Ti Evaluation (Qwen3-Coder-30B-A3B Wave-4 QLoRA+DPO, + 2PC Harness, $0.00): Actively governed 56/600 (9.3%) tasks, cut executed duplicate MCP calls -95.2% (42 → 2), cut zero-app stops -50.0% (2 → 1), shrank archive size -17.7% (9.006 → 7.416 MiB), lifted Morgan (55.5% → 57.0%), Alex (35.0% → 35.5%), and deep compositional LBF>=4 by +6.67 pp (22.67% → 29.33%, 17/75 → 22/75), and unlocked 46 complementary task wins over Wave-4 (Std. Harness)—yielding Oracle@2 = 50.33% (302/600) at $0.00 (Oracle@3 incl. Stock, Std. Harness = 53.33% [320/600]).
• Full-600 Cloud Generalization (GPT 5.6 Luna High / Low, + 2PC Harness, 1,200 Traces): On GPT 5.6 Luna (High, + 2PC Harness) (80.50%, 483/600 @ $1.38 vs. 81.33% @ $1.35 Std. Harness), 2PC lifted SWE-heavy Alex +2.0 pp (78.0% → 80.0%), LBF=3 by +5.21 pp (79.17% → 84.38%), and LBF>=4 by +4.00 pp (68.00% → 72.00%) with 38 complementary wins (Oracle@2 = 87.67% [526/600]), while GPT 5.6 Luna (Low, + 2PC Harness) (76.50%, 459/600 @ $1.12) lifted Morgan +3.0 pp (85.0% → 88.0%) and LBF=3 +3.13 pp (Oracle@4 Std. ∪ 2PC Luna = 89.50% [537/600]; Oracle@5 incl. + Grounded = 91.83% [551/600]).

4.2 Scale-Aware Defect-Only Governance: Grounded Harness v0.2.0 (INV-GH-01..10 / INV-STACK-11) Across 1,800 Official Traces

Inspecting why INV-STACK-10 + 2PC lost -3.50 pp on noisy PM persona Riley (78.0% → 74.5% on Luna High; 37.5% → 33.5% on 30B Wave-4) exposed a subtle over-steering failure mode: unconditional Step-1 CONTINUE_VERIFY stop vetoes on zero-mutation turns occasionally pressured models out of valid 1-step policy abstentions or read-only status replies. Synthesizing 11,400 DolphinBench traces alongside 120 clean-room adyen/DABstep analytics episodes (42/42 upstream scorer.py parity: 90.0% @ $0.81 [-74.5% cost, 2.54x faster] on Gemini 4 Argon and 2x accuracy [40.0% vs. 20.0% @ $0.00] on Gemma 4 31B), I built Tier 3B krusch-grounded-harness v0.2.0 (sovereign_harness_grounded.py + stack.groundedHarness, 68/68 Node + 13/13 Python tests, 5/5 clean-room gates):

• Why Defect-Only Stop Gating + Turn-0 Deduplication + Spin-Loop Blocking Outperformed Uniform 2PC: Instead of vetoing legitimate 1-step policy abstentions, inspectOutputDefects (INV-GH-05) approves clean completions in 0 extra turns (0 self-doubt nudges) and triggers role: "developer" recovery steers strictly on objective output defects (EMPTY_OR_NULL_BAILOUT, TRUNCATED_OUTPUT, SERIALIZED_TOOL_CALL_IN_TEXT, RAW_DATAFRAME_OR_NUMPY_REPR) or uncommitted staged diffs. Simultaneously, quote-safe call hashing (INV-GH-04) memoizes prior reads and blocks byte-identical spin loops (BLOCK_SPIN_LOOP, maxReadStreak = 0) while permitting fresh post-mutation verification re-reads (committed_mutations > prior_mutations), and Turn-0 delta tracking (INV-GH-01..03) strips duplicate memory chunks already present in the 4,200-token Turn-0 briefing.
• Google Gemma 4 31B (Thinking ON) + Grounded Harness — 340/600 (56.67%, NEW #1 Open-Weights @ $0.00): Scores Morgan 146/200 (73.0%), Alex 97/200 (48.5%), and Riley 97/200 (48.5%), with Mech 808/1,310 (61.68%), Sem 349/610 (62.66% invoked), 15 spin loops blocked, 4 policy-floor blocks, 1 output defect healed, 1,228 Turn-0 duplicates suppressed, and 42.47s median latency (sha256: eab60088...). This lifts Gemma 4 31B Stock (325/600 = 54.17%, Queued UUID 288671ad-afe0-40a3-93af-b834c9bbd306) by +2.50 pp (+15 net tasks: 43 gained vs. 28 lost) overall and +5.50 pp (+11 tasks: 86 → 97) on noisy Riley, widens the matched-scale dense domain gap over Qwen2.5-Coder-32B (35.33%) to +21.33 pp (+18.83 pp Stock), lifts Gemma 4 31B Oracle@3 to 66.17% (397/600) (Oracle@2 = 61.33% [368/600]), and beats all 8 non-GPT-5.6-Luna commercial cloud stacks up to $1,830.57.
• GPT 5.6 Luna (High Thinking) + Grounded Harness — 489/600 (81.50%) @ $1.35 (1.57x Faster): Scores Morgan 174/200 (87.0%), Alex 157/200 (78.5%), and Riley 158/200 (79.0%) in 30.03s median latency (1.57x faster than Luna High Base 47.02s / 81.33% [488/600])—beating Official #1 Hermes + GPT 5.6 Luna + Mem0 (70.67% @ $96.21) by +10.83 pp at 71.1x lower cost, recovering +4.50 pp on Riley (149 → 158) over INV-STACK-10 2PC with 5,318 Turn-0 duplicates suppressed and 0 self-doubt nudges, and lifting Luna High Oracle@2 to 89.33% (536/600) and Luna Oracle@5 to 91.83% (551/600) (sha256: 477047fc...).
• Gemini 4 Argon (Low Thinking) + Grounded Harness — 535/600 (89.17%) @ $23.92 (Argon Low Oracle@2 = 94.00%): Scores Morgan 183/200 (91.5%), Alex 176/200 (88.0%), and Riley 176/200 (88.0%) with 8,092 Turn-0 duplicate tool calls suppressed and 0 self-doubt nudges (-2.5% lower cost vs. $24.54 Std., exact McNemar p = 0.104 NS). Because Argon Low uses minimal internal thinking tokens and relies on Turn-1 search_persona_memory as a recency spotlight, delta deduplication trades 29 spotlight tasks for +17 new deep tail-recall passes alongside #1 Overall Argon Low (547/600 = 91.17% @ $24.54)—lifting Argon Low Oracle@2 to 94.00% (564/600) (sha256: 5ce8a7f8...).

5. Official 22-Run openai/gpt-5.6-sol Scoreboard (13,200 Traces · 8 Queued)

Building on my 9-run Wave-3/Wave-4 local ablation (5,535 verified traces), I evaluated 22 clean-room 600-task configurations (13,200 official traces across 6 model families: Google Gemini [5], OpenAI GPT-5.6 Luna [5], Google Gemma 4 [3], Alibaba Qwen [5], OpenAI GPT-OSS [2], and Nous Hermes [2]) on the byte-identical frozen 0o444 SovereignMemoryStore SQLite checkpoints. Every archive was graded with the official openai/gpt-5.6-sol (reasoning_effort="medium") judge across all 1,920 YAML assertions (1,310 mechanical + 610 field_llm_judge), verified offline with 0 errors via harness.export_submission --check (22/22 passed), and 8 flagship archives are queued on dolphinbench.ai alongside 14 clean-room model-matched, 2×2 reasoning/search, dense-vs-MoE scaling, + 2PC, and + Grounded harness controls:

*Every official archive evaluates the exact fixed 1,310 YAML mechanical checks and 610 field_llm_judge checks (1,920 total); parentheses on Wave-4/Stock open-weights rows show my zero-LLM 1,310-check inner-loop tool-argument gate prior to semantic SQLite post-state verification.

• Model-Matched GPT 5.6 Luna Isolates +10.83 pp (+10.67 pp Base) Purely from $0.00 Raw-RAG Memory & Runs 1.57x Faster: Running GPT 5.6 Luna (High) over my frozen 0o444 SQLite store reaches 81.50% (489/600, 30.03s = 1.57x faster) with + Grounded Harness and 81.33% (488/600, 47.02s) on Std. Harness for $1.35—beating Official #1 Luna High + Mem0 (70.67% @ $96.21) by +10.83 pp (+65 tasks) at 71.1x lower total cost (Luna High Oracle@2 = 89.33% [536/600], Luna Oracle@5 = 91.83% [551/600]), while Luna Low (Std. Harness) hits 76.67% (460/600) for $1.10 (87.4x cheaper).
• Wave-4 Beat Stock at Every Fine-Tuned Scale: Local Qwen3-Coder-30B-A3B (Wave-4 QLoRA+DPO, Std. Harness) climbed 42.00% → 42.67% (252 → 256/600, 1,123/1,310 = 85.73% local gate, +11 tasks on Riley) at $0.00 and 15.72s—and uniting Std. Harness with + 2PC Harness reaches Oracle@2 = 50.33% (302/600) at $0.00, outperforming MiniMax M3 + Mem0/Honcho/Hindsight (41.00–47.83% @ $148–$212) and Claude Sonnet 5 + Honcho/Mem0 (35.83% / 32.33% @ $1,566–$1,831). 120B eliminated Wave-3 prose refusals (39.17% → 39.50%), and 8B Q4_0 jumped 15.7x (7/600 → 110/600 = 18.33%).
• Why Local 30B (42.67%) Beats Sparse 120B (39.50%) & 80B (38.50%): Qwen3-Coder-30B-A3B chains multi-step list_* state discovery before writes on 320+ tasks (only 2 zero-tool turns; 1 under + 2PC), whereas scaling inactive MoE experts alone (80B / 120B at 3B–5B active params) leaves multi-constraint synthesis bounded near ~40%.

6. 6-Family Synthesis: Gemma 4 31B 56.67%, Model-Matched Luna 81.50%, Argon 91.17% & Scale-Aware Harnesses

6.1 What Kind of Local Model Excels at Multi-App Agentic Memory? (Gemma 4 31B 56.67% Grounded / 54.17% Stock vs. Coder-32B 35.33%)

Before testing dense non-coder models, it looked plausible that open-weights agents hit a hard ~42% wall on DolphinBench. Evaluating Apache-2.0 Google Gemma 4 31B (30.7B 100% active dense Q4_K_M) disproved that hypothesis immediately: stock Gemma 4 31B (Dense Stock, Std. Harness) (Queued UUID 288671ad-afe0-40a3-93af-b834c9bbd306) reaches 54.17% (325/600) with native <|thought|> thinking enabled and 52.00% (312/600) with thinking disabled (4.9x fewer output tokens: 174,226 vs. 853,604; Stock Oracle@2 = 61.50% [369/600]), while pairing Gemma 4 31B (Thinking ON) with + Grounded Harness v0.2.0 sets a NEW #1 Open-Weights SOTA of 56.67% (340/600) at $0.00 total cost (Morgan 146/200 [73.0%], Alex 97/200 [48.5%], Riley 97/200 [48.5%]; +2.50 pp overall, +5.50 pp on noisy Riley, 15 spin loops blocked, and Gemma 4 31B Oracle@3 = 66.17% [397/600])—beating all 8 non-GPT-5.6-Luna commercial cloud baselines on the official leaderboard up to $1,830.57 and surpassing GPT 5.6 Luna + Hindsight/Honcho (144/200 = 72.0%) on Morgan (146/200 = 73.0%).

Holding active parameter count fixed near ~31B dense, general-domain multi-app tool post-training (Gemma 4 31B: 56.67% Grounded / 54.17% Stock) outperforms code-specialized Qwen2.5-Coder-32B (Dense Stock, Std. Harness) (35.33%, 212/600, only 0.03 memory calls/task) by +21.33 pp (+128 tasks; +18.83 pp / +113 tasks Stock, McNemar p = 1.35 × 10⁻¹⁶) and sparse 3.3B-active Qwen3-Coder-30B-A3B (Wave-4 QLoRA+DPO, Std. Harness) (42.67%, 256/600) by +14.00 pp (+84 tasks; +11.50 pp / +69 tasks Stock, p = 4.96 × 10⁻⁷). Even 14.8B dense generalist Nous Hermes-4-14B (Dense Stock, Std. Harness) (34.50%, 207/600, 1.99 memory calls/task) statistically ties 2.2x-larger Qwen2.5-Coder-32B (p = 0.729) and beats Claude Sonnet 5 + Mem0 (32.33% @ $1,830.57)—confirming that workplace assistants need dense general-domain social/calendar/email post-training alongside coder backbones.

6.2 Two-Factor SOTA Decomposition, Goldilocks Reasoning & Retrieval Sufficiency (Luna 81.50% @ $1.35 · Argon Low 91.17% · Oracle@22 = 97.83%)

Because Official Leaderboard ranks #1–#5 all use OpenAI GPT 5.6 Luna (High), running model-matched GPT 5.6 Luna (High) over my frozen 0o444 SovereignMemoryStore isolates the exact return on memory: it achieves 81.50% (489/600) in 30.03s (1.57x faster) with + Grounded Harness and 81.33% (488/600, 47.02s) under Std. Harness for $1.35, beating Official #1 Luna High + Mem0 (70.67%, 424/600 @ $96.21) by +10.83 pp (+65 tasks) / +10.67 pp (+64 tasks—52.0% of my total +20.50 pp SOTA lead) at 71.1x / 71.0x lower cost, while GPT 5.6 Luna (Low, Std. Harness) hits 76.67% (460/600) for $1.10 (87.4x cheaper). Upgrading the agent backbone to Gemini 4 Argon (Low, Std. Harness) (1.84 memory calls/task) contributes the remaining +9.67 pp vs. Luna Grounded / +9.83 pp vs. Luna Base (82 vs. 23 unique wins, McNemar p = 5.96 × 10⁻⁹) to reach 91.17% (547/600, #1 Overall) at $24.54 (3.9x cheaper than Mem0; 89.17% [535/600] @ $23.92 with + Grounded), placing all four vertices of the global Cost–Accuracy Pareto frontier ($0.00 56.67% / $1.10 76.67% / $1.35 81.50% / $24.54 91.17%) on SovereignMemoryStore.

Comparing reasoning effort across three model families reveals a 3-regime Goldilocks reasoning curve: across my 2×2 Gemini 4 Argon factorial (2,400 traces; 3,000 total incl. + Grounded), Thinking × Agentic Search interacts significantly (ΔΔ = +3.17 pp, 95% CI [+0.27, +6.06], p = 0.0320) so Low (91.17%) beats Medium (89.17%) by reinvesting thinking budget into iterative search_persona_memory calls (+13.17 pp lift over Low Briefing-Only 78.00% @ $15.76); whereas on GPT 5.6 Luna (3,000 traces across 5 runs), High beats Low by +4.67 pp (p = 0.0020) under Std. Harness and +4.00 pp (p = 0.0116) under + 2PC Harness because Low cuts reasoning tokens 3.3x (282.0 → 85.3/task) and prematurely drops active memory searches -32.6% (476 → 321)—matching local Gemma 4 31B (1,800 traces; +2.17 pp from <|thought|>).

Across all 22 runs, ensemble union reaches Oracle@22 = 97.83% (587/600, bounding zero-LLM indexing misses to <= 2.17% [13/600], up from 97.00% Oracle@19), Argon Low Oracle@2 = 94.00% (564/600, +17 unique passes), Luna Oracle@5 = 91.83% (551/600; Luna High Oracle@2 = 89.33% [536/600]), and Oracle@12 = 81.17% (487/600) across $0.00 open-weights runs alone (+10.50 pp above #1 Mem0, up from 80.00% Oracle@11!), with a 4.19x superlinear multiplier at compositional depth LBF>=4 (89.33% [67/75] vs. 21.33% [16/75] on 30B Stock & Gemma 4 31B Stock) and flatter corpus-entropy decay (-3.50 pp Argon Med / -7.00 pp Argon Low vs. -10.00 pp Morgan → Riley and -12.00 pp Morgan → Alex on Mem0). See companion Paper #27 (Return on Memory) and @krusch/toolkit/cascade for L1 (8B) → L2 (Gemma 4 31B 56.67% / 30B @ $0.00) → L2.5 (Luna 76.67%–81.50% @ $1.10–$1.35) → L3 (Argon Low 91.17% @ $24.54) routing.

6.3 Uniform 2PC vs. Scale-Aware Grounded Harness v0.2.0 & Triage Boundaries

• Why Scale-Aware Defect-Only Governance Outperformed Uniform Stop Vetoes (Sections 4.1 & 4.2): Uniform INV-STACK-10 + 2PC Harness eliminates -95.2% (42 → 2) of duplicate MCP calls at $0.00 and lifts high-branching compositional tasks (+6.67 pp at LBF>=4 on 30B Wave-4, Oracle@2 = 50.33% [302/600]; +5.21 pp at LBF=3 and +4.00 pp at LBF>=4 on Luna High), but unconditional Step-1 CONTINUE_VERIFY vetoes incur a minor -0.17 to -0.83 pp over-steering tax on simple LBF=1–2 / Riley abstentions. Replacing uniform stop vetoes with scale-aware Grounded Harness v0.2.0 (INV-GH-01..10 / INV-STACK-11: 0-turn defect-only inspectOutputDefects stop gating + quote-safe BLOCK_SPIN_LOOP + Turn-0 delta deduplication) resolves that tradeoff—lifting Gemma 4 31B +2.50 pp overall and +5.50 pp on Riley to 56.67%, accelerating Luna High 1.57x (47.02s → 30.03s) to 81.50% while recovering +4.50 pp on Riley, and adding +17 unique tail-recall wins on Argon Low (Oracle@2 = 94.00%).
• Judge Calibration & 8B Q4_0 Triage Boundary: Local Referees (hermes3 80.33%, gemma4:e4b 53.00%) preserved identical rank order to gpt-5.6-sol (42.67%) for $0.00 inner-loop iteration, while 8B Q4_0's 18.33% ceiling (14.5% SWE/PM) confirms 4-bit 8B models belong in L1 triage rather than multi-app state mutation, and role-neutral Modelfiles prevent auditor refusal drift.

7. Reproducibility, Compute Ledger & Public Artifacts

# 1. Offline unit suites (13/13 Grounded + 6/6 2PC Python tests; 68/68 Node tests) & 5-gate AST verifier
.venv/bin/python -m unittest discover -s tests -p "test_*.py" -v && .venv/bin/python verify_zero_leakage.py --static-only

# 2. Local 600-task run, Tier 3A 2PC Action Harness, or Tier 3B Grounded Harness v0.2.0 ($0.00)
.venv/bin/python run_grounded_600_matrix.py --suite gemma4_31b_grounded --workers 8
PYTHONPATH=.:upstream .venv/bin/python -m harness.runner --release upstream --adapter sovereign_harness_grounded:BenchmarkHarness

# 3. Official openai/gpt-5.6-sol grading + offline upstream archive check across all 22 archives
.venv/bin/python package_official_submissions.py --only gemma4_31b_grounded --workers 12
PYTHONPATH=upstream .venv/bin/python -m harness.export_submission --release upstream --check submissions/sovereign_gemma4_31b_grounded.zip

• Compute & Cost Ledger: $0.00 API spend across 153.4s zero-LLM SQLite ingestion and all 12 official 600-task open-weights archives (7,200 official open-weights traces + counterfactuals across Node-02 Dual RTX 2080 Ti, Node-01 Dual RTX 3060, Node-03 RTX 2080 Ti, and on-demand 96 GB Blackwell G4 burst with 0 active sessions post-run), plus $105.70 across all five Gemini 4 Argon runs ($24.54 Low Std., $23.92 Low + Grounded, $24.57 Med Std., $16.91 Med Briefing-Only, $15.76 Low Briefing-Only) and $6.31 across all five GPT 5.6 Luna runs ($1.35 High + Grounded, $1.35 High Std., $1.38 High + 2PC, $1.10 Low Std., $1.12 Low + 2PC—less than 1/15th of a single $96.21 Mem0 run; $112.01 combined across all 10 cloud archives / 6,000 cloud traces, less than a third of one $345.64 Supermemory run).
• Public Datasets & Open Weights: krush420/sovereign-g4-benchmarks (22 dolphinbench_600 rows / 13,200 official traces + sanitized raw JSON audit logs) · krush420/sovereign-router-arena (Tab 5) · krush420/sovereign-code-audit-sft (18,131 rows) · Wave-4 adapters (30B, 120B, 8B) · github.com/kruschdev/krusch-stack · github.com/kruschdev/krusch-grounded-harness.