grep snippets without caller/callee blast-radius graphs, (2) they have zero persistent memory of project invariants or previously refuted false positives, and (3) the same model family grades its own homework—producing shortcut contagion. We replaced that brittle pattern with a 6-stage closed-loop audit and self-distillation pipeline trained through KruschStack (npx krusch-stack) to benefit KruschStack. Every audit turn is hydrated by our 3-Tier MCP substrate (krusch-context-mcp, krusch-git, krusch-fs-mcp), filtered through an 11-checkpoint investigation with mandatory Adversarial Steel-Manning, verified by an isolated Cross-Family Referee (Gemma 4 + Hermes 3 vs. Qwen3-30B-MoE / Gemini 4 Argon) until two consecutive zero-finding passes converge, and compiled into a 17,899-row test-gated SFT corpus (krush420/sovereign-code-audit-sft) that fine-tunes both our 30B Student LoRA (L2) and 120B Teacher LoRA (L2.5) on a single 96GB Blackwell G4 GPU.
119+ Full Multi-Phase Sweeps
+ Bounded OS Page Trees
+ 401 Argon Golden Remediations
QLoRA Token Accuracy on G4
| Open-Source Artifact | Live Repository | Role in the KruschStack Ecosystem |
|---|---|---|
| 🛠️ KruschStack MCP Spine | github.com/kruschdev/krusch-stack |
Unified 7-Pillar, 39-Tool MCP Server & CLI (npx krusch-stack init) that generated and verified the entire training corpus. |
| 🤖 Jean SRE Daemon | github.com/kruschdev/Jean-SRE-by-kruschdev |
24/7 Autonomous 11-Phase Code Auditor, 6-MCP Harness, Zero-Leak Dataset Sanitizer & Single-GPU 96GB Blackwell QLoRA Scripts. |
📊 SFT Dataset (17,899 rows) |
krush420/sovereign-code-audit-sft |
3 Splits on Hugging Face: argon_golden_remediations (401), elitist_120b_teacher (2,208), and code_audit_11phase_sft (15,290). |
⚡ 30B Student QLoRA (L2) |
krush420/qwen3-coder-30b-sre-lora |
QLoRA adapter + Ollama Modelfile for Qwen/Qwen3-Coder-30B-A3B-Instruct (loss 0.731 → 0.214, token acc 93.36%). |
🧠 120B Teacher QLoRA (L2.5) |
krush420/gpt-oss-120b-sre-lora |
Native MXFP4 QLoRA adapter + Ollama Modelfile for openai/gpt-oss-120b (loss 7.448 → 1.737, token acc 64.89%). |
1. Why Naive LLM Code Audits Fail in Production
If you point a raw frontier or local LLM at a repository and ask it to "find bugs," three failure modes immediately dominate:
- Truncation Hallucinations: Standard agents run a keyword search or read the first 100 lines of a file, see a helper function without an inline
try/catchor auth check, and flag a critical vulnerability—completely unaware that every inbound caller already validates the payload at the router boundary. - Amnesia & False-Positive Loops: Without governed episodic memory, an autonomous daemon sweeping a codebase every night will re-discover the same intentional architectural tradeoff on Lap 2 that a human engineer already refuted on Lap 1.
- Single-Family Shortcut Contagion: When the model that writes a fix (or shares the same pre-training family and reasoning trace) also acts as the judge, it rationalizes subtle spec violations. As documented in multi-agent verification research (arXiv: 2608.03744), a judge exposed to the generator's chain-of-thought inherits its blind spots.
Our autonomous SRE and code-audit engine (Jean SRE, running 24/7 across our 8-node bare-metal cluster) was engineered to solve all three failure modes at the substrate level—and then turn every verified audit and code repair into high-signal training data tailored specifically to our stack.
┌─────────────────────────────────────────────────────────────────────────────────────────┐
│ THE KRUSCHDEV CLOSED-LOOP AUDIT & DISTILLATION PIPELINE │
├─────────────────────────────────────────────────────────────────────────────────────────┤
│ STAGE 1 · 3-TIER MCP CONTEXT HYDRATION (`buildProjectAuditContext`) │
│ Tier 1 (`krusch-context-mcp`): Active Invariants (INV-*), Historical Bug Filter│
│ Tier 2A (`krusch-git`): Brace-Matched AST Symbols + Multi-Hop Caller CTEs │
│ Tier 2B/2C (`krusch-fs` + Argon): Page Trees, Rubric Switch & 3-Viewport UI │
├────────────────────────────────────────────┬────────────────────────────────────────────┤
│ ▼ │
│ STAGE 2 · 11-CHECKPOINT INVESTIGATION & ADVERSARIAL STEEL-MAN FILTER │
│ Phases `0_ORIENTATION` ──► `1.8_RISK_MAP` (Zero-Truncation Symbol Reads) │
│ ──► Phase `1.9_ADVERSARIAL_REVIEW` (Steel-Man Status Quo + Refute Traps) │
│ ──► Phase `2_MASTER_BACKLOG` (Verified `[BUG|SEC|ARCH|PERF|TEST-XX]` Cards) │
├────────────────────────────────────────────┬────────────────────────────────────────────┤
│ ▼ │
│ STAGE 3 · FROZEN SPEC, DISJOINT SUBAGENTS & STAGED 2PC REPAIR (`krusch-harness`) │
│ Freeze `spec.md` (`{ ok, tool, summary, data }`) ──► Disjoint File Fan-Out │
│ ──► Orchestrator Choke-Point Wiring ──► Postgres `staged_diffs` + Sandbox │
├────────────────────────────────────────────┬────────────────────────────────────────────┤
│ ▼ │
│ STAGE 4 · CROSS-FAMILY "REFEREE OVER JUDGE" CONVERGENCE (`lib/spec-verifier.js`) │
│ Generator (`Qwen3-30B-MoE` / `Argon`) vs. Isolated Referees (`Gemma 4` + │
│ `Hermes 3` at temp=0.0, zero reasoning trace) ──► 2 Consecutive Zero Passes │
├────────────────────────────────────────────┬────────────────────────────────────────────┤
│ ▼ │
│ STAGE 5 · PORTABLE SFT CURATION & ELITIST REJECTION SAMPLING (`scoreElitistTrajectory`) │
│ 100% Unit Test Gate + `$WORKSPACE_ROOT` Path Normalization + Citation Scoring │
│ ──► 9,377 Student SFT Rows + 1,168 Elitist 120B/Argon Teacher Rows │
├────────────────────────────────────────────┬────────────────────────────────────────────┤
│ ▼ │
│ STAGE 6 · DUAL-TRACK QLoRA DISTILLATION ON 96GB BLACKWELL G4 │
│ Track 1: `gpt-oss-120b` (128 MoE Experts Frozen in 4-bit MXFP4 + BF16 LoRA) │
│ Track 2: `Qwen3-Coder-30B-A3B` (16K ctx NF4/BF16 LoRA ──► 124.2 tok/s Local) │
└─────────────────────────────────────────────────────────────────────────────────────────┘
2. Stage I: How the 3-Tier MCP Stack Hydrates Context Before a Single Token Is Sampled
A language model is only as effective as the structural truth in its context window. Rather than letting an auditor wander blindly through a repository with raw shell commands, buildProjectAuditContext in jean_sre_sweep.js and our interactive 3-Tier MCP stack pre-hydrate a deterministic, token-budgeted context frame for every single audit phase.
A. Tier 1 (krusch-context-mcp): Bounded Invariant Hydration & Historical Bug Tagging
Every session and audit phase begins with Turn-1 Zero-Trust Hydration via krusch-context-mcp (v1.8.1). As our autonomous daemon accumulated over 2,350 episodic memories across 88 targets (a 60MB+ SQLite WAL + PostgreSQL 16 store), we engineered three critical hydration guarantees so models get maximum signal with zero latency or stale-memory confusion:
- Sub-5ms Bounded Wildcard Retrieval: Wildcard state hydration (
query: "*") executes a boundedORDER BY id DESC LIMIT 250query that omits raw 1,024-dim embedding JSON strings during candidate fetching and caps thepackTokenBudgetpost-retrieval deduplication slice—eliminating MCP-32001timeouts while packing the newest active invariants (INV-*) and architectural decisions first. historical_fixed_bugLabeling: In early audit laps, models occasionally read past bug post-mortems from working memory and re-filed them as open defects. Our context formatter explicitly tags resolved bug memories ashistorical_fixed_bugso the auditor knows what was already repaired and verifies regression protection instead of duplicating stale tickets.INV-12Span-Grounded SHA-256 Drift Detection: Memories anchored to specific file line ranges carry a cryptographic hash of the underlying code span. If a file mutates,krusch_context_healthflags the memory as drifted before the auditor can rely on an obsolete assumption.
B. Tier 2A (krusch-git): Relational AST Symbol Tables & Multi-Hop Caller Graphs
Before judging any module, the auditor queries krusch-git (v1.2.2) to hydrate the repository's structural graph from PostgreSQL 16 + pgvector:
- Source-Prioritized AST Indexing:
listProjectFilesprioritizes active source directories (src/,server/,lib/,app/,bin/,scripts/) and filters by language extension during traversal sobuildProjectAstIndexnever exhausts its file budget on static test fixtures or build artifacts. - Zero-Dependency Comment-Masked Brace Matching: Our AST chunker (
lib/ast-chunker.js) masks comments, string literals, template strings, and regexes viamaskCodewhile preserving exact newline (\n) offsets, starting brace depth atmatch.index + match[0].length - 1so every function, class, route, and method is indexed with exactstartLine-endLinecoordinates. - Caller/Callee Blast-Radius CTEs (
krusch_git_dependency_graph): For any symbol under review, recursive SQL Common Table Expressions trace every inbound caller and outbound callee across the repository—giving the model immediate proof of whether input validation, auth guards, or cancellation signals (AbortController) happen upstream.
C. Tier 2B & 2C (krusch-fs-mcp, Dynamic Rubric Switching & Gemini 4 Argon UI Telemetry)
- Language-Aware Rubric Injection:
buildProjectSpecAndDocsBriefandloadHomelabBaselineRulesinspect the target's manifest (pyproject.tomlvs.package.json) and automatically inject Python 3.12 standards (FastAPI,SQLAlchemy,pytestinside isolated/opt/venv) or Node.js 22 ESM standards (node:sqlite,oxlint,node --test). - Bounded Page Trees (
krusch-fs-mcp): Large specs and configs are inspected via structural tables of contents (krusch_fs_page_tree) and read in bounded page slices (krusch_fs_read_page) with SHA-256PageMemoryannotations. - Live Source Ground-Truth Ordering: In
buildProjectAuditContext, live source verification windows (buildPendingIssueCodeEvidence) are injected at the very bottom of the prompt—after any accumulated markdown checkpoints—with an explicit rule that live source code overrides any priorJEAN_INFLIGHThypothesis. - Multi-Viewport Visual UI Auditing (
auditPageWithArgon): For web applications,@krusch/toolkit/browserlaunches headless Chrome across375px,768px, and1440pxviewports, captures console/page/network errors, and routes the screenshots to Gemini 4 Argon (gemini-4-argon) alongside statickrusch-frontend-mcpdesign-rule checks.
// Excerpt from projects/jean-sre/src/jean_sre_sweep.js:
// Assembling the grounded 3-Tier MCP context frame for each audit checkpoint
function buildProjectAuditContext(projectPath, phase, inflightBlock = '', options = {}) {
const isColabBurst = Boolean(options.isColabBurst);
const sections = [];
// 1. Source-prioritized file tree (src, server, lib, app, bin, scripts first)
const files = listProjectFiles(projectPath);
sections.push(formatBoundedFileTree(files, isColabBurst ? 90 : 50));
// 2. Project manifest, spec.md invariants & pyproject.toml / package.json detection
const specBrief = buildProjectSpecAndDocsBrief(projectPath);
if (specBrief) sections.push(specBrief.slice(0, isColabBurst ? 2600 : 1800));
// 3. Tier 2A AST Symbol Table & Inbound/Outbound Dependency Graph
const astSummary = buildAstContextSection(projectPath, phase);
if (astSummary) sections.push(astSummary.slice(0, isColabBurst ? 2800 : 2000));
// 4. Runtime 120B Teacher-to-30B Student Distilled Prior (Refuted Traps + Tool Queries)
const teacherBrief = !isColabBurst ? loadDistilledTeacherBrief(projectPath, 1200) : '';
if (teacherBrief) sections.push(teacherBrief);
// 5. Live Hermetic Test Suite Execution (node --test / /opt/venv/bin/pytest)
if (shouldIncludeLiveTests(phase, isColabBurst)) {
const testSuiteResult = runProjectVerificationSuite(projectPath, {
force: phase === '1.7_VERIFICATION_AUDIT'
});
if (testSuiteResult?.report) sections.push(testSuiteResult.report.slice(0, 3600));
}
// 6. Live Source Verification placed LAST so ground truth overrides prior summaries
const pendingEvidence = buildPendingIssueCodeEvidence(inflightBlock, projectPath, phase);
if (pendingEvidence) {
sections.push(
`**🔍 Live Source Verification (GROUND TRUTH — overrides prior summary):**\n` +
pendingEvidence.slice(0, isColabBurst ? 8500 : 6500)
);
}
return sections.join('\n\n');
}
3. Stage II: The 11-Checkpoint Investigation & Adversarial Steel-Man Filter
Instead of asking an LLM to audit an entire repository in a single pass—which causes middle-of-context attention decay—our pipeline decomposes every audit into 11 sequential, state-checkpointed phases written to projects/<repo>/.agent/audit/:
| Checkpoint Phase | Primary MCP & Verification Tools | Engineering Objective & Zero-False-Positive Rule |
|---|---|---|
0_ORIENTATION |
krusch_context_retrieve, krusch_fs_page_tree, krusch_git_file_symbols |
Read AGENTS.md, spec.md, manifests, and top entrypoint symbols to establish the project's exact purpose and active invariants. |
1.1_STANDARD_MAPPING |
krusch_fs_locate, krusch_git_search_symbols, read_file |
Compare explicit linter/runtime rules (ESM vs. Python 3.12) against actual AST symbol and import conventions. |
1.2_COMPLIANCE_MATRIX |
krusch_context_retrieve, krusch_context_nudge, read_file |
Evaluate Pass A (Letter), Pass B (Spirit), and Pass C (Project Purpose) with verbatim file:line quotes. |
1.3_CRITICAL_PATH |
krusch_git_dependency_graph, krusch_git_file_symbols, read_file |
Trace the highest-value transaction end-to-end across every hop, reading the full startLine-endLine implementation of each hop. |
1.4_BOUNDARY_AUDIT1.5_ANTI_PATTERN1.6_STRUCTURAL_STRESS |
krusch_git_dependency_graph, krusch_git_semantic_search, krusch_fs_page_tree |
Audit layer coupling, hunt swallowed errors or uncancelled async calls via Hybrid BM25+Vector RRF, and flag files with >3 distinct responsibilities. |
1.7_VERIFICATION_AUDIT1.8_RISK_MAP |
runProjectVerificationSuite (node --test / pytest), auditPageWithArgon |
Execute the live test suite inside the container, map concurrency/transaction lock risks, and verify UI breakpoints. |
1.9_ADVERSARIAL_REVIEW |
read_file <path>:<start>-<end>, dependency_graph <symbol> |
Mandatory Steel-Man Gate: Re-read the full function body and all callers of the top 3 findings and write the strongest possible defense of the current code. |
2_MASTER_BACKLOG |
parseMasterBacklogCards, krusch_context_remember |
Emit structured task cards ([BUG-01], [ARCH-01], [SEC-01]) containing only findings that survived Phase 1.9 with verbatim file:line proof. |
Why Phase 1.9 (Adversarial Self-Review) Eliminates Hallucinated Tickets
The single highest-leverage checkpoint in our pipeline is Phase 1.9_ADVERSARIAL_REVIEW. Before any finding is promoted to master_backlog.md, the auditor is forced to execute an adversarial self-interrogation:
- Never Diagnose From a Truncated Preview: If a symbol preview says
[showing lines X-Y of N], the model is forbidden from claiming validation is missing until it reads the completestartLine-endLinerange. - Steel-Man the Status Quo: For its highest-severity candidate finding, the model must check every caller via
krusch_git_dependency_graphand articulate the strongest engineering case for why the existing code is already safe and compliant. - Explicit Refutation Logging (
4. NOT FLAGGED): Whenever Phase 1.9 proves a candidate bug was a false alarm (for example, an apparent SQL injection that is actually parameterized in the shared wrapper, or an ephemeral test fixture guarded bytest_*prefix checks), it logs the item under4. NOT FLAGGED / REFUTED. As we show in Section 6, those refutations are automatically extracted intoteacher_120b_distilled.jsonso smaller student models never fall into the same trap.
4. Stage III: Spec-First Remediation, Disjoint Subagents & Staged 2PC Execution
Once master_backlog.md produces verified task cards, remediation follows two strict execution tracks depending on blast radius:
- Single-Module Autonomous Healing (
krusch-harness): For localized defects (e.g., addingAbortControllercleanup to a ReactuseEffect, adding an async mutexwithMemoryDbLock, or replacing silentcatch {}blocks), the repair worker checks callers viakrusch_git_dependency_graph, stages a unified diff into PostgreSQL 16 (staged_diffs), runs the hermetic test suite inside an isolated sandbox, executeskrusch_context_nudge({ trigger: "pre_commit" }), and applies to disk via our<32mstwo-phase commit (2PC) journal. - Multi-Module L0 Spec-First Refactor & Disjoint Subagent Fan-Out (
INV-606): When a remediation or architectural upgrade touches 3 or more modules, the orchestrator freezes `spec.md` with exact function signatures and JSON return contracts ({ ok, tool, summary, data }), fans out disjoint 1–2 file slices to parallel subagents under aSTRICT BOUNDARY RULE(each verified locally vianode --checkandnpx oxlint), and wires shared choke points (src/server.js,src/agent/tools.js,client/src/App.jsx) exclusively in the parent orchestrator.
5. Stage IV: The Cross-Family "Referee over Judge" Convergence Gate
Even with a frozen specification and passing unit tests, how do you guarantee an autonomous coding agent didn't take a subtle architectural shortcut—like hardcoding a return value or bypassing a shared toolkit primitive?
In @krusch/toolkit (lib/spec-verifier.js), we implemented our Dual-Zero-Finding Convergence Gate grounded in Specification-First Convergence (arXiv: 2608.12440) and Agents Catching Agents: Shortcut Cascades (arXiv: 2608.03744):
- Orthogonal Model Families Across Separate GPU Nodes: When
qwen3-coder-30b-moeon Node-02 (Dual RTX 2080 Ti NVLink) generates or audits code, verification is dispatched over our10GbEfabric to Node-01 (Dual RTX 3060) runninggemma4:e4bandhermes3:latest—or escalated togpt-oss:120b(Blackwell G4) andgemini-4-argon. - Zero Reasoning-Trace Sharing (
runRefereeAuditPass): Unlike a standard LLM judge that reads the coding agent's conversation history, the Independent Referee receives only the frozenspec.mdand the raw source files attemperature: 0.0. It is forced to re-derive every invariant line-by-line from scratch. - Contagion Detection & Dual-Zero Convergence:
evaluateRefereeConsensuscompares the primary auditor's findings against the Referee's independent re-derivation. If the primary auditor reports0findings while the Referee discovers a defect,hasContagion: trueis triggered.checkConvergenceblocks completion until two consecutive passes yield zero findings (consecutiveZeroPasses >= 2) and 100% of unit/E2E tests pass.
// Excerpt from lib/spec-verifier.js:
// Detecting "Shortcut Contagion" (arXiv: 2608.03744) between Primary Auditor and Isolated Referee
export function evaluateRefereeConsensus(auditorFindings = [], refereeFindings = []) {
const normAud = Array.isArray(auditorFindings) ? auditorFindings : [];
const normRef = Array.isArray(refereeFindings) ? refereeFindings : [];
const auditorClean = normAud.length === 0;
const refereeClean = normRef.length === 0;
const missedByAuditor = normRef.filter(rf =>
!normAud.some(af => af.file === rf.file &&
(af.invariant === rf.invariant || af.description === rf.description))
);
// Contagion occurs when the primary auditor rubber-stamps code that the zero-trace Referee catches
const hasContagion = auditorClean && !refereeClean;
const consensus = (auditorClean && refereeClean) || missedByAuditor.length === 0;
return {
hasContagion,
consensus,
missedByAuditor,
combinedFindings: [
...normAud,
...missedByAuditor.map(item => ({ ...item, source: 'referee_independent' }))
]
};
}
6. Stage V: What Makes the Training Data High Quality? Inside Our SFT Curation & Rejection-Sampling Engine
Most organizations that attempt to fine-tune local coding models make a fatal mistake: they dump raw, unfiltered chat logs into a JSONL file and train on their own model's hallucinations, verbose filler, and machine-specific file paths. In our pipeline (projects/jean-sre/scripts/export-120b-distillation.js and src/harness.js), no trajectory enters the training set unless it survives a multi-stage deterministic quality gate.
A. The 5-Gate Training Data Quality Pipeline
- Gate 1 — Hermetic Test & Linter Verification (Execution Truth): Candidate code remediations (including Gemini 4 Argon Golden Remediations) are only recorded after the patched repository passes
100%of its automated test suite (node --testor/opt/venv/bin/pytest) and zerooxlinterrors inside an isolated container. - Gate 2 — Hard Anti-Hallucination & CoT Leak Filter: Both
recordTeacherSftTurnandscoreElitistTrajectoryimmediately discard (score = 0) any trajectory that contains aHALLUCINATION FLAG, leaked<think>reasoning tags, or truncated responses under 400 characters. - Gate 3 — Multi-Signal Elitist Trajectory Scoring (
scoreElitistTrajectory): Every candidate 120B/Argon turn is scored on concrete grounding signals:+10 pts— Argon Golden Remediation Provenance: End-to-end patches synthesized or verified bygemini-4-argonand confirmed by unit tests.+3 pts— Verbatimfile:lineCitations: Requires at least 3 exact coordinate citations (e.g.,src/unified-retrieval.js:37or#L190-L293) verified against the hydrated AST symbol table.+2 pts— Concrete Code Diffs: Contains>=2fenced implementation or unified diff blocks.+2 pts— Structured Audit Taxonomy: Emits formal card identifiers ([BUG-XX],[SEC-XX],[PERF-XX],[ARCH-XX],[TEST-XX]).+2 pts— Explicit Invariant & Refutation Grounding: Explicitly cites project invariants (INV-XX) or formal hypothesis verdicts (VERIFIED/REFUTED).
- Gate 4 — Stratified Project:Phase Bucket Capping (Anti-Skew): To prevent large repositories from dominating gradient updates,
exportElitist120bDatasetfilters forelitist_score >= 5, sorts descending by quality score, deduplicates by content signature, and caps selection to the top 4 trajectories per(project, phase)bucket across 83 distilled targets—while oversampling Argon Golden Remediations2xin the 120B teacher set and3xin the 30B student set. - Gate 5 — Portable Path Normalization (
normalizePortableSftText): Raw container logs contain machine-specific paths like/home/agent/homelab/projects/krusch-git/or/home/krusch/homelab/projects/. Before writing to JSONL,normalizePortableSftTextstrips host-specific prefixes into repo-relative coordinates and$WORKSPACE_ROOTacross all 9,377 SFT rows so the fine-tuned weights generalize cleanly to any environment.
// Excerpt from projects/jean-sre/scripts/export-120b-distillation.js:
// Rejection-sampling & scoring 120B and Gemini 4 Argon trajectories for Elitist QLoRA
export function scoreElitistTrajectory(row) {
if (!row || !Array.isArray(row.messages) || row.messages.length < 3) return 0;
const assistant = String(row.messages[2]?.content || '');
if (assistant.length < 400) return 0;
if (assistant.includes('HALLUCINATION FLAG') || assistant.includes('<think>')) return 0;
let score = 0;
// 1. Argon-verified + unit-test-gated golden remediations get maximum priority
if (row.source === 'argon_golden_remediation' || String(row.teacher_model || '').includes('argon')) {
score += 10;
}
// 2. Verbatim file:line citations (e.g., src/foo.js:42 or #L10-L25)
const lineCites = assistant.match(/[a-zA-Z0-9_./-]+(?::\d+|#L\d+)/g) || [];
if (lineCites.length >= 3) score += 3;
else if (lineCites.length >= 1) score += 1;
// 3. Concrete fenced code blocks or unified diffs
const codeBlocks = assistant.match(/```[\s\S]*?```/g) || [];
if (codeBlocks.length >= 2) score += 2;
else if (codeBlocks.length === 1) score += 1;
// 4. Structured audit IDs & explicit invariant/refutation checks
if (/\[(?:BUG|SEC|PERF|ARCH|TEST)-\d+\]/.test(assistant)) score += 2;
if (/\b(?:REFUTED|VERIFIED|INV-\d+)\b/.test(assistant)) score += 2;
return score;
}
B. Runtime Teacher-to-Student Distillation (teacher_120b_distilled.json)
Training weights via QLoRA is only half of the distillation story. Between fine-tuning runs, distillTeacherKnowledgeForStudent mines every completed 120B and Argon audit directory to generate a compact per-repository teacher_120b_distilled.json artifact (capped at 850–1,200 characters) that is injected directly into the local 30B Student's prompt on every sweep:
- Refuted False-Positive Traps (
DO NOT RE-FLAG): Up to 8 verified non-issues extracted from4. NOT FLAGGEDandHYPOTHESIS VERDICTblocks so the 30B Student never wastes tokens re-investigating safe code. - Teacher-Verified Backlog Anchors: Top 6 confirmed task cards with exact
file:lineevidence. - Golden MCP Tool Trajectories: Up to 6 high-yield repo-specific MCP queries (e.g.,
krusch_git_dependency_graph,krusch_git_search_symbols,krusch_fs_page_tree) that proved effective during 120B exploration.
7. Stage VI: Dual-Track QLoRA Fine-Tuning on 96GB NVIDIA Blackwell G4
When we provision our on-demand L2.5 Colab Burst Tier (./scripts/colab-ollama-g4.sh train-both)—backed by a 96GB NVIDIA RTX PRO 6000 Blackwell Server Edition (G4, sm_120)—we execute a Dual-Track QLoRA Fine-Tuning Pipeline that upgrades both our 120B Burst Teacher (krush420/gpt-oss-120b-sre-lora) and our 30B Local Workhorse Student (krush420/qwen3-coder-30b-sre-lora) on our exact krusch-stack conventions.
| Training Dimension | Track 1 · 120B Elitist Teacher QLoRA ( krush420/gpt-oss-120b-sre-lora) |
Track 2 · 30B Local Student QLoRA ( krush420/qwen3-coder-30b-sre-lora) |
|---|---|---|
| Base Model Architecture | openai/gpt-oss-120b(116.8B Total MoE · 5.1B Active · 128 Experts/Layer) |
Qwen/Qwen3-Coder-30B-A3B-Instruct(30.5B Total MoE · 3.3B Active · Local Workhorse) |
| Training Corpus & Oversampling | 2,208 Elitist Trajectories (elitist_120b_teacher · Score >=45 + 2x oversampled 401 Argon Golden Remediations) |
17,899 Total SFT Corpus (sovereign-code-audit-sft across 15,290 11-phase audits + 2,208 Elitist + 401 Argon Golden Remediations) |
| Quantization & VRAM Strategy | Native Blackwell 4-bit MXFP4 ( Mxfp4Config(dequantize=False) keeps all 128 MoE experts frozen in ~60.8GB VRAM via Pre-LN residual gradient flow on 96GB G4) |
4-bit NF4 Attention + BF16 3D MoE ( BitsAndBytesConfig with torch.bfloat16 compute dtype and gradient checkpointing) |
| LoRA Target Modules & Rank | BF16 Attention Projections Only:q_proj, k_proj, v_proj, o_proj( r=16, alpha=32, dropout=0.05 · 11.94M trainable params, 46MB adapter) |
BF16 Attention Projections:q_proj, k_proj, v_proj, o_proj( r=16, alpha=32, dropout=0.05 · 25.17M trainable params, 15MB adapter) |
| Convergence & Token Accuracy | Loss 7.448 → 1.737 (eval_loss=1.8054)Token Accuracy: 20.11% → 64.89% |
Loss 0.731 → 0.214 (eval_loss=0.3358)Token Accuracy: 81.07% → 93.36% (90.35% eval) |
| Deployment Target & Throughput | ollama create gpt-oss-sre:120b -f Modelfile on 96GB Blackwell G4 (~117.7 tok/s over SSH loopback tunnel) |
ollama create qwen3-coder-30b-sre:latest -f Modelfile on Node-02 Dual RTX 2080 Ti NVLink (124.2 tok/s · 100% GPU resident) |
Why Native Blackwell MXFP4 Makes Single-GPU 120B QLoRA Possible
De-quantizing a 120B-parameter model into 16-bit precision requires ~240GB of VRAM—normally demanding a multi-GPU H100/A100 node. Because NVIDIA's RTX PRO 6000 Blackwell (sm_120) natively supports 4-bit microscaling floating-point (MXFP4), our Track 1 script passes Mxfp4Config(dequantize=False) to Hugging Face transformers:
- All 128 routed MoE experts per layer stay frozen in hardware-native 4-bit MXFP4, occupying just
~60.8GBof VRAM. - We attach BF16 LoRA adapters (
r=16, alpha=32) exclusively to the attention projection matrices (q_proj,k_proj,v_proj,o_proj), wrapping the frozen MXFP4 MLP forward pass so gradients flow cleanly through the Pre-LN residual stream across all 36 layers. - Trainable parameters stay at
11.94M(46MBsafetensors adapter footprint), allowing the entire 120B Teacher fine-tune to run inside a single 96GB G4 GPU and layer directly onto Ollama's existinggpt-oss:120bbase blob without duplicating 65GB of disk storage.
# Excerpt from projects/jean-sre/scripts/train_gpt_oss_120b_qlora_g4.py:
# Freezing 128 MoE experts in native Blackwell 4-bit MXFP4 while training BF16 Attention LoRA
def build_quantization_config():
from transformers import Mxfp4Config
return Mxfp4Config(dequantize=False)
model = AutoModelForCausalLM.from_pretrained(
"openai/gpt-oss-120b",
quantization_config=build_quantization_config(),
device_map="auto",
torch_dtype=torch.bfloat16,
attn_implementation="sdpa",
trust_remote_code=True,
)
model = prepare_model_for_kbit_training(model, use_gradient_checkpointing=True)
# Target BF16 attention projections only so the 128 MXFP4 experts remain frozen in 4-bit (~60.8GB VRAM)
lora_config = LoraConfig(
r=16,
lora_alpha=32,
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM",
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
)
The Compounding Flywheel Effect
Because the training examples are constructed using the exact same buildProjectAuditContext MCP hydration frame that the models see at inference time, the fine-tuned 30B Student and 120B Teacher don't just learn generic Python or JavaScript syntax. They learn how to read krusch-git AST dependency graphs, how to honor krusch-context-mcp invariants (`INV-*`), how to import `@krusch/toolkit` primitives instead of writing inline boilerplate, and how to refute false positives during Phase 1.9 adversarial reviews.
Every time a frontier escalation (`gemini-4-argon`) or a 120B burst audit solves a hard bug and passes 100% of unit tests, that trajectory permanently upgrades our local weights—turning cloud inference from a recurring operating expense into a compounding capital asset.
Explore the Full 2026 Sovereign AI Tech Stack
npx krusch-stack init · 7 MCP Servers · @krusch/toolkit · 2PC Execution Harness