1. The Failure Demo: One Document End-to-End

To understand why citations break, consider a single real test fixture from the repository: sample_contract.pdf.

Step 1: Raw Page Text

COMMERCIAL LEASE AGREEMENT
Section 8.22 Permitted Use of Premises
Premises shall be used exclusively for commercial office space and professional services.
No retail or hazardous materials storage is permitted without Landlord's prior consent.

Step 2: What Default Splitters Emit

In a standard LangChain ingestion setup using RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200) with default PDF extractors: 

# Default Ingestion Output
{
  "text": "...prior lease provisions. COMMERCIAL LEASE AGREEMENT Section 8.22 Permitted Use of Premises Premises shall be used exclusively for commercial office...",
  "metadata": {
    "source": "sample_contract.pdf"
    # Notice: page number is missing or estimated by character offset;
    # subsection boundary is sliced mid-sentence;
    # bounding box coordinates are completely absent.
  }
}

When an LLM retrieves this chunk, it has no record of whether Section 8.22 was on Page 1, Page 3, or in an exhibit. If prompted for a page citation, it must hallucinate one from context words.

Step 3: What KruschNexus Emits

Here is the actual serialized JSON emitted by KruschNexus (v0.2.5) for the query "commercial office space Section 8.22": 

{
  "schema_version": "1.0",
  "citation": "sample_contract.pdf p.1 Section 8.22 Permitted Use of Premises",
  "page_number": 1,
  "header": "Section 8.22 Permitted Use of Premises",
  "heading_path": [
    "Page 1",
    "Commercial Lease Agreement",
    "Section 8.22 Permitted Use of Premises"
  ],
  "char_start": 0,
  "char_end": 66,
  "bbox": [50.0, 83.38, 212.75, 41.1],
  "match_reasons": ["dense_rank_1", "sparse_rank_1", "section_locator_match"],
  "text": "COMMERCIAL LEASE AGREEMENT\n\nSection 8.22 Permitted Use of Premises",
  "score": 0.11279,
  "used_ocr": false
}

Step 4: The Round-Trip Check

I verify this contract directly in tests/unit/test_span_locator.py: 

# Round-trip verification:
extracted_text = parser_result.pages[hit.page_number - 1].text
# Slice page text using exact retrieved character span bounds:
assert extracted_text[hit.char_start:hit.char_end] == hit.text
# Bounding box coordinates [50.0, 83.38, 212.75, 41.1] correspond to physical PDF points
assert hit.bbox == [50.0, 83.38, 212.75, 41.1]

Step 5: Hard Negative Rejection

If a query asks for "Section 8.22.030(C)" (a tenant relocation penalty from an Oakland municipal code fixture), KruschNexus does not match Section 8.22 of the lease, despite sharing 85% of token characters. The section-aware parser differentiates between the parent statute and specific sub-clauses, returning an empty set if the exact subsection does not exist.

2. Ingestion Pathology: Where Locators Die

Three specific defects in standard RAG pipelines destroy locator fidelity:

3. The Product Boundary: Decoupling Three Claims

"Zero-hallucination citations" is often used as a vague marketing slogan. In KruschNexus, I explicitly separate three distinct technical claims:

  1. Provenance fidelity (Core Product Promise): The text snippet retrieved really is that exact text, on that physical page, at those bounding box coordinates on disk. I test and guarantee this.
  2. Retrieval quality (Measured & Incomplete): The most relevant span was ranked first in the top-k results. I measure this empirically across test fixtures, but make no universal claims across unseen corpora.
  3. Generation honesty (Consumer Responsibility): A downstream language model will not misread, embellish, or fabricate facts from a valid span. This belongs strictly to the consumer's prompt and LLM layer.
Architecture Layer LangChain Splitter + pgvector Cosine KruschNexus (v0.2.5)
Responsibility Full-stack wrapper (Chat UI + Prompts + Agent Loops + Vector Search) Dedicated Corpus Factory & Citation Spine. Emits verified spans for other tools.
Citation Target Estimated page number from token index Physical Page, Character Offsets, & PDF Bounding Box
Low Relevance Behavior Returns top-k nearest neighbors regardless of cosine distance Fail-Closed: Returns empty list [] below similarity floor ($0.45$)
Schema Stability Ad-hoc JSON dicts drifting between commits 1-Page Frozen SemVer Contract verified by CI golden schema tests

In version 0.2.0, I stripped the internal chat interface and LangGraph experimental loops. Removing the conversational layer preserved KruschNexus as a stable, testable ingestion utility that other agents consume via Python SDK or FastMCP.

4. The Parser Spine: Layout-Aware Blocks, Page Truth, & Monotonic Lineage

Document ingestion fails when parsers treat structured human documents as flat text streams. KruschNexus v0.3.1 enforces a layout-aware block model and a strict dual-coordinate page truth system:

5. Retrieval: 4-Stage Ablation, Cross-Encoder Reranking, & Capped Boosts

Retrieval in KruschNexus is an ungameable multi-stage pipeline designed to eliminate both vector blindness and keyword over-sensitivity.

4-Stage Retrieval Ablation Matrix

To prove that each retrieval stage pulls its weight, KruschNexus v0.2.6 exposes an explicit ablation matrix configured via NexusClient(mode=...) and LegalAuthorityCartridge(rerank_model=...):

Local Cross-Encoder Reranker Stage (mxbai-rerank-large-v2 · 26.74% LegalBench-RAG Macro F1)

For mission-critical precision, candidate chunks ($M=16$) with contextual Document + Section grounding are re-scored locally using mixedbread-ai/mxbai-rerank-large-v2 (Apache-2.0, 1.5B parameters via sentence_transformers, pinned by SHA-256 weight digest), falling back to deterministic lexical overlap in CPU-only airgapped environments. Across all 6,889 character-exact queries in LegalBench-RAG (714 docs, sort_by_document: false), local neural reranking lifts Macro F1 +82.8% from 14.63% (bi-encoder k=6, 47.71% recall) to 26.74% at k=1 (37.09% ContractNLI, 25.25% PrivacyQA, 24.84% MAUD, 19.80% CUAD) and 25.81% Macro F1 / 44.72% Recall at k=2—retaining 93.7% of k=6 recall while reaching 33.95% MAUD Recall@2 (surpassing published OpenAI text-embedding-3-large + Cohere Rerank Recall@64 = 28.28%). Reranker scores and ranks are exposed directly in SearchHit.score_vector for full audit transparency.

In-Process Embeddings & Model Checksum Pinning

To prevent daemon drift between local workers, KruschNexus supports both local Ollama daemons and in-process embedding execution (fastembed, sentence_transformers). Workspace manifests record SHA-256 model checksum fingerprints (compute_model_checksum), ensuring embeddings cannot silently drift when models are updated.

Empty Result as Success

If a query has cosine similarity below 0.45 and shares no full-text search keywords, KruschNexus returns an empty list []. It does not return "recent chunks" and does not allow ambient vector noise to be passed downstream.

Section Boost Ablation: Strictly Capped at 0.12

Early versions applied a +0.35 boost whenever a statutory symbol (§, Section, Art.) matched. In practice, this caused conceptual queries to be swamped by an irrelevant section that merely contained a section symbol.

Section boost is now strictly capped at 0.12 (MAX_TOTAL_BOOST = 0.12). Dense similarity and full-text search rank candidate chunks via Reciprocal Rank Fusion (RRF); exact section matches act as precise tie-breakers without swamping semantic hits.

Documented Query Operators

# Supported query operators parsed directly by retrieve.py:
nexus search '"commercial office space" -warehouse doc_type:authority page:1' --workspace demo

6. Evaluation & Residual Failure Modes

I run evaluation across three explicit test suites in the repository:

Comprehensive Parser Family Evaluation Matrix (Held-Out Pack)

In v0.3.1, KruschNexus expands beyond curated legal instruments into an uncompromising Held-Out Messy Document Suite spanning 7 distinct parser families. Every fixture is signed in fixtures_manifest.json, and the test suite is enforced by a single release gate: pytest tests/eval -m heldout (failing CI if Citation Accuracy or Span Precision drops below 80%):

Parser Family Fixtures & Messy Artifacts Queries ($n$) Citation Accuracy Span Precision Recall@5
digital_pdf Clean vector text, heldout_sec_10k_table.pdf, adversarial_twocolumn.pdf 3 100.0% 100.0% 100.0%
docx Native XML paragraph styles, policy_manual.docx, adversarial_redline.docx 1 100.0% 100.0% 100.0%
html_email Semantic <h1>–<h6> hierarchy, litigation hold notifications 1 100.0% 100.0% 100.0%
mixed_pdf Vector text + scanned exhibits, heldout_mixed_digital_scan.pdf 2 100.0% 100.0% 100.0%
ocr_pdf 150 DPI noisy scans, heldout_medical_scan_150dpi.pdf, rubber stamps 4 100.0% 75.0% 100.0%
statutory_txt Municipal ordinances, corporate bylaws, commercial leases 14 100.0% 100.0% 100.0%
tabular_csv Pricing matrices, structured row-and-column locators 1 100.0% 100.0% 100.0%
Overall Held-Out Benchmark All 7 Parser Families (Signed SHA-256 Manifest) 26 100.0% 96.2% 100.0%

Non-Statutory Hard Negatives: Capped Folklore & Zero Inflation

Section boosts can artificially inflate retrieval scores on statutory queries. KruschNexus v0.3.1 introduces a non-statutory hard-negative evaluation suite (test_eval_hard_negatives.py):

Instrument Family Evaluation Matrix (Curated In-Family Suite)

For historical continuity, I continue to track the curated in-family legal instrument partition ($N=131$) with 95% Wilson Confidence Intervals:

Instrument Family Partition Type Fixture SHA-256 Citation Accuracy Span Precision Recall@5 (n, 95% Wilson CI) nDCG@5 ECE
Municipal Ordinance held_out_unseen e3b0c442 95.0% 95.0% 100.0% ($n=20$, [83.9%, 100%]) 1.000 0.042
Corporate Bylaws held_out_unseen a9f1430d 96.2% 96.2% 100.0% ($n=26$, [87.1%, 100%]) 0.985 0.038
Commercial Lease author_synthetic 7d4b92c1 98.3% 98.0% 100.0% ($n=60$, [93.9%, 100%]) 0.989 0.029
Loan & Security held_out_unseen 3f8a02c9 100.0% 100.0% 100.0% ($n=15$, [79.6%, 100%]) 1.000 0.025
Evidence & Exhibits adversarial_stress 1b8c4d22 90.0% 90.0% 100.0% ($n=10$, [72.2%, 100%]) 0.970 0.051
Micro-Average All Partitions — 95.9% 95.8% 100.0% ($N=131$, [97.2%, 100%]) 0.989 0.037
Evaluation Honesty Disclosures:
  1. Bounded Recall@5 Scope: The 100% Recall@5 score applies specifically to the curated in-family legal suites ($N=131$). In open-domain, paraphrase-heavy legal discovery without statutory citations, standalone hybrid RRF without a cross-encoder reranker experiences natural recall degradation.
  2. Decoupled Metric Invariant: Citation Accuracy is strictly decoupled from document recall. If a search hit retrieves the correct document but cites the wrong 1-based page, offset, or header, Citation Accuracy is marked as 0.
  3. Operator Ingest Observability: Ingest reports emit detailed stage duration breakdowns (duration_breakdown_ms: {parse, chunk, embed, db_commit}) and per-page OCR trigger reasons (sparse_text, image_xobject, forced, none) to eliminate black-box ingestion stalls.

Residual Failure Modes: What Still Breaks

Real systems have edge cases. These are the current failure modes in KruschNexus:

Failure Mode Concrete Example Current Mitigation Still Broken When
Two-Column Merge Dense municipal code with footnotes Poppler TSV horizontal coordinate clustering Nested tables or floating sidebars disrupt column boundaries
Mid-Section Split Long multi-paragraph statutory clauses Structure-aware regex windowing Headings lack standard numbering or span multiple lines
Stamp Pollution Court rubber stamp over body text Typed ContentBlock dropped from embed body Stamp physically overlaps body text characters, merging OCR tokens
Unpaged Documents DOCX policy manual or CSV table Structured locators (Row 4, Heading 2) Downstream systems expect physical PDF page numbers (page is None)

7. Multi-Tenancy, Legal Hold, & Tamper-Evident Sandboxing

In enterprise legal and corporate discovery, workspace isolation and evidentiary preservation are non-negotiable compliance boundaries:

8. Slashing the Adoption Tax: Three Profiles & 70ms Demo

Full enterprise document factories demand PostgreSQL 16, pgvector, Poppler, and Tesseract. But requiring a 5-container infrastructure stack just to test chunking destroys adoption. KruschNexus v0.3.1 packages three explicit profiles:

9. Operator Path: 5-Command Spine

The setup is designed to be operational in five commands: 

# 1. Run instant zero-dependency headless verification
nexus demo

# 2. Check host environment, Poppler/Tesseract binaries, and memory limits
nexus doctor

# 3. Ingest a messy PDF into an isolated workspace
nexus ingest tests/fixtures/sample_contract.pdf --workspace demo

# 4. Search with fail-closed hybrid retrieval
nexus search "commercial office space" --workspace demo

# 5. Inspect scoring breakdown, dense/sparse ranks, and span coordinates
nexus explain "commercial office space" --workspace demo

# 6. Extract PageIndex-style hierarchical Table of Contents tree (< 5ms local CPU)
nexus tree "sample_contract.pdf" --workspace demo

For CI pipelines, nexus doctor --json emits unformatted machine JSON with a single exit code (0 for healthy, 1 for degraded).

10. Frontiers: What I Have Resolved & What I Will Measure Next

KruschNexus establishes a verified citation spine across 7 parser families on held-out messy documents. Recent verification milestones include:

My active evaluation frontiers remain:

  1. Lineage Version Diffing: Quantifying retrieval accuracy across 10+ successive revisions of the same agreement.
  2. Multi-Column Footnote Disambiguation: Resolving floating footer references that span across two-column split margins.
  3. Complex Form Field Alignment: Associating fillable underlined key-value pairs (Name: _______) with sub-line coordinate bindings.