The standard RAG pattern (chunk, embed, retrieve top-k) breaks on financial documents. A new ArXiv paper measures the failure and proposes a replacement: deterministic agentic operations that expose structure instead of hiding it behind cosine similarity.
Why Embedding Search Fails on Financial Documents
Financial statements, audit reports, and regulatory filings have properties that destroy chunk-based retrieval:
- 86.8% of content lines are table rows in a 780-page government financial report
- Thousands of near-identical figures compete in the same embedding space (₹1,234 appears hundreds of times with different meanings)
- A figure inherits its unit from a header a median of 13 lines above it
- Chunk boundaries routinely separate a number from whether it is in lakh or crore (an error of two orders of magnitude)
Even a table-aware chunker leaves 27-30% of numeric chunks with no fiscal-year header at every chunk size tested. The problem is structural: embedding-based retrieval treats documents as bags of semantic fragments when financial documents are relational databases rendered as text.
The READ Architecture
READ (Reliable Embedding-free Agentic Document-search) replaces top-k retrieval with three deterministic operations exposed over the Model Context Protocol:
- Normalized lexical search: BM25-style term matching with document-specific normalization
- Structural navigation: Follow table headers, section hierarchies, cross-references
- Bounded span reads: Extract contiguous text regions with known start/end offsets
Each operation returns a position in the document, not a similarity score. An agent trajectory becomes a replayable audit trail: “Found ‘₹1,234’ at line 4,523, navigated to table header at line 4,510, confirmed fiscal year 2024-25.”
Operation Flow
# Pseudo-code for a READ agent loop
def answer_financial_query(query: str, document: StructuredDoc) -> Answer:
# Step 1: Lexical search for candidate locations
candidates = document.normalized_search(query.extract_terms())
# Step 2: Navigate to structural context for each candidate
contexts = []
for candidate in candidates:
header = document.navigate_to_header(candidate.line)
fiscal_year = document.navigate_to_temporal_marker(candidate.line)
contexts.append({
"value": candidate.text,
"unit": header.unit,
"period": fiscal_year,
"line": candidate.line
})
# Step 3: Bounded read to verify and extract
for ctx in contexts:
span = document.read_span(ctx.line - 5, ctx.line + 5)
if verify_context(span, query):
return Answer(
value=ctx.value,
unit=ctx.unit,
period=ctx.period,
audit_trail=[candidate.line, header.line, fiscal_year.line]
)The agent does not rank by similarity. It navigates structure, verifies context, and returns a provable path.
Performance on Verified Financial Questions
On 51 verified questions from a 780-page government financial report:
| Retrieval Method | Accuracy | Statistical Significance |
|---|---|---|
| Dense retrieval (baseline) | 15.7% | - |
| Dense retrieval (tuned) | 35.3% | - |
| Agent + top-k tool | 27.5% | - |
| READ (agentic operations) | 58.8% | p_Holm = 2×10⁻⁵ vs baseline |
| BM25 (lexical baseline) | ~58% | Not statistically different from READ |
The result separates embedding-based from embedding-free retrieval, not agentic from lexical. The gain comes from exposing structure, not from iteration alone. An agent given the same loop but a top-k tool reaches only 27.5%.
Interpretability and Audit Requirements
Financial institutions deploying agents for compliance, due diligence, and regulatory analysis face a liability problem: black-box retrieval creates unauditable decisions. When a regulator asks “Why did your system cite this figure?”, the answer cannot be “cosine similarity = 0.87.”
READ’s deterministic operations produce audit trails:
- Line-level provenance: Every cited figure maps to a document line number
- Structural path: The agent’s navigation steps (header lookup, cross-reference resolution) are replayable
- Version control: Operations are stateless functions over immutable document representations
This matters for:
- Regulatory filings: Auditors need to verify every cited figure
- Due diligence: Legal teams need to trace claims back to source documents
- Compliance reports: Regulators demand explainability for automated analysis
Deployment Shape
A READ system has three components:
- Document parser: Converts PDFs/Word docs into structured representations (tables, sections, cross-references)
- MCP server: Exposes normalized_search, navigate_to_header, read_span as tools
- Agent orchestrator: Loops over operations, maintains state, produces audit trails
Latency requirements:
- Document parsing: Batch process, cache results (minutes to hours for 1000-page docs)
- Operation execution: Sub-second per operation (lexical search, pointer navigation)
- Agent loop: 5-10 operations per query (5-10 seconds total)
State management:
- Document structure is read-only after parsing
- Agent state is a stack of (operation, result, line_number) tuples
- No vector index to update or tune
Failure Modes
READ fails when:
- Document structure is ambiguous: Tables without headers, inconsistent formatting
- Cross-references are implicit: “See above” without line numbers or section IDs
- Temporal markers are missing: Figures without fiscal year context
- OCR errors corrupt structure: Misaligned columns, missing table boundaries
Mitigation strategies:
- Pre-flight validation: Reject documents below a structure-quality threshold
- Fallback to human review: Flag ambiguous cases for manual verification
- Hybrid retrieval: Use embedding search for unstructured sections, operations for structured sections
When Embedding Search Still Wins
Embedding-based retrieval remains superior for:
- Semantic similarity: “Find sections discussing climate risk” (no exact keyword)
- Unstructured narrative: Management discussion, risk factors, forward-looking statements
- Cross-document synthesis: “Compare risk disclosures across 50 companies”
The paper’s result is domain-specific: financial documents with heavy table content, precise numeric queries, and audit requirements.
Technical Verdict
Use READ-style agentic operations when:
- Documents are >50% tables or structured data
- Queries target specific figures, dates, or cross-references
- Audit trails and explainability are non-negotiable
- You can afford upfront document parsing cost
Stick with embedding-based retrieval when:
- Documents are narrative-heavy (annual letters, risk discussions)
- Queries are semantic (“What are the main risks?”)
- You need zero-shot deployment on arbitrary documents
- Sub-second cold-start latency is required
The real insight: retrieval architecture should match document structure. Financial documents are relational databases rendered as text. Treat them that way.