Chunk the document, embed the chunks, return the nearest neighbours of the query. That pipeline assumes a number carries its meaning with it. On a 780-page government finance report where 86.8% of lines are table rows, it does not — the unit is declared in a header a median of 13 lines above the figure, so a chunk boundary between the two turns one currency scale into another a hundred times larger. Swap the retriever for three deterministic tools an agent can call — search, outline, read these lines — and accuracy on 51 verified questions goes from 15.7% to 58.8%. Plain keyword search gets 51.0%, and the authors say so.
This document declares its scale once, in a heading — "(Rs. in lakh)" over one statement, "(Rs. in crore)" over the next — and every figure below inherits it. The two differ by a factor of a hundred. Chunking cuts that inheritance, and it does it quietly: the chunk is still fluent, still relevant, still ranks well. Flip between the two chunking strategies and read what actually reaches the model.
Notice what the fix does not fix. Table-aware chunking takes unitless chunks from 67.4% to 0.3%, and the share of chunks with no year attached sits at 27.2–30.1% no matter what chunk size you choose. That is the general shape of the problem: chunking commits to one partition of the document before it has seen a single question, and no chunk size is right for every field a question might need.
The alternative deletes the vector store entirely and hands the model three deterministic functions of the document text, exposed over the Model Context Protocol so any agent can call them. None of them ranks anything. Step through them and see what each one does to the same lookup.
The load-bearing idea is the third one. Chunk-and-embed decides where the document divides before any question exists; a bounded read lets the agent pick its span after it has seen where the evidence sits — so when a figure needs the heading thirteen lines above it, the agent just extends upward and takes it. The trail it leaves is a list of line ranges you can re-read, rather than a similarity score you cannot argue with.
Fifty-one questions, verified against the document by hand, sorted into what they ask for. Every system uses the same generator model, so the only variable is how evidence reaches it. Step through the question types — the headline gap is not spread evenly, and one category shows no gap at all.
Aggregation is the clean case: dense retrieval scores 0 of 5, because summing a column requires every row, and top-k by construction returns a handful of rows that individually looked most like the question. Navigation is the honest counter-case — "which statement covers this?" is exactly the query a semantic index is built for, and both systems score 50%. The gap lives in exact figure lookup, not in retrieval generally.
The authors ran the comparison that most papers with this result would leave out, and it changes the conclusion. Cycle the three axes and watch the ranking rearrange.
So the claim that survives is narrower than the headline: this evidence separates embedding-free from embedding-based retrieval on exactness-critical documents, not agents from lexical search. Keyword search costs a third as much, is more often grounded in text it actually retrieved, and is statistically indistinguishable from the agent. If your documents look like this one, the cheapest correct move may be to turn the embeddings off before you build anything.
None of this says embeddings are bad. It says they lose exactness on documents whose meaning is carried by layout — where the unit, the year and the column heading live somewhere other than the row you matched. Set what your corpus looks like and what you ask of it.
Two things are worth stealing regardless of which retriever wins for you. Measure how often a retrieved chunk still contains the unit, the year and the column heading its numbers depend on — that is a one-afternoon audit and it is the number that predicts this failure. And keep the line ranges your answer came from, because "here are the seven lines I read" is a debuggable artefact and a similarity score is not.