Context isn't a summary. It's a filing system.

A double issue today. The first three papers make the same argument from different ends: the context window is not a place to keep a summary, it is a filing system, and what decides whether your agent works is whether it can still lay hands on the exact bytes it saw an hour ago. The second three are about a quieter kind of engineering judgement — the threshold your router defers on, whether a tool call ever stops to ask a human, whether an accepted completion is still in the file an hour later. Each one takes a default nobody checked and replaces it with a measurement.

Today's pick

99.4% vs 88.1%

average exact-answer accuracy on a fact buried in a compacted transcript, against the best of five common context-management strategies (two models, 3,000 attempts)

Addressable Recall Compaction for Long Context-Window Control in AI Agents

Every long-running agent eventually overflows its window, and the standard repairs all destroy information for good: drop the oldest turns, paraphrase them into a summary, or push them to a vector store and hope a similarity search finds them again. This paper keeps every tool observation verbatim in an append-only log under a short id, and when the window fills it swaps the old ones in the transcript for a stub — first lines, last lines, byte count — plus that id and a recall command that puts the exact original back without re-running the tool. Asked later for a fact buried in the compacted history, it answered correctly 99.0% of the time on an 8B model against 79.6% for the best of the five baselines, while the summariser sat at 71.8%. The detail worth stealing: recall has a sweet spot, not a ceiling — accuracy fell when the agent could pull more back, because a transcript refilled with recalled text is just a full window again.

open the explainer → abstract PDF

84% vs 78%

correct answers on a hundred-thousand-document search task, with the agent's tool calls per question falling from 99.1 to 23.9

A New Role for Relevance: Guiding Corpus Interaction in Agentic Search

Most search agents use a relevance score the way a search box does — rank the documents, hand the model the top few — which caps the agent at whatever those passages happen to contain. Questions that need evidence found in one place and checked in another go straight through that cap. This team leaves the agent with plain text search and file reads and spends the relevance scores on the order of the work instead: sort the files so a single-threaded scan hits promising ones first, seed the agent with the ten most relevant paragraphs before it starts, and re-sort the raw matches before the model reads them. Accuracy went from 78% to 84% while tool calls per question fell from 99.1 to 23.9, and all three uses of relevance added accuracy and cut steps together. If you have an embedding model behind a top-k endpoint, this is the cheapest experiment on the list.

open the explainer → abstract PDF

50–87% fewer tokens

trajectory tokens a coding agent burned to find the same code across five models, when repository context was served up front instead of rediscovered with search and read

CodeNib: A Multi-View Data System for Serving Repository Context to Coding Agents

A coding agent starts every task by rediscovering the repository — searching, opening files, following definitions — then throws it away, so the next ticket pays for the same work again. This system builds three reusable views per commit (a text index, embeddings, and a symbol graph), maps every result back to a path and line range so they compose, and serves ranked search, symbol navigation and a bounded slice of context from one runtime. Across five models the cheapest arm that still found the right code used 50–87% fewer trajectory tokens than letting the agent search and read its way there, and refreshing the views after a commit ran a median 8.7× faster for the graph and 25.4× for the embeddings than rebuilding. Copy its honesty too: static symbol navigation matched a live language server on 632 of 1,000 requests, reported alongside the 4.7× latency win rather than behind it.

open the explainer → abstract PDF

49 of 72

model-and-benchmark pairings where the calibrated deferral rule beat the best hand-tuned confidence threshold — against 6 ties and 17 losses

Conformal Cascade: Distribution-Free Accuracy Guarantees for Multi-Tier LLM Inference

Cheap-model-first routing normally escalates when some confidence number — a token probability, or how often repeated samples agree — drops below a threshold you picked by hand, and that number tells you nothing about how often you will actually be wrong. This paper throws the threshold away and uses set size instead: sample the small model 16 times, keep every answer that appeared often enough to clear a cut-off learned from a couple of hundred labelled examples, and accept the small model only when exactly one answer survives. Because the cut-off comes from conformal prediction — a distribution-free technique that converts a held-out sample into an error budget — the answers you accept carry a provable error ceiling rather than a vibe, and it beat the best hand-tuned heuristic on 49 of 72 pairings across four model families. It needs no training and no model internals, so it retrofits onto a two-tier setup you already run, and loosening how many answers you let stand cut expected spend 53% for 5.3 points of accuracy — a dial the threshold never gave you.

open the explainer → abstract PDF

37.2% ask first

share of 1,723 open-source tool-calling applications that stop a tool call and wait for a human before it runs — in the rest, the model invokes any enabled tool unconditionally

An Empirical Study of Model Context Protocol Applications

The protocol that standardises how a model talks to external tools deliberately says nothing about the application consuming them, so every client invents its own answer to whether a human ever sees a tool call before it fires. Mining 1,723 open-source applications, this study finds the ecosystem converged on the easy parts — 85.2% keep server definitions in a config file, 81.1% use an official client library — and barely at all on oversight: 90.8% log tool activity and 77.2% let you switch servers on or off, but only 37.2% block on a human approval, and 20% have neither a blocking prompt nor an on-off list. Logging tells you afterwards and an on-off list only bounds which tools are live, so in most of these applications an injected instruction naming an already-enabled tool simply runs. Two things to take away: put the blocking gate on the tools that write, spend or send rather than on all of them, and if you audit which servers an app depends on, do not key it to a filename — only 30.7% use the obvious one.

open the explainer → abstract PDF

31% get deleted

share of edit histories where the developer removed the accepted completion rather than fixing it — 23% removed it immediately

Learning from 53.6K Real-World Developer Edits of AI-Generated Code

Acceptance rate is what every code-completion product reports, but accepting is a keystroke rather than an outcome, and this dataset follows what happens afterwards by snapshotting the file every time a developer pauses for a second — 5,831 edit histories from 1,141 developers across 20 models. Survival is bimodal rather than average: the median accepted completion still has 63% of its text intact, but the distribution piles up at both ends, either kept nearly whole or thrown out entirely, with 31% of histories containing an outright removal and 23% removing it on the spot. The edits also arrive in a reliable order — removals first at around 24 minutes, then readability repairs at 28, intent-tuning at 49, functional rewrites at 59 — so a telemetry window under fifteen minutes catches the deletions and misses the repairs that salvage a suggestion. Log the trajectory rather than the accept: measure what fraction is still present an hour later, because that separates a suggestion that helped from one that handed someone a cleanup job.

open the explainer → abstract PDF