No such package, here's the code.

Three papers with the same shape: the component you assumed was carrying the weight isn't. A coding model's sense of whether a task is even possible, a cache evictor's importance score, and three quarters of an agent's prompt scaffold.

Today's pick

60% vs 27%

impossible tasks answered with working-looking code, versus refused

Refusing the Impossible: A Taxonomy and Benchmark for Code Hallucination in Large Language Models

Ask a coding model for something that cannot exist — a package that was never published, a compressor that shrinks every input, a decider for a problem proven undecidable — and it writes plausible, compiling code anyway on about 60% of attempts, refusing on only 27%. The authors wrote 270 such impossible prompts across six languages and paired them with 91 near-identical solvable versions; no model ever wrongly refused a solvable one, so this is missing scepticism rather than caution overshooting. Two details should change how you test: fabrication clusters on invented package names (98% of the npm prompts, 89% of the Rust ones) far more than on violations of theory, and how the request is phrased predicts the outcome about 2.5 times better than which model answers it. Put a handful of deliberately impossible tasks in your coding-agent eval and score whether the agent says no.

open the explainer → abstract PDF code

32-43%

more tokens per second at matched accuracy, by dropping the scoring step

Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

Every method for shrinking a long-generation key-value cache scores each cached token by how much it will matter later and keeps the winners; this paper shows the score does almost none of the work. Pin the original prompt in cache permanently, then evict everything else uniformly at random, and you match the strongest scoring evictor across four models and five reasoning tasks while serving 32-43% more tokens per second, because there is no scoring pass to run. The explanation is that a reasoning trace is redundant twice over — the model restates what it still needs as it works, and each attention head keeps its own copy — so a random draw retains enough of it; almost the whole apparent gap between rival scorers was just whether their score happened to protect the question. If you tune your own serving stack, pinning the prompt is the change worth making, and the scorer you are paying for may be optional.

open the explainer → abstract PDF code

+11.9 pts

task success, from spending the tuning budget on one slot instead of four

Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents

Automatic prompt optimisers for agents usually evolve the whole scaffold — persona, task strategy, format rules, loop discipline — as one blob of text. Split it into four separately tuned slots and measure each one's contribution and, on a household-chores benchmark, essentially all the value sits in a single slot: the self-correction rules (don't repeat an action that did nothing, verify before finishing), worth +11.9 points alone, while role, strategy and format rules were each worth nothing measurable. Splitting the tuning budget evenly across the four was worse than useless — 16 trial runs per slot fell below the threshold the optimiser needs before it will accept any edit, so every slot froze at its empty starting text — whereas spending half that budget on the control slot alone reached 76.1% against a 64.2% baseline. Before buying more optimisation budget, work out which part of the scaffold is actually failing and spend all of it there.

open the explainer → abstract PDF