The safety check that fires backwards

Three papers today about defaults that keep returning a green light long after they stopped checking anything. A similarity gate scores how much the wording moved and gets read as whether the meaning held. An instruction file grows forever because appending costs a line and deleting costs a proof nobody can run. And step-by-step reasoning is billed on every request when it only pays on some of them.

Today's pick

0 of 56

meaning-breaking edits caught by a shipped similarity gate, which approved all 15 faithful rewrites too

Similarity Gates Approve Reversals: A Validity Audit of Embedding-Cosine Thresholds in Agent Systems

Comparing two blocks of text by embedding cosine and firing below a fixed cutoff is standard plumbing — deduplication filters, semantic caches, drift guards, answer graders — and it gets read as asking whether the meaning still holds. It answers a different question: how much of the wording changed. Reversing an instruction takes one word and leaves roughly 72% of the tokens in place, while an honest restatement replaces nearly all of them, so the gate runs backwards: the audited production guard caught 0 of 56 meaning-breaking edits and scored "withhold the study drug" against "administer the study drug" at 0.9608 on a 0.60 trigger. Add pairs that share wording and flip the decision to the gate's test suite — four of four suites the author opened had none — and stop reading a threshold on it as a check on meaning.

open the explainer → abstract PDF

+226%

growth in the standing instructions an agent instruction file holds over its lifetime, at +4.9 net per commit

Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding

The file holding your coding agent's standing instructions only ever grows. Across 247,694 instruction lifetimes in 1,867 repositories, those files more than tripled in instruction count over their lifetime (+226%), gained 4.9 net instructions per commit, and the older an instruction got the less likely anyone was to delete it — the opposite of what you would see if deletions were driven by instructions going stale. The argument is worth stealing: appending costs one line, but deleting safely means knowing which failure the line was written for, and once that reason is gone you are reduced to testing combinations of what remains. The fix tested here is comments — a note beside each instruction recording the failure that produced it, stripped out before the prompt reaches the model — which held excess instructions to +1.4% where the uncommented arm reached +211.3%.

open the explainer → abstract PDF

+68 pts against +0 pts

what step-by-step reasoning recovers on word problems, against what it recovers on multiple-choice questions, same model

When Chain-of-Thought Helps and When It Hurts: An Empirical Investigation of the Serial-Depth Bottleneck in LLM Reasoning

Step-by-step reasoning gets billed on every request and treated as a general accuracy upgrade. Across three open models and five benchmarks in a pre-registered study, it recovered 54 to 68 points on tasks needing many dependent steps — word problems, competition maths — and 0 to 4.6 points on tasks that are one lookup deep, such as multiple-choice knowledge questions. Inside a single benchmark the same gradient appears: direct answering fell from 45.5% to 15.4% as the number of chained steps rose, while the same model reasoning out loud stayed flat near 85%. Treat it as a way to buy serial steps the forward pass cannot fit, not as a quality setting — and note the one significant regression, a 28.7-point drop on code generation from the smallest model.

open the explainer → abstract PDF