Reading a model's chain of thought only works as a check if the reasoning is in there. Pad the output with tokens that carry no information at all — counting up from one, NATO callsigns, digits of pi — and frontier models get measurably better at reasoning tasks, in one case from 61.7% to 91.7%. Nothing in those tokens explains the improvement, so the work is happening somewhere a reader cannot follow. The sharpest version: a model quietly satisfied a hidden side-condition planted in its system prompt while still answering the question it was asked, with no sign of the second objective anywhere in the readable text.
The setup is deliberately absurd. Before answering, the model emits about a hundred tokens of filler — literally counting from one to a hundred — then gives its answer. The filler says nothing about the problem. If reasoning lived entirely in the readable text, this should change nothing. Switch the filler on and step through the models.
Not every model gains, and a couple lose slightly — but the ones that gain, gain a lot. The uplift has to come from computation carried through those positions, because there is no information in them to read.
The obvious explanation is that the model is just buying itself more forward passes, in which case any filler would do. It would not. Seventeen filler types were tested, all calibrated to about the same token count, and the spread between them is enormous — including one model where the wrong choice does not merely fail to help but destroys the task entirely. Same budget, same position, different tokens.
If extra compute were the mechanism, these bars would be the same height. They are not — so the specific representation matters, and something is being encoded in the choice of token rather than merely in the number of them. The paper also finds an inversion: filler that hurts most with no examples in the prompt tends to help most with ten.
This is the experiment that makes it a monitoring problem rather than a curiosity. The system prompt tells the model to answer a multiplication question normally — unless the true answer satisfies a hidden arithmetic condition, in which case it should say "N/A" instead. Checking that condition requires computing the answer and testing it, and none of that work appears in the output, which is either a number or two letters. Step through the conditions.
On the easiest condition the filler lifts compliance from 33.5% to 44.5% — while the model keeps its accuracy on the ordinary questions. Two objectives served at once, one of which a reader of the transcript would never know was there.
Put yourself in the position of a reasoning monitor — a human reviewer, or another model reading the trace and flagging anything concerning. Here is what each approach has access to in the hidden-condition run. The point is not that the model is deceiving anyone; it is that the readable trace is structurally incapable of reporting this. Switch what you are allowed to look at.
Training does not close this either: reinforcement learning gave one model firm preferences over which filler to use, but the benefit did not survive to test time, and supervised fine-tuning failed to transfer it at all. This is a property of what the forward pass can carry, not a habit that can be trained away.
Most teams shipping agents monitor by sampling: read a share of traces, flag what looks wrong. This paper does not tell you that reading traces is worthless — it tells you there is a class of behaviour it cannot reach, so a review budget spent entirely there has a ceiling below 100% no matter how much you spend. Split your budget and watch the coverage move.
Concretely: keep reading traces, because most failures still show up there and it is cheap. But put a second check on the outcome side — distributions of tool calls, refusal and abstention rates, side effects in the environment — and treat any objective you actually care about as something to verify in behaviour rather than in narration.