/p/2026-08-06 · explainer
Paper explainer · 2608.02639 · Anand & Chattaraj

Your system prompt is fighting itself.

One system message that asks for valid JSON, a word cap, three citations and a fixed tone is four instructions, and a model obeys each of them far less often together than alone. Stack twenty machine-checked rules and the per-rule follow rate falls from about 96% to 60.4% on Claude Sonnet 4.6, 43.3% on Gemini 2.5 Flash and 20.1% on GPT-5-mini. The collapse is structured, not noise: 15 pairs of these rules cannot both be satisfied, and "output valid JSON" alone is incompatible with nine others. A single rewriting pass over the stack, computed once and reused, returns up to 11 points — to the weak models, and to nobody else.

01 · The problem

Every rule you add makes the others less likely to hold

The usual mental model is that instructions are independent: each one either lands or it doesn't, and adding a twenty-first shouldn't disturb the first twenty. The benchmark here tests that directly. It has 22 atomic instructions whose satisfaction can be checked by a program rather than a judge — valid JSON, at least three bullets, under fifty words, begin with "Answer:", include a confidence score, and so on — and it samples random stacks of them at sizes one, five, sixteen and twenty, roughly 300 trials per cell. Step through the stack sizes and watch what happens to the rate at which any given rule in the stack is honoured.

Interactive · grow the stacktap a stack size
050%100%


    

Two details make this worse than it first reads. The rate shown is per instruction, so at twenty rules and a 60% follow rate the odds that a single response satisfies the whole stack are far below 60%. And the degradation is non-linear — flat-ish through five, then a cliff — which is exactly the shape that hides in testing, because the prompt you tested with eight rules behaves nothing like the prompt you shipped with sixteen.

02 · The mechanism

Some of your rules cannot both be true

The collapse is not the model getting confused by length. Of the 231 possible pairs in this instruction set, 15 are logically unsatisfiable — no response exists that satisfies both — and they are not exotic. A JSON object cannot also carry a markdown heading, so asking for structured output silently voids every section header you also requested. A lowercase-only response cannot begin with "Answer:". Beyond those, about 12% of the pairs that are jointly satisfiable still fail more often together than the two rates separately predict. Tap an instruction to see everything it cannot coexist with.

Interactive · the conflict maptap any instruction
rules it cannot coexist with
rules it is compatible with
How many rules each instruction rules out15 impossible pairs across 22 instructions
0510

One instruction accounts for nine of the fifteen impossible pairs. Format demands are hubs because they constrain the whole surface of the response; lexical demands like "avoid the word however" are cheap because they constrain almost nothing.

03 · The method

Compile the stack once, reuse it on every query

The remedy tested here is deliberately unambitious: no training, no routing, no per-request cost. One model call takes the raw instruction dump and rewrites it — clustering instructions by category and merging the overlapping ones, resolving conflicts by writing an explicit precedence line, and reformatting the result as a numbered checklist ordered reasoning first and formatting last. The output is computed once per instruction set and pasted into every request after that. Step through the three transforms on a stack of eight.

Interactive · the three transformstap a transform

      

    

The obvious objection is that the compiled prompt is simply longer, or differently ordered, and any of that could explain a gain. The paper runs both controls: a token-padded version of the raw dump that adds words without restructuring, and a deterministic reorder that groups the instructions without an LLM rewrite. Switch between the models and compare the four conditions at a stack of twenty.

Interactive · four conditions, stack of twentyfollow rate · advice task
050%100%

04 · The proof

The fix helps exactly the models you deploy at scale

This is the result that changes what you do on Monday. The authors hold the provider constant and vary only model size, compiling with the same model every time, and the recovery is not uniform — it is inversely proportional to how good the target already was. Weak-but-not-floored models gain most; frontier models gain nothing, and in one case measurably lose, because they already impose the same structure internally and the rewrite only removes information. Across the nine models the rank correlation between raw follow rate and recovery is −0.85. Pick a provider and read the ladder.

Interactive · the scaling ladderpoints of follow rate recovered · stack of 20
−80+12

The honest reading of the negative numbers: the apparent harm to the strong models did not survive the paper's own statistics. Resampling whole stacks rather than individual trials, the cross-model gains hold at p ≤ 0.001 while the same-model loss does not (p = 0.49). So the finding is "large gain for weak targets, no reliable change for strong ones" — which is still the actionable half, because the cheap model is the one you put behind the high-volume endpoint.

05 · In your stack

Audit the prompt you already shipped illustrative

Here is the version of this you can run today, before writing any code. Tick the requirements your production system message actually contains. The widget maps each to the benchmark's instruction and reports how many pairs among them are logically unsatisfiable — using the paper's enumerated conflicts, not a guess. The projected follow rate is interpolated from the paper's measured curve for a mid-tier model and is illustrative; the conflict count is not.

Interactive · your requirementstap to toggle each requirement
impossible pairs you just created
0
Enumerated from the paper's conflict list — not modelled.
projected per-rule follow rate illustrative
What breaks first

The numbers

What the paper actually reports

What this does not show

In practice