The scaffolding around a frozen model — who it is, how to approach the task, what format to emit, when to back off and retry — is usually optimised as one flat string. Split it into four separately evolvable slots, run the same optimiser under the same budget, and measure each slot's contribution, and the answer is stark: on a household-chores benchmark, +11.9 points of the total come from the loop-discipline slot and the other three are individually worth nothing. Worse, the obvious way to spend a tuning budget makes it disappear. Split 64 trial runs evenly and each slot gets 16 — below the threshold the optimiser needs before it will accept any edit at all — so all four freeze at their empty starting text and the whole run ties with doing nothing. Spend 32 on the one slot that matters and the agent goes from 64.2% to 76.1%, on half the budget.
Reflective prompt optimisers treat an agent's harness as a single blob of text to mutate. This paper's only structural change is to cut it into four mutually exclusive slots, each with a rule about what may and may not go in it, and evolve them separately. Nothing else about the optimiser changes.
The separation is what makes attribution possible. With one flat string you can only ask whether optimisation helped; with four slots you can ask which part of the harness the help came from, by running the agent with exactly one evolved slot in place (leave-one-in) and with exactly one taken out of the full harness (leave-one-out).
The headline comparison is the boring one. Under an equal budget, evolving the four slots separately does not beat evolving one flat string, and neither beats the stock harness the agent shipped with. All three tie.
A paper could stop there and report a null. This one asks where the value went instead, and the answer changes what you should do with a tuning budget.
Leave-one-in is the clean measurement: start from the stock harness, drop in one evolved slot, and see what it is worth on its own. Do it four times and the distribution is not close to even.
The control slot is the only one whose interval clears zero. Role looks mildly positive and does not survive a significance test; task strategy and format rules are flat to three decimal places. Leave-one-out tells a slightly different story — pulling the role slot out of the finished harness costs more than pulling the control slot out — which is what you expect when slots interact and the total is not the sum of the parts. The paper's own reading is that credit assignment has to come before structured evolution, not after.
It helps to see what the winning slot actually evolved into, because it is not clever. It is four rules a careful engineer would have written by hand.
Here is the part that costs people money without them noticing. The optimiser needs a minimum number of trial runs before it can accept and keep a single mutation — it has to sample a minibatch, propose an edit, and re-score the candidate on a validation set. Below that floor it cannot commit to anything, so it returns the seed it started from.
At 64 runs split four ways, each slot gets 16 — right at the floor — and the paper reports that zero of four slots accepted a single candidate. All four came back empty, which is why the structured method tied with the stock harness in section 01. It was not that the decomposition was wrong. It was that the decomposition starved every part of it at once.
Note the shape of the last two rows. Half the budget spent on the control slot alone is significant at p = 0.0046. The full budget spent on the same slot scores lower and only reaches borderline. The authors flag this honestly as a sample-size question rather than a claim that more budget hurts — at n = 134 these two are not cleanly separable — but the direction of the useful result is not in doubt: concentrating recovered a gain that splitting destroyed.
Run the whole thing again on a shopping task instead of a household one and everything ties, including the concentrated allocation. Every slot freezes empty there too — but for a different reason, and the difference is the point.
On the household benchmark, starving the slots produced a null and concentrating the budget broke it. On the shopping benchmark, concentrating the budget produced the same null — so this is not budget starvation, it is the absence of anything for a control rule to fix. The shopping task has a dense reward and no recurring, describable loop failure, so there is no repeated mistake for a reflective optimiser to name in a sentence.
You can see the same thing inside the household benchmark, in which task types the gains landed on.
The gains cluster on exactly the task types the evolved rule addresses — the ones that need a container opened before something can be taken, or an object carried to a light and checked. That is the signature of a real mechanism rather than a lucky seed: the rule names a failure, and the tasks that contain that failure are the ones that improve.
The transferable procedure is short. Read a sample of failed traces before you buy any optimisation. If the same recoverable mistake shows up repeatedly and you can write it down in one sentence, you have a control-slot problem and concentrating a budget on it will pay. If you cannot write that sentence, no amount of prompt evolution will find it for you.
And size the spend against the floor rather than against your intuition, because the floor is the thing that turns a reasonable-looking plan into a run that changes nothing.
The arithmetic here is illustrative — your optimiser's floor depends on its own minibatch and validation sizes, which you can measure in one run by checking whether any candidate was ever accepted. What transfers is the trap: an even split across n parts is the allocation most likely to leave every part below its floor, and it is also the allocation everyone reaches for first.