Put a manager in front of the model: it writes a plan, curates a task list, hands single tasks to a worker, and decides when to stop — all of it coordinated through files in a shared workspace rather than a growing conversation. That is the shape most agent frameworks push you toward, and this measures what it is actually worth across nine models on the same 100 competitive-programming problems. It is not a uniform win and the paper does not pretend otherwise: one model gained 23.4 points, two gained 8 to 10.6, and one lost 9 because the planning stage talked itself out of a correct algorithm. About a fifth of the largest gain was not planning at all — it was the scaffold rescuing runs that had simply blown their context window. And the strongest case for it is the one nobody leads with: it roughly triples the token bill, and still buys accuracy more cheaply than moving up a model tier.
The design choice worth copying is where the state lives. Instead of one conversation that grows until it stops fitting, the manager and worker coordinate through files: the problem statement, a plan, a task list with status and results per item, a notes file that accumulates findings, and the current best solution. Each call reads what it needs and writes back.
The loop is deliberately dull. The manager writes a plan; the worker names the core difficulties; the manager curates a task list; the worker executes exactly one task; a verifier runs the sample tests; a finaliser emits the solution. Ten manager-to-worker rounds is the whole budget, and the manager revises the list after each round and decides when to stop.
Across nine models the same scaffold produced everything from a 23.4-point gain to a 9-point regression. The three models measured over five passes each moved decisively and significantly: a mid-size open-weight model went from 63.0 to 86.4, and two hosted models gained 8.0 and 10.6. A frontier model reached 91 in a single pass, up from 85 — a point estimate, not a measured effect, and the paper says so.
The regression is the useful part. On one problem, the scaffold's ideation stage looked at a correct quadratic-time approach, dismissed it as complex and error-prone, and replaced it with slower cubic code carrying extra bugs. A planning stage does not only add planning; it adds a place for the model to talk itself out of something that was already right.
Before crediting the planning, look at what the scaffold removed. In the single-call arm on the model that gained most, the context cap was hit 150 times and 35 runs produced no code at all. Under the manager: 5 cap hits, zero empty solutions. Of those 35 no-code cells, the manager passed 25 and failed 10.
That accounts for roughly 5.0 points of the 23.4 — about a fifth. It is not planning, it is the workspace keeping each call small enough to finish. Worth separating, because if that is your failure mode there are cheaper fixes than a multi-agent architecture.
The rest is genuine decomposition, and the transcripts are specific about what it looks like: surfacing an efficiency insight before coding, splitting a two-objective algorithm into its parts, and writing an explicit reduction into the notes before implementation.
The scaffold roughly triples the tokens, and the honest way to price that is against the alternative you would otherwise buy: a larger model. On that comparison it wins twice over.
An upper-tier hosted model with a manager reached 85.0 against a frontier model's 87.4 — two and a half points behind, at $11.71 a pass against $61.11, roughly a fifth of the price. And a mid-tier model with a manager matched the upper-tier model's single call on accuracy, 77.8 against 77.0, at $1.50 against $3.41.
The decision this paper actually supports is narrow and useful: for a given model and a given accuracy target, is a manager cheaper than the next tier up? Both sides are measurable in an afternoon.
The chart above already answers it for the configurations the paper measured. What it cannot tell you is which side of the line your model falls on — one of the nine got worse, and the only way that surfaced was running it.