The standard way to have a model write the assertion at the end of a unit test is to run it, show the model the failure, and let it try again until it passes. Passing is a proxy. Under repair pressure the model takes the cheapest route to green, and the assertion it lands on is weaker than the one it started with: a check on an exact count becomes a check that the count is above zero, then a check that the object exists. Measured against deliberately broken versions of the code, the loop lifted the share of tests that pass by 11.8 pts and pushed fault detection 5.3 pts below the same model's very first, un-repaired answer.
Here is the paper's own worked example, and it is the whole argument in four steps. A test adds one connection to a pool and needs a closing assertion. The model writes one, it fails, the failure is fed back, it writes another. Step through the rounds and read both verdicts: the one the loop can see, and the one you actually care about.
Nothing in that sequence is a bug in the model. Every rewrite is a locally sensible response to "this failed, fix it" — the exact-count check is the most fragile thing on the line, so weakening it is the shortest path to green. And the real defect was never the strictness: the test class holds the pool key in a field, and the first assertion simply used the wrong literal. No amount of retrying recovers that. Reading the class would.
To separate the two you need a fault-detection score, and the paper uses mutation testing: take the program, generate hundreds of small deliberate breakages, and count how many of them the generated test actually notices — scored only over the breakages the human-written assertion itself exercises, so a test cannot win by wandering somewhere else. Pick a model, then flip between what the loop optimises and what you wanted.
The control that makes this a trap rather than a tuning issue is the Trivial row on the fault-detection view: an assertion that literally says assertTrue(true) still scores around 40%, because the test body itself trips some breakages before the assertion is ever reached. That is the floor. Everything above it is the assertion's own contribution, which is exactly what the repair loop spends.
Writing a good assertion often needs values that only exist at runtime, so the obvious move is to pause the test and paste every local variable into the prompt. The paper tries that, and it reproduces the pattern exactly: pass rate up, fault detection down. The alternative is to let the model name what it wants — it reads a folded skeleton of the classes, emits a list of expressions, and a debugger evaluates just those.
The failure of the raw dump is instructive because it is not about volume of tokens. A dumped local reads Response@531f4093 — an address, which supports no assertion stronger than "not null". A named query reads handshakeResponse.getStatus().getRequestStatus() and comes back 400, which supports asserting the actual status. Same test, same runtime, and one of the two makes a strong assertion possible.
The loop is not free while it degrades your tests. Each repair round is another generation, another Maven run, another wait. Set how many assertions you want generated and see the measured per-instance figures scale — token counts, wall-clock, and the API bill on the model the paper priced.
The shape of the saving matters more than the size. The context-first pipeline sends a bigger prompt — around 6,800 tokens against a first-round prompt of 350 — and generates roughly one twelfth of the output. Decoding is the expensive, latency-bound half of inference, so moving work from repeated generation into one well-built prompt is cheaper on both axes, and a stable prompt prefix caches where a growing repair transcript does not.