A structured-output failure is normally one number: the output did not match what you asked for. This paper splits that number in two — did the value you planted come back anywhere, and did it come back at the path the schema required — and the two diverge sharply as the schema deepens. At four levels of nesting the stronger models still surface 91% to 96% of the values, while 24% to 74% of the ones they surfaced sit at the wrong position. Structure breaks before content does, a single accuracy score cannot see the difference, and the cheapest lever is the shape of the schema you hand over.
The setup is deliberately airless: the model is handed a schema and a list of values with the exact paths they belong at, and asked to emit the filled structure. Because the values are unique tokens — phonetic-alphabet words and fixed number sequences — you can check two independent things afterwards. Value presence asks whether each planted value appears anywhere among the leaves. Value placement accuracy asks whether it appears at its assigned path. Step the schema from two levels deep to four and watch the two measurements come apart.
The pale bar behind each solid one is the same model's recall. At two levels the two are almost the same bar; at four they are a gap of 22 to 61 points depending on the model. Nothing about that gap shows up in a validator, because every one of those values is a legal string sitting at a legal path.
Grading structured output as a single pass/fail conflates two failure modes that need completely different fixes. A value error means the model got the content wrong: it wrote something you never gave it. A placement error means the content is perfect and the address is wrong: your value is in the output, one branch over. The paper's decomposition is four small metrics — presence, placement, schema compliance, and the displacement rate that ties them together — so the two show up separately. Step through four outputs, all of which a monolithic grader marks simply "wrong".
The one worth adding to your evaluation first is the displacement rate — the share of recalled values that ended up misplaced. It is the number that tells you whether to fix your prompt and your schema or reach for a bigger model, and it is a division of two things you are probably already able to measure.
Four properties of the target structure were varied one at a time, on a small model at the middle complexity, to see which one moves placement. Two of them are things you control entirely and can change this afternoon: whether your field names describe what they hold, and whether the same name appears at more than one path. Each is worth around eight points of displacement on its own. Depth is the brutal one, and the number of values you ask for at once is — surprisingly — almost irrelevant.
Read the last two together. Going from one level of nesting to three takes displacement from 0.4% to 28.5%; going from five planted values to twenty leaves it flat around 50 to 60%. The model is not running out of room. It is losing track of where, and depth is what makes where hard.
The second half of the paper turns the decomposition into a training signal. Reinforcement learning with a verifiable reward normally scores an emitted structure against a reference; here the reward is built from the placement metric directly — 1.0 × placement accuracy + 0.3 × schema compliance — with presence deliberately left out, because paying for presence would reward dumping every value somewhere legal. A 7B instruction model with a small adapter, ten samples per prompt, 500 steps. The interesting part is not the in-domain jump but that it transfers to schemas the model never trained on.
The honest caveat is in the table domain. Format validity rose from 26.5% to 85.5% — the model learned to emit well-formed tables — but getting a value into the right cell of a grid stayed near the floor at 0.06 to 0.09. Addressing a path in a tree and addressing a coordinate in a grid are not the same skill, and only one of them was fixed here.
The production shape is an extraction endpoint: a document goes in, a filled schema comes out, and something downstream reads specific paths. Set your own volume and fields below. The three outcome shares applied are the paper's measured numbers for the model and complexity you pick — correctly placed, present but misplaced, and missing altogether. Misplaced is the row that matters, because those records pass a schema validator and quietly feed the wrong number to whatever reads them.
Flatten the schema before you upgrade the model. Moving the same six models from four levels of nesting to two takes the worst displacement rate in the table from 73.8% to 16.3%, and the best from 24.2% to 0% — a bigger swing than any model change in the same table buys you.