When a model gets a task wrong, the reflex is to add another few-shot example. This paper runs the controlled version of that instinct: five tasks, each specified three ways — the rule stated in words, a set of worked examples chosen to cover exactly the same ground, or both — across fourteen open-weight models at two sizes and both base and instruction-tuned. Rules win overall. Adding the examples on top of the rule produces no statistically significant gain on any of the five tasks, and simply scaling the example count gives mostly flat, diminishing or negative returns. The cost side is not close either: on the hardest arithmetic task the rule takes 158 tokens and the demonstrations that cover it take 7,022.
The design is what makes this readable. Each task exists in exactly one form; only the way it is communicated changes. The rules condition writes the task out in words and then gives the test problem. The examples condition gives valid input-output demonstrations and then the same test problem. The combined condition gives the rule, then the demonstrations, then the problem. Nothing else moves — same model, same test items, same decoding.
The examples are not chosen casually. For each task and difficulty the authors fix a coverage target — every argument order, every violation type, every combination of shared attributes — and use the smallest demonstration set that hits it. That is the fairest possible version of "just show it": the examples genuinely contain the rule, they simply do not state it.
Here is where the asymmetry becomes structural rather than empirical. A rule describes a mechanism, so its length tracks the mechanism's complexity. A covering example set has to enumerate cases, so its length tracks the number of cases — which, for anything with permutations or combinations in it, explodes.
Follow the operator task across its three difficulties. Three arguments needs six demonstrations; four needs twenty-four; five needs a hundred and twenty. The rule that generates all of them grows from 118 tokens to 158. One of those curves is the factorial and the other is a sentence.
Fitting a mixed-effects model over the whole grid, rule-based learning significantly outperforms example-based learning overall, and the combined condition — rule plus examples — is not significantly better than the rule alone. That second result is the one worth sitting with, because the combined prompt is what most production prompts actually are: a description of the task, followed by a few demonstrations for good measure.
The gradient across those five is the practical finding. The rule advantage is largest where the task is algebraic — comparisons of the same-or-different kind, or applying an unfamiliar operator in a fixed order — because that structure is defined over variables and is exactly what a demonstration fails to make visible. It shrinks where the input itself carries strong distributional cues, as a board position does, and where the task leans on knowledge the model already has.
Instruction tuning significantly improves rule-based and combined learning and leaves example-based learning statistically unchanged. That is a compounding effect, not a one-off: the models you are most likely to be calling in production are precisely the ones tuned to follow a stated rule, and none of that tuning was aimed at making them better at inferring one from demonstrations.
The paper's cleanest numeric contrast comes from its hardest case for rules — a task where sequences must be checked against a category defined either conjunctively (every condition must hold) or disjunctively (any one condition may hold). Under example-based learning the two are not equally learnable.
The pattern under examples is consistent and the explanation is mechanical: positive instances of an "any of these" category are not similar to each other, so there is no shared surface for a learner generalising by resemblance to latch onto. Under rules there is no such pattern, because a rule says "or" and the shape of the category is stated rather than inferred. If your classifier's category is a disjunction — an escalation trigger, a set of unrelated disqualifying conditions — that is exactly the case where demonstrating it is weakest.
Turn it into the decision you actually make. You have a context window, a slice of it is going on specifying the task, and the rest is available for the thing the user is actually asking about — the document, the ticket history, the retrieved passages. Set a budget and see what each style of specification leaves you.
The arithmetic is illustrative — your tasks are not these five, and the paper is explicit that none of this shows rules are always the better specification in practice, where user instructions are ambiguous, incomplete and noisy in ways a controlled rule prompt never is. What survives the caveat is the ordering of the work: write the rule first, measure, and only then find out whether any examples are earning their tokens.