Teaching a model to say "I'm not sure" normally starts with a labelling job: ask it thousands of questions, mark which answers were wrong, then fine-tune it to decline exactly those. This paper deletes the labelling job. Rank the training questions by the average log-probability the frozen model gave its own answer, retarget the least-confident slice to a fixed refusal, keep the model's own answer everywhere else, and train a small adapter. Across six open models from 1B to 8B it lands within noise of the labelled version — 0.95 to 1.10 times its hallucination rate at matched coverage. The blind spot is the obvious one: the confident mistakes never trip the wire.
A language model states a false fact as fluently as a true one, but the fluency is not uniform underneath. Average the log-probability the model assigned to each token of its own answer and you get one number per question — a doubt score, computed once, from the frozen base model, with no labels and no second model involved. Rank a question set by that number and the wrong answers pile up at the bottom. How reliably is the question this section answers: pick a model and see how well its own confidence separates its mistakes from its hits.
This is a useful signal, not a great one. Every model clears chance comfortably and none is near perfect — the range across the six is 72.5 to 85.8 on a scale where 50 is a coin flip. Note also that it does not simply improve with size: the largest Qwen model separates its own errors slightly worse than the smallest one does.
The training recipe is identical in all three arms; only the rule choosing which questions get retargeted changes. That is what makes the comparison clean — the difference between arms is a supervision cost, not an architecture. Step through them and read what each does to the training set.
The threshold is the only knob, and it is a coverage knob: push it up and the model answers less often but is right more often when it does. Nothing in the pipeline needs to know which answers were actually correct.
Here is the claim the paper is built to support. On every one of the six models, the label-free arm and the label-supervised arm land within each other's confidence intervals at matched coverage — a ratio of 0.95 to 1.10 between their hallucination rates, with no difference detected anywhere. Below, each model's journey from answering everything to answering selectively.
One control kills the obvious alternative explanation. Take the same low-confidence questions and, instead of retargeting them to a refusal, train harder on getting them right — the "just memorise more" arm. It stays at the base hallucination rate with near-full coverage. The gain comes from knowing when to stop, not from learning more facts.
Now the part to design around. A doubt score built from the model's own probabilities can only catch the mistakes the model is uneasy about. It is silent on the ones it is sure of — and those are the majority of what is left after training. Toggle between the two halves of the residual.
There is a shape to where the errors live, and it is the one you would guess: rarity. Split questions by how popular the entity being asked about is, and the least popular quartile hallucinates two to three times as often as the most popular. That is the part of your traffic where a confidence gate matters most and where it also has the most to be uncertain about.
Put this in front of a product surface — a knowledge-base assistant, a support deflection bot, any place a model answers a factual question in front of a customer. Set your volume and pick a model tier; the outcome shares are the paper's measured numbers for that model, and the traffic is yours. The trade is not subtle: far fewer wrong answers, far fewer answers.
The abstained questions are not lost, they are routed: to retrieval, to a larger model, to a search box, to a person. That is what makes the coverage cost payable — and it is why the useful version of this is a gate in front of a fallback, not a model that shrugs at the customer.