Acceptance rate is the number every code-completion product reports, and accepting is a keystroke rather than an outcome. This dataset follows the code afterwards, snapshotting the file every time a developer pauses for a second — 5,831 edit histories from 1,141 developers across 20 different models. Survival is bimodal, not average: the median completion still has 63% of its text intact, but the distribution piles up at both ends. 31% of histories end with the completion removed outright and 23% remove it on the spot.
Take every accepted completion and measure how much of its original text is still there later. The median comes out at 63%, which sounds like the picture of a helpful assistant needing modest cleanup. It is the wrong summary, because the distribution has two humps and almost nothing in the middle: code is either kept close to intact or it is thrown out. A median lands in the empty valley between them and describes a case that rarely happens. Flip between the two readings.
This matters because the two humps are different products. The right-hand hump is the assistant working. The left-hand hump is a developer reading a suggestion, accepting it, and then spending time undoing it — a cost that never appears in an acceptance-rate dashboard, and one that a higher acceptance rate can actively increase.
The edits sort into four intents, and the split is lopsided in a way that should reframe how you read your own telemetry. Formatting and readability fixes — the kind of change you would forgive — are a small minority. The dominant category is changing behaviour: adding methods, altering logic, correcting API calls. That is not polish. That is the model having produced working-looking code that did the wrong thing. Step through them.
Read the top two rows together: 56% of edits change behaviour and 9% delete the suggestion outright, against 14% tidying and 10% nudging toward personal style. Roughly two thirds of the editing effort is correction rather than taste.
The four intents arrive in a consistent order, measured as the average time from acceptance to that kind of edit. Deletions come first, then readability repairs, then intent-tuning, then functional rewrites nearly an hour out. Half of all edits land inside the first fifty minutes and 72% inside the first day. So the window you measure over is not an implementation detail — it silently picks which verdict you record. Drag it and watch what you would have concluded.
The trap is that a short window is not merely noisy, it is biased toward looking bad — deletions are the earliest edits, so a five-minute window catches the removals and misses every functional fix that would have shown the suggestion being salvaged. A window past an hour is the first one that sees all four intents.
The second half of the paper uses the data rather than describing it: predict the edit a developer is about to make. This is the task an assistant needs to be good at if it wants to offer the fix rather than wait to be corrected. Fine-tuning small open models on real edit histories moves them past few-shot frontier models — not because they are better at code, but because nobody's pretraining corpus contains this. Commits record the final state; they never record the intermediate one somebody actually typed. Switch tasks.
The gap the fine-tune closes is data, not scale. Three billion parameters with the right supervision outscores a frontier model shown a handful of examples, which tells you the bottleneck for edit prediction is a corpus nobody has been collecting — and that if you run an assistant, you are sitting on one.
If you ship a completion feature, you almost certainly report accepts. Apply this paper's rates to that number and you get a rougher, more useful figure: how many accepts survived contact with the developer. The removal rates are the paper's; the traffic is yours.
Two instrumentation changes fall out of this, and both are cheap. Log the retained fraction of each accepted completion at a fixed offset past an hour, not at accept time. And record removals separately from edits, because they are the one category where the suggestion made things worse rather than merely imperfect.