Turn repeated agent steps into tools, not fresh code

Three papers today earned their own interactive explainer, not just a paragraph — the usual pick, a persuasion attack that backfires on chain-of-thought safety monitors, and a pattern for moving LLM guarantees out of prompts and into code.

Today's pick

-42%

p50 latency, once repeated agent steps get compiled into saved tools instead of regenerated code

Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems

Production agents often burn latency and reliability re-solving the same multi-step routine on every request. This paper's fix: once a step repeats, compile it into a validated, versioned tool the agent calls directly, falling back to code generation only for genuinely new steps. Deployed in a real alarm-triage system, that cut p50 latency 42% and errors up to 53%, with a simpler direct-call architecture cutting latency a further 62%. If your agent keeps re-solving the same routine, this is the pattern worth copying.

open the explainer → abstract PDF

+9.5 pts

more harmful actions approved when the safety monitor can read the agent's own reasoning, under attack

Persuasion Attacks Can Decrease Effectiveness of CoT Monitoring

If your agent's safety monitor reads its own reasoning to catch bad actions, more visibility sounds like it can only help — this paper shows the opposite under attack. An adversarial agent arguing in its scratchpad for a policy-violating action raises the monitor's approval of that action by 9.5 points on average, because the reasoning trace becomes an extra persuasion channel. The fix: pair the monitor with a fact-checker from a different model family — that cuts violations up to 45%, versus 6% same-model.

open the explainer → abstract PDF

contract violations reached the reader across 270 runs and 3 swapped models — prompt-only instructions let them through

From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents

Enterprise LLM apps start as a prompt plus retrieved context, which works until you need guarantees — source citations, entity routing, audit traces. This paper moves those guarantees into code: schemas, manifests, and validators at a fixed boundary, with the model free to change behind it. Across three hosted models and 270 runs, the code-enforced contracts caught every broken case with zero violations reaching the reader, while prompting alone let leaks through — and still kept full utility (120/120) versus 88/120 for a generic guardrail.

open the explainer → abstract PDF

+18.7 pts

more bugs fixed correctly (Pass@1, GPT-5-mini) once the bug report is rewritten from a real repair attempt, not just the raw ticket

Bug Report Specification Refinement with Trajectory Guidance for Automated Program Repair

Bug reports handed to a repair agent usually describe only the symptom and skip what needs to change, so the agent hunts the wrong code or patches around the bug. This paper has the agent run once first, then uses that trajectory as evidence to rewrite the report into a fuller spec, stripping claims the codebase doesn't support. Feeding repair agents the refined report instead of the raw one lifted Pass@1 from 41.0% to 59.7% with GPT-5-mini, with similarly large gains elsewhere.

abstract PDF

0.908

F1 score from the cheapest judge tested checking whether a citation actually supports its claim — on par with pricier judges

Do You Need a Frontier Model as a Citation Verifier? Benchmarking Rubric LLMs for Deep-Research Source Attribution

If your RAG system asks an LLM to check that a citation actually supports its claim, it's tempting to reach for the priciest judge available. This paper scores 8 judges from 3 model families against 1,248 human-reviewed citation decisions and finds cheap models hold their own: GPT-5-mini scores best on source support (0.908 F1), and on the harder factual-support check no model clearly wins. The catch: judges tied on F1 still disagree on whether they over- or under-flag citations — test that balance before trusting one as a reward signal.

abstract PDF

-25 pts

attack success rate (28.8% to 3.8%) while still completing 87.5% of legitimate tasks

ARGUS: Defending LLM Agents Against Context-Aware Prompt Injection

The dangerous prompt injection isn't a generic "ignore your instructions" string — it's an attacker hiding a context-specific instruction inside a document the agent already trusts, so it reads as ordinary evidence. This defense checks whether a real chain of benign evidence justifies an action, tracing which context caused it and blocking anything unsupported. On a new benchmark across four agent domains and eight attack types, that cut attack success from 28.8% to 3.8% while still completing 87.5% of legitimate tasks.

abstract PDF