One of the most intuitive ideas in AI-assisted programming is that a model should learn from its mistakes: show it the test output from its failed code, and it will produce a better fix on the next attempt. New research suggests that for smaller models, this intuition is wrong, and the fix is to pretend the failure never happened.
A paper from Yuvraj Verma at the University of Seoul, published on arXiv this week, systematically compares different retry strategies for code-generation models. The finding is striking: for models below 7 billion parameters, blind resampling (simply generating a fresh solution without seeing the previous attempt) outperforms every form of feedback-based self-repair, while consuming 2.5 to 5.5 times fewer tokens.
The root cause is anchoring. When a model is fed its own prior output alongside a failure signal, it tends to produce nearly the same code: 33 to 68 percent of retries yield a program indistinguishable from the first attempt. Without that feedback, under blind resampling, the repetition rate falls to between 2 and 14 percent. The model locks onto its initial solution, and the test output alone is not enough to dislodge it.
The experiment tested four matched-budget retry conditions across the MBPP+ benchmark: blind resampling, a content-free placebo failure notice, genuine execution feedback from the failed test run, and full reflection where the model wrote a verbal analysis of its mistake before retrying. All conditions received the same number of total attempts, isolating the effect of feedback type rather than extra tries.
Below 7B parameters, blind resampling was the strongest condition. Execution feedback added nothing measurable over the placebo. At 7B parameters, blind resampling remained statistically tied with the best feedback-based condition. The anchoring penalty was so predictable that its magnitude across six configurations (spanning two model families and two precision levels) could be forecast from baseline quality alone, with a correlation coefficient of 0.96.
The finding has practical implications for the design of AI coding agents. Self-repair is widely treated as a default component of code-agent pipelines, almost always evaluated against a baseline that does not retry at all. The paper argues this confounds the value of feedback with the value of an extra attempt. When the attempt count is controlled, the feedback itself contributes nothing for small models.
Reflection (asking the model to analyze its mistake) was the one intervention that measurably weakened the anchor, but at a high token cost. For cost-sensitive applications where every API call matters, the simplest strategy appears to be the best: try again, and do not look back.