Beyond Second Chances: A Transition Analysis of Peer Feedback in LLM Revision
Year: 2026
Type: preprint
Abstract: When multiple LLM-generated reasoning paths independently answer a question and then revise after exchanging peer responses, aggregate accuracy sometimes improves and sometimes declines. We introduce a paired-transition framework that decomposes revision-stage gains into a retry baseline and two peer-feedback channels: an exposure channel (behavioral shift from generic, task-irrelevant peer context) and an evidence channel (incremental repair from task-relevant rationales). A paired ablation with up to six conditions across three tasks (Logic Grid Puzzle, GSM8K, FOLIO) and three open-weight model families (Gemma-4-31B, Qwen3.6-27B, Mistral-Small-3.2-24B), with pre-registered hypotheses and Holm-corrected bootstrap tests, reveals that the evidence channel provides a robust positive repair signal across all confirmatory families, while the exposure channel is model-dependent---ranging from mildly stabilizing to strongly destabilizing---falsifying the hypothesis that peer exposure universally reduces revision damage. A cross-size peer-source swap on Logic Grid shows that peer-trace quality can be the dominant bottleneck in evidence-rich settings. An adjudicated blinded-coding study provides convergent evidence that successful repairs more often contain novel reasoning steps than matched controls. These findings transform the question of whether peer revision helps into a structural analysis of which feedback components work, when, and why.
Author: Lin, Chien-Tung
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7166715378
Cites:
Cited by:
Related to: