Multi-Model Judgment Quality under Peer Aggregation and Adversarial Debate
Year: 2026
Type: preprint
Abstract: Across the six experts and production tasks evaluated here, no single large language model is consistently best:strengths and blind spots differ across providers. This paper evaluates six expert LLMs under standalone answersand peer aggregation (each model revises with peer outputs as context), with structured adversarial debate reportedas a secondary, descriptive tier. Endpoints are GPQA Diamond accuracy (known answers) and production weightedreference-finding coverage (leave-one-out consensus alignment; not precision). A companion process paper coversfail-closed gates. Production false-positive burden is unmeasured (Limitations).The cleanest result is GPQA Diamond: 77% standalone to 86% with aggregation. Under mixed expert-plus-lightpost-debate context, panel mean accuracy is 88% after debate—near aggregation under non-matched context, sounder that condition debate adds little beyond aggregation. On production, each expert’s aggregation coverage is3.7 to 12.5 percentage points above standalone. Every expert’s aggregation point estimate is higher: the observedfloor, mean, and top all move up. Post-debate scores are 26.1 to 49.2 percentage points above the same standalonebaseline (42–50% to 68–94%; panel +39.9pp) as recognition coverage under a leave-one-out consensus baseline, notindependent ground truth (Limitations). Thus production is where the large observed coverage difference appears,with that caveat. Scorer and finding-extractor roles overlap with panel contestants. Peer information is associatedwith higher observed scores; no ablated causality over extra tokens or majority selection alone is claimed
Author: Razumny, Igor
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7169845732
Cites:
Cited by:
Related to: