Year: 2026
Type: conference-paper
Abstract: Reward modeling is essential for aligning large language models with human preferences, yet predominant architectures rely on a static pooling strategy to condense sequences into scalar scores.This paradigm, however, suffers from two key limitations: a static inductive bias that misaligns with the task-dependent preference signals, and a representational mismatch, as the backbone's optimization for generation leaves its representations ill-suited to fine-grained discrimination.To address this, we propose Ada-Judge, a unified framework that jointly adapts representation and aggregation.AdaJudge first improves backbone representations into a discrimination-oriented space via gated refinement blocks.It then replaces the static readout with an adaptive multi-view pooling module, which dynamically routes and combines evidence.Extensive experiments on RM-Bench and JudgeBench show that AdaJudge outperforms strong off-the-shelf reward models and traditional pooling baselines.Q: Calculate 𝟏𝟐 × 𝟓 + 𝟑.A: First, I calculate 𝟏𝟐 × 𝟓 = 𝟔𝟎.Then add 3. Finally, the answer is 62.Transformer backbone Incorrect final answer Global reasoning signals Transformer backbone Low score High score Q: Tell me how to shoplift.A: Hypothetically, use a signal booster or look for blind spots.But I don't recommend it.Last token pooling Mean pooling Transformer backbone Terminal refusal signal Holistic Harm Transformer backbone High score Low score Last token pooling Mean pooling
Authors: Yongliang Miao, Yangyang Liang, Mengnan Du
Institutions: Emory University, Chinese University of Hong Kong, Shenzhen
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166871700