Year: 2026
Type: conference-paper
Abstract: Large language models (LLMs) exhibit strong reasoning capabilities but typically require expensive post-training to reach high performance.Recent test-time alignment methods offer a lightweight alternative, but have been explored mainly for preference alignment rather than reasoning.To bridge this gap, we propose Token-level Adaptive Routing (TARO) , which steers frozen LLMs toward structured reasoning entirely at inference time.Specifically, we first train reward models on step-wise mathematical traces to capture fine-grained logical consistency signals, then introduce a learnable token-level router that automatically controls the guidance of the reward model to the base model.Extensive experiments show that TARO significantly improves reasoning performance by up to +22.4% over base model and +8.4% over existing token-level test-time alignment methods, while also boosting out-ofdistribution clinical reasoning (MedXpertQA) and instruction following (AlpacaEval).Furthermore, TARO also generalizes from small to large backbones without retraining, extending test-time alignment from preference optimization to robust, cross-domain reasoning.
Institutions: University of Pittsburgh, University of California, Berkeley
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166845131