KAD: A Framework for Proxy-based Test-time Alignment with Knapsack Approximation Deferral
Year: 2026
Type: other
Abstract: With the accelerated scaling in large language models' (LLMs) sizes, comes the prohibitive surge in computation budget allotted for model's alignment and fine-tuning for downstream tasks. And while factual knowledge has been shown to be learned earlier during pre-training, a model still requires further fine-tuning to adhere to task requirements and stylistic preferences. We propose a novel approach to circumvent costly large model fine-tuning via test-time alignment, i.e. using guidance from a small aligned model. Our approach is based on a new formulation of token-specific cascading as a 0-1 knapsack problem. In this setting, we derive primal and dual approximations of the optimal deferral decision. We further demonstrate the advantages of our method experimentally both in task performance and speculative decoding speed.
Source: Underline Science Inc.
DOI: 10.48448/7n9v-cr82
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7155358161