Thermo Pulse: Entropy-Feedback Temperature Control for Autoregressive Language Model Decoding
Year: 2026
Type: preprint
Abstract: Sampling temperature is usually fixed for an entire generation, even though the model’s next-token uncertainty changes from step to step. We study Thermo Pulse, an inference-time controller that sets the temperature at every step from the entropy of the renormalized top-five next-token distribution and from its first difference, using a clipped proportional–derivative (PD) law. We also evaluate a hybrid that composes the PD temperature with a relative-probability cutoff removing candidates below 5% of the most likely token. On Qwen2.5-3B-Instruct we compare nine decoding configurations, including greedy decoding, fixed temperatures, an entropy-threshold rule, a Mirostat-v2-style controller and P-only/D-only ablations, on 200 programmatically generated logic items and 200 mathematics items under five paired seeds (18,000 generations). The hybrid reaches 27.7% logic and 29.5% mathematics accuracy, against 26.7% and 28.6% for the PD controller without the cutoff and 27.6% and 25.4% for fixed temperature 0.7. None of the four prespecified primary contrasts is statistically established: the largest, +4.1 percentage points in mathematics relative to fixed temperature 0.7, has a family-cluster bootstrap 95% interval of [−0.8, 9.4] and a Holm-adjusted p of 0.609. Greedy decoding attains the highest mathematics accuracy (33.0%). Step-level instrumentation shows that the hybrid cutoff is active on about half of the generation steps while removing on average under 0.5% of probability mass. We present these results as an instrumented and reproducible case study of entropy-feedback decoding rather than as evidence of a general improvement. Code, data and raw results are available at https://github.com/enkiluv/thermo_pulse.
Source: Preprints.org
Author: Myung Ho Kim
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225166495