Understanding temperature tuning in energy-based models
Year: 2026
Type: article
Abstract: Generative models of complex systems often require parameter adjustments to produce useful outputs. For example, energy-based models for protein design are sampled at an artificially low “temperature” to generate novel, functional sequences. This temperature tuning is a common yet poorly understood heuristic used across machine learning contexts to control the trade-off between generative fidelity and diversity. Here, we develop an interpretable, physically motivated framework to explain this phenomenon. We demonstrate that in systems with a large “energy gap” separating a small fraction of meaningful states from a vast space of unrealistic states, learning from sparse data causes models to systematically overestimate high-energy state probabilities, a bias that lowering the sampling temperature corrects. More generally, we characterize how the optimal sampling temperature depends on the interplay between data size and the system's underlying energy landscape, and that this interplay is sufficient to elicit the tuning phenomenon; regularizing while training can contribute, but is not necessary. Crucially, our results show that lowering the sampling temperature is not always desirable. We identify the conditions under which it results in better generative performance. Our framework reveals that temperature tuning is indicative of both specific properties of the true data distribution and the limits of the learned model due to insufficient data availability.
Source: Physical Review Research
DOI: 10.1103/g8cm-7ps7
Language: English
Subfield: Molecular Biology
Domain: Life Sciences
Open Access status: gold
Estimated APC paid: $2,625
OpenAlex ID: W7204943435