Year: 2025
Type: conference-paper
Abstract: Large language models (LLMs) rely heavily on sampling methods to generate diverse and highquality text.While existing sampling methods like top-p and min-p have identified the detrimental effects of low-probability tails in LLMs' outputs, they still fail to effectively distinguish between diversity and noise.This limitation stems from their reliance on probability-based metrics that are inherently sensitive to temperature scaling.Through empirical and theoretical analysis, we make two key discoveries: (1) the pre-softmax logits exhibit a clear statistical separation between informative tokens and noise, and (2) we prove the mathematical equivalence of min-p and top-(1-p) under uniform distribution over logits.These findings motivate the design of top-nĻ, a novel sampling method that identifies informative tokens by eliminating noise directly in logit space.Unlike existing methods that become unstable at high temperatures, top-nĻ achieves temperature-invariant token selection while preserving output diversity.Extensive experiments across reasoning and creative writing tasks demonstrate that our method consistently outperforms existing approaches, with particularly significant improvements in high-temperature settings.
Authors: Chenxia Tang, Jianchun Liu, Hongli Xu, Liusheng Huang
Institution: University of Science and Technology of China
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: G2651838923, G645416012
OpenAlex ID: W4412945431