Year: 2026
Type: conference-paper
Abstract: Diffusion language models (DLMs) are theoretically efficient but are limited by fixedlength decoding and poor compatibility with KV caches.Block diffusion mitigates these issues, yet still enforces a fixed block size and costly training.We introduce Next Sequence Prediction (NSP), which unifies next token and next block prediction, enabling the model to adaptively determine generation length at each step.NSP reduces to standard next token prediction when the length is one.Building on NSP, we propose the Sequential Diffusion Language Model (SDLM), which retrofits pretrained autoregressive language models (ALMs) at minimal cost.Specifically, SDLM performs diffusion inference within fixed-size mask blocks, but dynamically decodes contiguous subsequences based on model confidence, thereby preserving KVcache compatibility and improving robustness to the varying uncertainty and semantics across the sequence.SDLM matches or surpasses strong autoregressive baselines with only 3.5M training samples, while achieving 2.0 × ˜3.4× higher throughput than Qwen2.5 across three model scales (3B, 7B, 32B) with confidencebased decoding.Furthermore, when sampling with self-speculative decoding, SDLM attains up to 5.7× end-to-end speedup measured as wall-clock tokens per second (TPS).Notably, SDLM-32B achieves performance comparable to Qwen2.5-32B with up to 2.5× speedup, demonstrating the scalability of our approach.Project page and codes: https://github. com/OpenGVLab/SDLM
Authors: Yangzhou Liu, Yue Cao, Hao Li, Gen Luo, Shihao Wang, Zhe Chen, Weiyun Wang, Xiaobo Liang, Biqing Qi, Lijun Wu, Changyao Tian, Yanting Zhang, Yuqiang Li, Tong Lu, Yu Qiao, Jifeng Dai, Wenhai Wang
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
Award: G3969642130
OpenAlex ID: W7083834108