Work

From AR to Diffusion: Efficiently Adapting Large Language Models with Strictly Causal and Elastic Horizons

Year: 2026

Type: conference-paper

Abstract: Diffusion models promise efficient parallel text generation but rely on bidirectional attention, creating a structural mismatch with pre-trained Autoregressive (AR) models.This incompatibility precludes reusing robust AR priors, necessitating prohibitive pre-training from scratch.To bridge this gap, we propose FLUID, a framework that efficiently adapts AR backbones to the diffusion paradigm.By enforcing Strictly Causal Alignment, FLUID enables seamless initialization from standard GPT-style checkpoints, circumventing the need for massive pre-training.Furthermore, we introduce Elastic Horizons, an entropy-driven mechanism that dynamically modulates denoising strides based on local information density rather than fixed schedules.Experiments demonstrate that FLUID achieves state-of-theart performance while reducing training costs by orders of magnitude, effectively reconciling established AR foundations with efficient parallel generation.

Source: Proceedings of the conference - Association for Computational Linguistics. Meeting

DOI: 10.18653/v1/2026.acl-long.958

Authors: Xiangyu Ma, Teng Xiao, Zuchao Li, Lefei Zhang

Institution: Wuhan University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: National Natural Science Foundation of China

Award: G7898550485