Work

In-Place Instruction Following in Diffusion Language Models

Year: 2026

Type: preprint

Abstract: Diffusion Large Language Models (dLLMs) generate text via bidirectional iterative denoising, naturally supporting user-specified constraints anchored at arbitrary output positions, a paradigm known as In-place Prompting (IPP). We formalize this as the In-place Instruction Following (IIF) task and construct IIF-Bench, a hierarchical benchmark spanning literal, style, and discourse-function constraints, paired with a rubric-based local-global evaluation protocol. An inference-time attention-bias probe suggests that vanilla dLLMs often under-prioritize constraint spans during denoising. We then propose GRAFT, an IPP-oriented post-training framework combining constraint-aware SFT and preference optimization. On four representative dLLMs, GRAFT raises the average IIF score from 57.75 to 73.10 (+15.35 points), with absolute gains of 15.91 and 15.57 points on literal and discourse-function constraints, while preserving general generation ability.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2609.07160

Authors: Nie, Zheng, Li, Zherui, Zhang, Jiaming, Wang, Kun, Zhou, Zhenhong, Guo, Yufei

Institutions: National University of Singapore, Nanyang Technological University, Peking University

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W7212037879

Cites:

Cited by:

Related to: