Work

Don't Answer Twice: Diffusion Language Models That Only Generate What Changed

Year: 2026
Type: preprint
Abstract: Language model requests often contain near-duplicates of earlier prompts. While autoregressive models must regenerate the entire response for each request, masked diffusion language models can place a previous answer on their canvas and only regenerate the segments changed by the new input. We investigate the conditions under which this method succeeds and the specific requirements for its implementation. Reuse is not free. Naive reuse fails due to length exposure bias, which occurs because training provides spans where the number of masks always equals the content length, leading the model to treat a slot's length as a factual constraint. LLaDA-8B restores between 94-100% of its correct numbers when using a slot of the correct length, but this drops to 0-47% if the slot is one token longer (compared to 0-36% for Dream-7B). Furthermore, a JSON scaffold with generous slots fails to recover any of 255 real fields correctly. To address this, we learned a NULL mode for 37 minutes on a laptop GPU which transforms the slot length from an equality into an upper bound, improving accuracy for real fields with two spare tokens from 16% to 93%. By using this mode, residual decoding solves 85.3% of 300 held-out near-duplicate math problems, which outperforms the 74.3% accuracy achieved by decoding from scratch (p < 0.001) and performs at a level similar to the cached answer provided in the prompt at 85.7%. This approach achieves these results using only an eighth of the forward passes and a tenth of the wall time. We also observe that residual decoding is more robust than the prompt itself, as an irrelevant clause causes a 13.1 point drop in the prompt's performance but only a 2.6 point drop for residual decoding (83.7% vs 73.9%, p = 0.028). A learned conflict head allows reuse for reworded requests in one additional pass, achieving 91.5% accuracy compared to 79.7% for scratch and 82.4% for the prompt (both p ≤ 0.004) while using 0.24x the passes of scratch. When testing on 1,795 real GSM-Plus perturbations, verifying the canvas answer once under the prompt achieves 77.7% compared to 74.4% for the prompt alone (p < 0.0001) at 0.578x the prompt's passes. In free-form chat, reuse reaches the same performance as scratch at 0.31x the passes but provides no advantage over a plain semantic cache. All targets were fixed before testing on sets that no system had previously decoded, and every miss is reported.
Language: English
Open Access status: green
OpenAlex ID: W7233598281
Cites:
Cited by:
Related to: