Quantitative Factorization and Stability Guarantees for Decoupled Diffusion World Models
Year: 2026
Type: article
Abstract: Decoupled diffusion world models separate the prediction of future geometry from the synthesis of future appearance. In a staggered RGB--depth formulation, an action-conditioned model first predicts the next depth latent, after which an action-free model generates the RGB latent conditioned on that predicted depth and the RGB history. This architecture offers a useful route to exploiting action-labelled trajectories for dynamics while using larger collections of RGB--depth data for appearance learning. Its probabilistic justification, however, relies on conditional-independence approximations, and its practical performance can be affected by depth prediction error and autoregressive distribution shift.We develop a quantitative theory for this factorization. First, we prove an exact decomposition of the expected joint Kullback--Leibler divergence into two conditional mutual-information terms, measuring structural departures from the proposed conditional independences, and two conditional model-fitting terms. This identity distinguishes approximation error caused by the factorization from error caused by imperfect diffusion models. We derive an expected total-variation bound by Pinsker's inequality. Second, we establish a depth-conditioning robustness bound: under a stated regularity condition on the RGB conditional law, the divergence induced by substituting predicted depth for true depth is controlled by the mean-square depth prediction error. Third, we give finite-horizon bounds for autoregressive rollouts, both for coupled state trajectories and for probability distributions in Wasserstein distance. We also prove a path-space KL bound that accumulates one-step conditional modeling errors over a rollout horizon.The results are theoretical guarantees under explicit assumptions, not empirical claims about a particular trained system. We conclude with estimators, diagnostics, and controlled experiments designed to test the assumptions and quantify the bounds in diffusion world models. The framework provides a principled way to measure when staggered factorization is justified, how depth errors propagate into appearance, and how local model errors accumulate over time.
Source: Figshare
Authors: Rajit Chatterjea, Yunhu Nam
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7234237175
