Work

Dream4D: Lifting Camera‐Controlled I2V towards Spatiotemporally Consistent 4D Generation

Year: 2026
Type: article
Abstract: Generating spatiotemporally consistent 4D content requires joint modeling of high‐fidelity spatial details and realistic temporal dynamics. However, existing methods typically lack world knowledge learned from expansive 4D observations. As a result, generated scenes often struggle to maintain pose and temporal consistency in complex dynamic environments, leading to degraded generation quality. To address these challenges, we propose Dream4D , a novel 4D generation framework that integrates spatiotemporal priors from video generation with explicit sequential camera control. Our method adopts a three‐stage pipeline. We first employ a vision–language model (VLM) to determine the camera trajectory that best captures the intended view from the input image and the instruction text. Next, a pose‐conditioned diffusion model generates geometrically consistent image sequences along this trajectory. Finally, we use an advanced 4D generator to reconstruct a coherent and persistent 4D representation conditioned on the generated video. Dream4D jointly leverages the rich temporal priors of video diffusion models and the geometric awareness of reconstruction models to achieve high‐quality image‐to‐4D generation. Extensive experiments demonstrate that our approach significantly outperforms existing methods in both pose and temporal consistency.
Language: English
Open Access status: green
OpenAlex ID: W4416975529
FWCI: 1.462
Cites: 43
Cited by: 1
Related to: 10