Year: 2025
Type: preprint
Abstract: Understanding and reconstructing the complex geometry and motion of dynamic scenes from video remains a formidable challenge in computer vision. This paper introduces D4RT, a simple yet powerful feedforward model designed to efficiently solve this task. D4RT utilizes a unified transformer architecture to jointly infer depth, spatio-temporal correspondence, and full camera parameters from a single video. Its core innovation is a novel querying mechanism that sidesteps the heavy computation of dense, per-frame decoding and the complexity of managing multiple, task-specific decoders. Our decoding interface allows the model to independently and flexibly probe the 3D position of any point in space and time. The result is a lightweight and highly scalable method that enables remarkably efficient training and inference. We demonstrate that our approach sets a new state of the art, outperforming previous methods across a wide spectrum of 4D reconstruction tasks. We refer to the project webpage for animated results: https://d4rt-paper.github.io/.
Source: arXiv (Cornell University)
Authors: Zhang, Chuhan, Moing, Guillaume Le, Koppula, Skanda, Rocco, Ignacio, Momeni, Liliane, Xie, Junyu, Sun, Shuyang, Sukthankar, Rahul, Barral, Joëlle K., Hadsell, Raia, Ghahramani, Zoubin, Zisserman, Andrew, Zhang, Junlin, Sajjadi, Mehdi S. M.
Topic: Advanced Vision and Imaging
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7114770741
Cites:
Cited by:
Related to: