Work

Text-driven light-field content editing for three-dimensional light-field display based on Gaussian splatting

Year: 2024

Type: article

Abstract: Three-dimensional (3D) light-field displays can provide natural stereoscopic visual perception and an intuitive viewing experience. However, the high production threshold and the lack of user-friendly editing tools for light-field images make it difficult to efficiently and conveniently generate 3D light-field content that meets various needs. Here, a text-driven light-field content editing method for 3D light-field display based on Gaussian splatting is presented. The selected key views propagate the editing effects to other views through perception fusion, avoiding time-consuming editing iterations. A designed refinement module employs attention-based latent feature alignment to enhance consistency across multi-views, while multi-channel independent update and average normalization bring more stable and reliable editing results. With the efficient light-field coding method based on splatting, 3D light-field images can be directly generated from the edited Gaussian scene space that is optimized via a generation control training strategy. Extensive experimental results demonstrate that the proposed method can rapidly generate high-quality 3D light-field content that aligns with the given text editing instructions while providing two optional user interaction extensions to make the editing results more controllable and flexible.

Source: Optics Express

DOI: 10.1364/oe.547233

Authors: Luyu Ji, Xinzhu Sang, Shujun Xing, Xunbo Yu, Binbin Yan, Jiahui Yang

Language: English

Topic: Advanced Optical Imaging Technologies

Subfield: Media Technology

Field: Engineering

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,270

Funders: National Natural Science Foundation of China, Shenzhen Science and Technology Innovation Program

Awards: G2077793873, G5229087831