Real-Time 3D Volume Deformation and Visualization by Integrating NeRF, PBD, and Parallel Resampling
Year: 2024
Type: article
Abstract: Research combining deep learning-based models and physical simulations is making important advances in the medical field. This extracts the necessary information from medical image data and enables fast and accurate prediction of deformation of the skeleton and soft tissue based on physical laws. This study proposes a system that integrates Neural Radiance Fields (NeRF), Position-Based Dynamics (PBD), and Parallel Resampling to generate 3D volume data, and deform and visualize them in real-time. NeRF uses 2D images and camera coordinates to produce high-resolution 3D volume data, while PBD enables real-time deformation and interaction through physics-based simulation. Parallel Resampling improves rendering efficiency by dividing the volume into tetrahedral meshes and utilizing GPU parallel processing. This system renders the deformed volume data using ray casting, leveraging GPU parallel processing for fast real-time visualization. Experimental results show that this system can generate and deform 3D data without expensive equipment, demonstrating potential applications in engineering, education, and medicine.
Authors: Sangmin Kwon, Sojin Jeon, Juni Park, Dasol Kim, Heewon Kye
Institution: Hansung University
Language: English
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
Funder: Hansung University
OpenAlex ID: W4401717153