Work

Natural Language-Driven Viewpoint Navigation for Volume Exploration via Semantic Block Representation

Year: 2025

Type: article

Abstract: Exploring volumetric data is crucial for interpreting scientific datasets. However, selecting optimal viewpoints for effective navigation can be challenging, particularly for users without extensive domain expertise or familiarity with 3D navigation. In this paper, we propose a novel framework that leverages natural language interaction to enhance volumetric data exploration. Our approach encodes volumetric blocks to capture and differentiate underlying structures. It further incorporates a CLIP Score mechanism, which provides semantic information to the blocks to guide navigation. The navigation is empowered by a reinforcement learning framework that leverage these semantic cues to efficiently search for and identify desired viewpoints that align with the user's intent. The selected viewpoints are evaluated using CLIP Score to ensure that they best reflect the user queries. By automating viewpoint selection, our method improves the efficiency of volumetric data navigation and enhances the interpretability of complex scientific phenomena.

Source: IEEE Transactions on Visualization and Computer Graphics

DOI: 10.1109/tvcg.2025.3634651

Authors: Xuan Zhao, Jun Tao

Institution: Sun Yat-sen University

Language: English

Topic: Data Visualization and Analytics

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: National Natural Science Foundation of China

Awards: G1989875989, G3450232332