Work

Code Reproducibility Check: Integrating Attention Tracking (Eye/Head/Motion) and LLMs in VR Tutorial

Year: 2026

Type: article

Abstract: Virtual environments offer ecologically valid experimental settings while maintaining experimental control and supporting generalizability. This tutorial presents a Unity-based system that integrates attention tracking with large language models (LLMs) to generate contextual descriptions of attended objects in extended reality (XR) environments. To provide a practical, reproducible workflow integrating desktop-based simulated attention tracking, immersive head and controller tracking, and hardware-based eye tracking to deliver AI-generated contextual feedback. This tutorial includes implementation scripts, platform configuration guidance, a demonstration video, troubleshooting information, and project materials. This novel method automates the extraction of visual and contextual information from XR environments in real time, reducing manual annotation workload and enabling more efficient and scalable data generation.

Source: Zenodo (CERN European Organization for Nuclear Research)

DOI: 10.5281/zenodo.23764113

Authors: Li, Yue, Yağız Canıgüroğlu, Ali, Yildirim, Funda

Institution: University of Twente

Language: English

Topic: Gaze Tracking and Assistive Technology

Subfield: Human-Computer Interaction

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W7224503756

FWCI:

Cites:

Cited by:

Related to: 10