Work

Visual Generative Artificial Intelligence (GenAI) in Mixed Reality (MR): A Survey of Techniques, Interaction Designs, and Evaluations

Year: 2026

Type: article

Abstract: GenAI is increasingly reshaping spatial interactions in immersive environments, bridging diverse input and output modalities to create intelligent user experiences. This survey provides a structured examination of visual GenAI techniques adapted and applied to MR contexts, which include Virtual Reality (VR), and Augmented Reality (AR), emphasizing their roles in enabling rich, adaptive, and naturalistic interactions. We screened publications from international venues between 2018 and 2025, followed by an in-depth review of 102 papers. We focus on the applications of visual content creation in 2D&3D and vision-language models (VLMs), highlighting their integration practices within MR. Furthermore, we investigate the co-evolution of generative models and MR interaction design, including immersive visualization, controllable generation, and synthetic data creation. The survey concludes by outlining recent challenges and potential research directions to advance the synergy of visual GenAI and MR techniques.

Source: International Journal of Human-Computer Interaction

DOI: 10.1080/10447318.2026.2713822

Authors: Duotun Wang, Linjie Qiu, Linlin Cai, Yue Jiang, Mingming Fan, Zeyu Wang

Institutions: Hong Kong University of Science and Technology, The Hong Kong University of Science and Technology (Guangzhou), University of Utah

Language: English

Topic: Augmented Reality Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Natural Science Foundation of China, Basic and Applied Basic Research Foundation of Guangdong Province

Awards: G1604839531, G7763482670, G7774517483