Work

DeepVision: Heads-up Computing and AI in Education

Year: 2024

Type: conference-paper

Abstract: Heads-up computing together with AI can enhance in-class learning experiences. In this position paper, we propose the development of a multimodal AI system called DeepVision that integrates Automatic Speech Recognition (ASR), Large Language Models (LLM), Large Vision Models (LVM), Information Retrieval (IR) and Inclusive User Experience Design (IUX) to convert real-time lectures into multiple knowledge representations. These will be visualized on heads-up communication devices such as Augmented Reality (AR) and Mixed Reality (MR) devices. The initiative is a collaboration between Habitat Learn Limited (HLL) and the University of Southampton, leveraging HLL's existing software and extensive data repository to address the challenges of traditional and digital learning environments, especially for students with disabilities or language differences.

DOI: 10.1145/3675094.3678991

Authors: Yunjia Li, Haiming Liu, Mike Wald

Institution: University of Southampton

Language: English

Topic: Augmented Reality Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

SDGs: Quality education, Reduced inequalities

Open Access status: gold

Funder: Universitas Brawijaya

Award: G2326783353