Year: 2021
Type: dissertation
Abstract: Extended reality (XR) is a continuum encompassing various current and future virtual technologies such as augmented reality (AR) and virtual reality (VR). These technologies have been shown to increase learning outcomes when applied to training. For example, virtual and augmented reality have been shown to improve performance during assembly task training over traditional methods. However, even an advanced technology like XR won’t provide a benefit if the information is ill-timed or poorly suited to the problem. One potential way to solve this issue is with adaptive automation. Adaptive automation provides the ability to off-load tasks from a human to a computer. It has been shown to increase productivity with the added benefit of reducing human-labor costs. Previous research has shown that applying automation at the wrong levels or during the wrong stage of human information processing can result in decreased performance. Furthermore, if there are inaccuracies or gaps in the automated system, resulting in brittle automation, a human can develop an overreliance on the automation leading to costly mistakes. One place where adaptive automation is already being applied to XR technology is in the field of intelligent tutoring systems (ITS). Designers of XR ITS are increasingly automating the role of a human instructor in an effort to decrease the instructor to learner ratio. While works touting the development of this type of adaptive XR training systems exist, few address the design and efficacy of adaptations that do not directly pertain to the learning materials and pedagogy such as troubleshooting software and hardware use. Without this information, it is impossible for today’s researchers and developers to know how to design and implement these types of adaptive XR training systems. Therefore, this dissertation will identify and evaluate effective triggers and adaptations for XR training environments so that they can be replicated for future applications. This research sought to solve this problem in two stages. In the first stage, human XR instructors were used as models for potential adaptive automation schema. Since automation is the computer execution of a task once performed by a human, it was natural to choose human instructors as models for this work. Therefore, 11 semi-structured interviews were conducted with XR training and simulation experts. The questions posed during the interviews were crafted to determine how these instructors identified and mitigated confusion in learners. The information from these interviews was then analyzed for themes and synthesized with existing adaptive automation models. During the analysis phase, identifying and mitigating confusion were found to be analogous to the functions of triggers and adaptations in the adaptive automation model. These interviews resulted in two sets of recommendations. First, when designing adaptive triggers in XR, verbal triggers should be prioritized, followed by physical triggers. A head-mounted display (HMD) can be used to monitor verbal and physical triggers using a factory standard microphone and inertial measurement unit. Second, verbal and physical channels should also be prioritized when designing adaptation methods. This can be done through recorded audio messages, textual interfaces, and by providing demonstrative animations and models in an XR environment. Finally, adaptations with increasing levels of specificity and intrusiveness allow learners to solve problems independently. The second phase of this research implemented these recommendations within an XR task. A simple triangle completion task was chosen, in which a learner must teleport to three different positions in a VE using two different techniques. Afterwards, the learner is tested on their ability to remember where they started by pointing at the origin and clicking using an XR controller. Triggers and adaptations were developed to assist in the correction of five different erroneous behaviors during the task. These included incorrect button presses, inaction, and attempting to teleport to the wrong location. Finally, the resultant unsupervised, remote, adaptive XR simulation was evaluated to determine the efficacy of the adaptations and compared to a version of the simulation with no adaptations and a human instructor present to provide feedback, when necessary. The results of this experiment validated that the adaptations were sufficient at providing instructions to participants during the remote unmoderated study because the participant success rate was equal to that of the lab-based study (89%). In addition, participants’ performance and completion times were not statistically different from those of the lab group. The adaptations triggered when expected and had the intended effect of helping learners correct their mistakes. Finally, participants gave feedback about the intrusiveness, helpfulness, and quantity of adaptive feedback they received. It was found that the quantity of feedback was adequate without being too intrusive, however, there were mixed reviews about the subjective helpfulness of the feedback. Ultimately, this research was successful at increasing the efficacy of adaptive XR systems, and reducing the time and cost associated with humans facilitating XR training.
Author: Melynda Tess Hoover
Institution: Iowa State University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Keywords: adaptive automation, extended reality, augmented reality, virtual reality, instructor modeling, physical triggers, head-mounted displays, intelligent tutoring systems, adaptive training systems, human-automation interaction, expert interviews, training system design, training system evaluation
SDG: Quality education
Open Access status: gold
Funder: Academy of Finland
OpenAlex ID: W4393304404
