Year: 2007
Type: dissertation
Abstract: Fusion of information from multiple modalities in Human Computer Interfaces \n(HCI) has gained a lot of attention in recent years, and has far reaching \nimplications in many areas of human-machine interaction. However, a major \nlimitation of current HCI fusion systems is that the fusion process tends to \nignore the semantic nature of modalities, which may reinforce, complement or \ncontradict each other over time. Also, most systems are not robust in \nrepresenting the ambiguity inherent in human gestures. In this work, we \ninvestigate an evidential reasoning based approach for intelligent multimodal \nfusion, and apply this algorithm to a proposed multimodal system consisting of \na Hand Gesture sensor and a Brain Computing Interface (BCI). There are three \nmajor contributions of this work to the area of human computer interaction. \nFirst, we propose an algorithm for reconstruction of the 3D hand pose given a \n2D input video. Second, we develop a BCI using Steady State Visually Evoked \nPotentials, and show how a multimodal system consisting of the two sensors can \nimprove the efficiency and the complexity of the system, while retaining the \nsame levels of accuracy. Finally, we propose an semantic fusion algorithm based \non Transferable Belief Models, which can successfully fuse information from \nthese two sensors, to form meaningful concepts and resolve ambiguity. We also \nanalyze this system for robustness under various operating scenarios.
Source: UWSpace (University of Waterloo)
Author: Bakkama Srinath Reddy
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: Pan African Materials Institute
OpenAlex ID: W2588081605