Examining the Effect of Crossmodal Perception and Implicit Memory on Gestural Input Behaviour
Year: 2019
Type: dissertation
Abstract: This thesis explores the influence of crossmodal perception and implicit memory on gestural input \nbehaviour in Human-Computer Interaction (HCI). Although there has been previous research \ninto gesture modelling and evaluation for screen-based interaction system, there is very limited \nresearch on how gesture inputs, which performed in a different spatial-temporal scale, can be \nenhanced by a perceptual phenomenon, crossmodal correspondences (CCs). \nInformed by the theoretical account of embodied cognition, this thesis seeks to contribute to \nunveiling the potential of leveraging embodied experience on CCs in gesture-related interactions \nwith two focuses: the relationship between perception and action, and between memory and \naction. \nThe sub-question 1 relates to the first focus. Two experiments in study 1 addressed this question \nby investigating the modulation effect of CCs on continuous arm-based gestural input. The \nsub-question 2 relates to the second focus. Three experiments in study 2 addressed this question \nby exploring whether and how implicit memory influence consecutive hand-based inputs. \nThe thesis then turn back to the main research question by combining the two foci. The \npurpose is to deepen the understand on how interaction task and the gestural motion quality \ncan be improved. Two experiments in study 3 extend the experimental approach of study 2 and \nexamine augmented crossmodal feedback and implicit memory priming with continuous handbased \nand arm-based gestural input respectively. \nAt this point, the results of the three studies highlighted the spatial characteristic of the crossmodal \ninformation employed, as well as the spatial accuracy of gestural performance. Finally, \na further study was conducted to evaluate the temporal performance of both the hand-based and \narm-based gestural input behaviours. \nThe thesis concludes by drawing together the results of the studies into a series of implications \nrelating to the design and application of embodied interaction. This work contributes to \nthe area of Multimodal Interaction and HCI by providing empirical evidence of how embodied \nperceptual experience can be leveraged to enhance gestural input quality and interaction performance. \nIt also provides a set of design implications and an application outlook to aid design and \ninspire future research.
Author: Feng, Feng
Institution: Queen Mary University of London
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7046913519
