Work

Towards a better understanding of sensory substitution : the theory and practice of developing visual-to-auditory sensory substitution devices

Year: 2014
Type: dissertation
Abstract: Visual impairment is a global and potentially devastating affliction. Sensory substitution \ndevices have the potential to lessen the impact of blindness by presenting vision via another \nmodality. The chief motivation behind each of the chapters that follow is the production of \nmore useful sensory substitution devices. The first empirical chapter (chapter two) \ndemonstrates the use of interactive genetic algorithms to determine an optimal set of \nparameters for a sensory substitution device based on an auditory encoding of vision (“the \nvOICe”). In doing so, it introduces the first version of a novel sensory substitution device which \nis configurable at run-time. It also presents data from three interactive genetic algorithm \nbased experiments that use this new sensory substitution device. Chapter three radically \nexpands on this theme by introducing a general purpose, modular framework for developing \nvisual-to-auditory sensory substitution devices (“Polyglot”). This framework is the fuller \nrealisation of the Polyglot device introduced in the first chapter and is based on the principle \nof End-User Development (EUD). In chapter four, a novel method of evaluating sensory \nsubstitution devices using eye-tracking is introduced. The data shows both that the copresentation \nof visual stimuli assists localisation and that gaze predicted an auditory target \nlocation more reliably than the behavioural responses. Chapter five explores the relationship \nbetween sensory substitution devices and other tools that are used to acquire real-time \nsensory information (“sensory tools”). This taxonomy unites a range of technology from \ntelescopes and cochlear implants to attempts to create a magnetic sense that can guide \nfurther research. Finally, in chapter six, the possibility of representing colour through sound is \nexplored. The existence of a crossmodal correspondence between (equi-luminant) hue and \npitch is documented that may reflect a relationship between pitch and the geometry of visible \ncolour space.
Author: Thomas D. Wright
Language: English
Open Access status: green
OpenAlex ID: W2558725545
Cites: 195
Cited by: 1
Related to: 9