Work

Techniques for customized binaural audio rendering with applications to virtual rehabilitation

Year: 2012
Type: dissertation
Abstract: Multimodal interfaces represent a key factor for enabling an inclusive use of new technologies by everyone. To achieve this, realistic models that describe our environment are of topical importance, in particular models that accurately describe the acoustics of the environment and communication through the auditory modality. Models for spatial (or 3-D) audio can provide accurate information about the relation between the sound source and the surrounding environment, and this information cannot be substituted by any other modality. However, being multimedia systems currently focused mostly on graphics processing and integrated with simple stereo or surround sound, today’s spatial representation of audio tends to be simplistic and with poor interaction potential. Furthermore, current auralization technologies rely on invasive and/or expensive reproduction devices (e.g. head-mounted displays, loudspeakers), which cause the user to perceive a non-integrated experience due to an unbridged gap between the real and virtual worlds. On a much different level lie binaural sound rendering approaches (i.e. based on headphone reproduction). Most of the binaural rendering techniques currently exploited in research rely on the use of the so-called Head-Related Transfer Functions (HRTFs), i.e. peculiar filters that capture the transformations undergone by a sound wave in its path from the source to the eardrum and typically due to reflection and diffraction effects on the torso, head, shoulders and pinnae of the listener. Such characterization allows virtual positioning of sound sources in the surrounding space by filtering the desired signals through a pair of HRTFs, thus creating left and right ear signals to be delivered by headphones. In this way, three-dimensional sound fields with a high immersion sense can be simulated and integrated within multimodal frameworks. However, such techniques bear relevant limitations. First, they may request considerably large computational resources, especially in the case where one needs to simulate several sound sources in the surrounding space. Second, and most important, HRTF filters are usually presented under the form of acoustic signals recorded through dummy heads: this means that anthropometric differences among different subjects are not taken into account. Contrariwise, along with the critical relative position between listener and sound source, anthropometric features of the human body have a key role in HRTF characterization: while non-individualized HRTFs represent a cheap and straightforward mean of providing 3-D perception in headphone reproduction, listening to non-individualized spatialized sounds may likely result in evident sound localization errors such as incorrect perception of source elevation, front-back reversals, and lack of externalization, especially in static conditions. On the other hand, individual HRTF measurements on a significant number of subjects is often both time- and resource-expensive. Structural modeling of HRTFs ultimately represents an attractive solution to these shortcomings. As a matter of fact, if one isolates the contributions of the listener’s head, pinnae, ear canals, shoulders, and torso to the HRTF in different subcomponents - each accounting for some well-defined physical phenomenon - then, thanks to linearity, he can reconstruct the global HRTF from a proper combination of all the considered effects. This thesis presents one such model that can be employed for immersive sound reproduction, with a particular focus on the pinna contribution to the HRTF. The pinna plays a primary part in the perception of source elevation by introducing major spectral modifications, yet the relation between acoustic phenomena due to the pinna - mainly resonances and sound reflections - and anthropometry has not been understood up to date. Instead, a promising correspondence between reflection points on pinna surfaces and frequencies of notches occurring in the high-frequency range of the HRTF spectrum is formally found here. Such a relevant result allows for an interesting form of content adaptation and customization of the structural model, as it includes parameters related to the user’s anthropometry in addition to the spatial ones. The proposed approach has also implications in terms of delivery, since it operates by processing a monophonic signal exclusively at the receiver side (e.g., on a terminal or mobile device) by means of low-order filters, allowing for reduced computational costs. Thanks to its low complexity, the model can be used to render scenes with multiple audiovisual objects in a number of contexts such as computer games, cinema, edutainment, and any other scenario where realistic sound spatialization and personalized sound reproduction is a major requirement. Remarkably, the specific areas for which the proposed model is thought for are those of virtual rehabilitation and rehabilitation robotics, two of the most potentially interesting application fields for research in sonic interaction design today. The final goal of research in these areas is to facilitate re-integration of patients with neurological disorders into social and domestic life by helping them regain the ability to autonomously perform activities of daily living (ADLs, e.g., eating, or walking); however, much work is still needed to address challenges related to hardware, software, control system design, as well as effective approaches for delivering treatment. As a matter of fact, ADLs embody complex motor tasks for which current rehabilitation systems lack the sophistication needed in order to assist patients during their performance. In particular, it is recognized that a large number of degrees of freedom ought to be used in robot-assisted rehabilitation, and that multimodal feedback often plays a key role in both forementioned application fields. Although several rehabilitation systems which make use of multimodal virtual environments with visual and haptic feedback already exist, the consistent use of auditory feedback is less investigated. A thorough analysis of literature reported in this thesis confirms this impression, showing that the potential of auditory feedback is largely underestimated in such systems. Five different proposed experiments allow investigation of the role that novel auditory feedbacks presented during gait training and tracking movements play in improving performance in healthy participants, providing a basis for a future comparison with neurologically injured patients. In particular, usefulness of task-related sound feedback and sound spatialization in coordinating the user’s movements during simple target following tasks is attested. Results thus suggest that constructive and well-designed multimodal feedback can definitely be used to improve performance and learning in complex motor tasks, thanks to the high level of attention, engagement, and presence provided to the user. Such studies represent a novelty in the current literature on virtual rehabilitation and rehabilitation robotics, especially concerning the use of sonification techniques to convey information in a rehabilitation scenario.
Language: English
Open Access status: green
OpenAlex ID: W1369664
Cites: 127
Cited by: 1
Related to: 10