Get started
Work

Richting Numerieke Acquisitie van Individuele Hoofdgerelateerde Overdrachtsfuncties uit Fotogrammetrische Scans voor Immersieve Audio-ervaringen

Year: 2025
Type: dissertation
Abstract: The growing adoption of Extended Reality (XR) technologies across various industries has highlighted the importance of spatial audio in enhancing the realism and immersiveness of virtual scenes. Spatial audio rendering plays a crucial role in enabling accurate sound localisation in interactive audiovisual XR experiences. This is achieved through binaural rendering based on Head-Related Transfer Functions (HRTF). However, current XR engines typically employ generic HRTF, which often lead to poor sound localisation and fail to match the accuracy provided by personalised data. This limitation emphasises the need for individualised HRTF, intrinsically linked to the users' unique anatomical features, mainly their ear shapes. Obtaining individualised HRTF on a large scale remains a critical challenge due to the extensive time, effort, and resources required for traditional acquisition techniques. These include acoustic measurements or numerical simulation tools, e.g. the Boundary Element Method (BEM) and Finite Element Method FEM), applied to detailed 3D scans of the user's anatomy. While these methods can produce accurate HRTF, they are impractical for widespread use. Consequently, alternative approaches for acquiring personalised HRTF are deemed necessary to establish a scalable and affordable framework for individual HRTF acquisition. Recent advancements in photogrammetry, combined with the high computational power of modern consumer-grade devices, offer promising alternatives for generating individualised HRTF at scale. Photogrammetry enables the creation of 3D models of a user using ubiquitous camera hardware, reducing the need for expensive equipment. However, several challenges persist with this approach, including inaccuracies in capturing complex geometries, particularly the intricate structures of the human pinnae, as well as the computational demands associated with calculating HRTF over a broad frequency range. Additionally, the role of acoustic impedance modelling in HRTF computation, which may influence the accuracy of spatial audio rendering, is not fully understood. This is partly due to technical difficulties in measuring the high-frequency acoustic impedance of human skin or dummy head ear materials, leaving this aspect as an open research area. This thesis aims to develop a comprehensive pipeline for acquiring perceptually accurate individual HRTF at scale, specifically targeting the enhancement of spatial audio rendering in XR environments. The research explores the potential of photogrammetry for head and ear geometry acquisition. Analyses conducted on the photogrammetric scan of a dummy head and its related HRTF demonstrate that the outcome provides relatively good horizontal plane spatial sound cues. However, elevation perception is severely hindered, which is attributed to geometrical errors in specific regions of the scanned pinnae, evaluated by comparison to reference laser-scanned data. The application of advanced BEM and FEM approaches to compute the HRTF on the acquired scans is evaluated. These techniques offer the advantage of a tunable error level, enabling more precise or efficient simulations. Furthermore, realistic acoustic impedance Boundary Conditions (BC) are applied to the modelled pinnae geometry in these simulations to assess the impact of this parameter on HRTF computations. While incorporating impedance modelling reduces the spectral deviation between measured and computed data in comparison to commonly employed rigid BC simulations, the perceptual effect of using realistic impedance BC is found to be low. In addition, the research explores the use of Deep Neural Networks (DNN) to reduce inaccuracies in photogrammetric ear scans, with the objective of improving the quality of HRTF derived from noisy photogrammetric data. The effectiveness of different DNN architectures is assessed. These models are adapted to the specific shape and error characteristics of the acquired geometries, targeting the denoising to the most acoustically relevant regions of the pinna. The results indicate that, while some DNN provide modest improvements, the benefits are marginal, and further optimisation is required to achieve larger gains in accuracy. Finally, the thesis presents the use of the numerically computed HRTF in immersive XR environments. Two applications are developed to showcase the use of HRTF for spatial audio rendering and enable the experimental assessment of the numerically acquired data. This can be used to further assess and validate the perceptual fit of the HRTF computed through the proposed framework, providing insights for the development of this and alternative HRTF personalisation techniques.
Source: Lirias
Author: Di Giusto, Fabio
Language: English
Open Access status: green
OpenAlex ID: W7113132157
Cites:
Cited by:
Related to: