Sonification of Multimodal Medical Imaging with Temporal Dynamics for Diagnostic Augmentation
Year: 2026
Type: preprint
Abstract: In modern medical imaging, data is highly complex and extensive, and processing by the human eye is not efficient. Radiologists face a high cognitive load and are prone to fatigue when reading multimodality studies, which may include MRI, CT and DCE-MRI. Such imaging data is rich in temporal and spatial patterns that may not be recoverable through visual observation alone. This work formulates diagnostic sonification as a cross-modal encoding problem: learning a map from a high-dimensional multimodal volumetric imaging feature space to a low-dimensional perceptual audio code that preserves diagnostically relevant structure while remaining perceptually discriminable. We state the encoding objective as a composite of diagnostic, perceptual, ambiguity and smoothness terms, and give a complete, fully determined hand-specified instantiation of the encoder — covering tissue intensity, contrast enhancement, boundary gradient, texture, spatial position, and temporal dynamics including contrast wash-in and wash-out — which serves as both an interpretable baseline and an initialization for the learned encoder. This manuscript reports the design stage: problem formulation, a survey of medical and physiological sonification, a technology-gap analysis, characterization of two public imaging datasets, the full encoder specification, and an evaluation protocol for the empirical study that follows. That study is pre-registered here: ten encoder-level experiments requiring no human listeners, six directional hypotheses with explicit falsification criteria, fixed decision rules, and a perceptual-validity gate that determines how the primary result may be framed. The learned encoder is specified but not trained, and no empirical results are reported in this manuscript.
Author: GUPTA, ADISHREE
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7203855565
Cites:
Cited by:
Related to: