AudioMNIST: Exploring Explainable Artificial Intelligence for audio analysis on a simple benchmark
Year: 2023
Type: article
Abstract: Explainable Artificial Intelligence (XAI) is targeted at understanding how models perform feature selection and derive their classification decisions. This paper explores post-hoc explanations for deep neural networks in the audio domain. Notably, we present a novel Open Source audio dataset consisting of 30,000 audio samples of English spoken digits which we use for classification tasks on spoken digits and speakers' biological sex. We use the popular XAI technique Layer-wise Relevance Propagation (LRP) to identify relevant features for two neural network architectures that process either waveform or spectrogram representations of the data. Based on the relevance scores obtained from LRP, hypotheses about the neural networks' feature selection are derived and subsequently tested through systematic manipulations of the input data. Further, we take a step beyond visual explanations and introduce audible heatmaps. We demonstrate the superior interpretability of audible explanations over visual ones in a human user study.
Authors: Sören Becker, Johanna Vielhaben, Marcel Ackermann, Klaus-Robert Müller, Sebastian Lapuschkin, Wojciech Samek
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $2,700
Funders: European Commission, Deutsche Forschungsgemeinschaft, Bundesministerium für Bildung und Forschung, Ministry of Science, ICT and Future Planning, Bundesministerium für Bildung, Wissenschaft, Forschung und Technologie, Horizon 2020 Framework Programme, Toyota Motor Engineering and Manufacturing North America, HORIZON EUROPE Framework Programme, Institute for Information and Communications Technology Promotion, HORIZON EUROPE European Innovation Council
Awards: G1395773357, G1585467943, G1808290430, G2033548659, G2959821122, G3340689995, G3479333676, G3515117348, G3525334839, G430232657, Verbundprojekt: Bernstein Fokus Neurotechnologie - Nichtinvasive Neurotechnologie für Mensch-Maschine Interaktion - Teilprojekte A1, A3, A4, B4, W3, Zentrum, G5030270982, Verbundprojekt BIFOLD-BZML: Berlin Institute for the Foundations of Learning and Data, G5441272027, D-JPN Verbund: Adaptive Gehirn-Computer-Schnittstellen (BCI) in nichtstationären Umgebungen, Intelligent Total Body Scanner for Early Detection of Melanoma, Trusted Extremely Precise Mapping and Prediction for Emergency Management, G7261992824, Verbundprojekt BIFOLD-BBDC: Berlin Institute for the Foundations of Learning and Data, EXC 2046: MATH+: Berlin Mathematics Research Center, G7958756422, Verbundprojekt: TraMeExCo - Transparenter Begleiter für medizinische Anwendung
OpenAlex ID: W4389619460
