Work

Towards Explainable Person-of-Interest-based Audio Synthesis Detection

Year: 2025

Type: conference-paper

Abstract: Generalization and explainability are two key challenges in synthetic audio detection. Effective detectors should not only reliably classify unseen data from unknown synthesis algorithms, but also provide insight into their decision-making process and explain why a given input was classified as real or fake. To promote generalization we use the Person-of-Interest approach, which allows us to detect synthetic audio using a model trained only on real data, provided that some pristine audio of the putative speaker is provided. To support explainability, we instead use an encoder-decoder backbone such that the bottleneck features ensure syntactic and semantic fidelity to the input, as well as enable reliable decisions. Experiments show that our approach outperforms both state-of-the-art models based on supervised learning and methods based on speaker verification.

Source: Zenodo (CERN European Organization for Nuclear Research)

DOI: 10.5281/zenodo.16993619

Authors: Pianese, Alessandro, Cuccovillo, Luca, Poggi, Giovanni, Le Roux, Thomas, Aichroth, Patrick

Institutions: University of Naples Federico II, Fraunhofer Institute for Digital Media Technology

Language: English

Topic: Speech Recognition and Synthesis

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: European Commission

Award: vera.ai: VERification Assisted by Artificial Intelligence