Work

Audio Deepfake Detection With hybrid CNN and ViT

Year: 2025
Type: dissertation
Abstract: Audio deepfakes, generated by automated speech synthesis and voice conversion software, pose a growing threat to digital safety, privacy and the integrity of various media. Audio deepfaking is less studied than visual deepfaking, despite may challenges faced under varied noisy environments. This study attempts to address this gap by examining the efficacy of four architectures for detecting synthetic audio: Convolutional Neural Networks (CNN), Convolutional Recurrent Neural Networks (CRNN), Mobile Vision Transformer (MobileViT), and the Patchout Spectrogram Transformer (PaSST). This study trained and evaluated the models using the FakeAVCeleb dataset, supplemented with additional samples of hospital noise. Our models were tested in both clean (audio) and noisy environments, and in order to improve robustness we applied RandAugment to CNN and CRNN; RandAugment generates spectrogram distortions to improve sample diversity in training. The experimental data showed that while all models performed with high accuracy and F1-scores above 0.97 in clean audio content, measured performance markedly degraded in noisy inputs. Where CNN and CRNN fall below 0.56 F1-scores, the MobileViT and PaSST took slight performance drops with F1-scores above 0.60 and above 0.55 respectively. The findings from this study highlight the sensitivity of modern detection systems to background noise, as well as the merits of transformer based architectures in real-world conditions. This study highlights the power of data augmentation and hybrid architectures to create strong and workable audio deepfake detection systems, using a systematic comparison of these models.
Language: English
Open Access status: green
OpenAlex ID: W7167305126
Cites: 11
Cited by:
Related to: 10