Query-by-Example Ljudsök med Akustiska Ordinbäddningar : Omvandling av wav2vec 2.0-inbäddningar med Kontrastiv Inlärning
Year: 2025
Type: dissertation
Abstract: The daily creation and consumption of audio and video content has grown dramatically, making large-scale speech analysis essential for trendspotting and market analysis. While transcription-based models are common, they are often costly and prone to errors, especially in low-resource languages. Query-by-Example (QbE) audio search offers an intuitive way to retrieve spoken content by using audio queries instead of text. In this thesis, we investigate how contrastive embedding learning can improve clustering of acoustic word embeddings, to filter out speaker characteristics and background noise, and to enhance QbE search performance. Starting from pre-trained wav2vec 2.0 embeddings, we train a projection network using triplet loss to better align similar word instances while increasing separation between dissimilar words. We evaluate this approach in two phases: (1) a clustering evaluation, which measures the embedding space structure before and after transformation, and (2) a QbE-focused information retrieval evaluation to quantify improvements in word retrieval accuracy. Our experiments compare a Swedish pre-trained model named VoxRex to a multilingual model XLS-R and conclude that the Swedish model shows high potential for QbE audio search. In contrast, the XLS-R needs further work before being viable. This work contributes partly to going completely transcription-free in audio search, which is an ambitious but attainable goal that could save both time and computational costs for businesses in the automatic speech recognition field, and also to understanding how contrastive learning can be used to construct a phonetically structured embedding space.
Author: Öberg, Wilhelm
Institution: KTH Royal Institute of Technology
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7052207407
