Work

Multimodal Metric Learning for Tag-Based Music Retrieval

Year: 2021

Type: conference-paper

Abstract: Tag-based music retrieval is crucial to browse large-scale mu-sic libraries efficiently. Hence, automatic music tagging has been actively explored, mostly as a classification task, which has an inherent limitation: a fixed vocabulary. On the other hand, metric learning enables flexible vocabularies by using pretrained word embeddings as side information. Also, met-ric learning has proven its suitability for cross-modal retrieval tasks in other domains (e.g., text-to-image) by jointly learning a multimodal embedding space. In this paper, we investigate three ideas to successfully introduce multimodal metric learning for tag-based music retrieval: elaborate triplet sampling, acoustic and cultural music information, and domain-specific word embeddings. Our experimental results show that the proposed ideas enhance the retrieval system quantitatively and qualitatively. Furthermore, we release the MSD500: a subset of the Million Song Dataset (MSD) containing 500 cleaned tags, 7 manually annotated tag categories, and user taste profiles.

Source: IEEE International Conference on Acoustics Speech and Signal Processing

DOI: 10.1109/icassp39728.2021.9413514

Authors: Minz Won, Sergio Oramas, Oriol Nieto, Fabien Gouyon, Xavier Serra

Institutions: Universitat Pompeu Fabra, Pandora (United States)

Language: English

Topic: Music and Audio Processing

Subfield: Signal Processing

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Ministerio de Economía y Competitividad, Ministry of Economy

Award: G3402147602