Work

Evaluation Of Algorithms Using Games: The Case Of Music Tagging.

Year: 2009

Type: article

Abstract: Search by keyword is an extremely popular method for retrieving music. To support this, novel algorithms that automatically tag music are being developed. The conventional way to evaluate audio tagging algorithms is to compute measures of agreement between the output and the ground truth set. In this work, we introduce a new method for evaluating audio tagging algorithms on a large scale by collecting set-level judgments from players of a human computation game called TagATune. We present the design and preliminary results of an experiment comparing five algorithms using this new evaluation metric, and contrast the results with those obtained by applying several conventional agreement-based evaluation metrics. 1.

DOI: 10.5281/zenodo.1417646

Authors: Law, Edith, West, Kris, Mandel, Michael I., Bay, Mert, J. Stephen Downie

Institutions: Carnegie Mellon University, University of Illinois Urbana-Champaign, Columbia University

Language: English

Topic: Music and Audio Processing

Subfield: Signal Processing

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W2127870748

FWCI: 6.057

Cites: 3

Cited by: 167

Related to: 10