Year: 2015
Type: conference-paper
Abstract: Multi-modal semantics has relied on feature norms or raw image data for perceptual input.In this paper we examine grounding semantic representations in raw auditory data, using standard evaluations for multi-modal semantics, including measuring conceptual similarity and relatedness.We also evaluate cross-modal mappings, through a zero-shot learning task mapping between linguistic and auditory modalities.In addition, we evaluate multimodal representations on an unsupervised musical instrument clustering task.To our knowledge, this is the first work to combine linguistic and auditory information into multi-modal representations.
DOI: 10.18653/v1/d15-1293
Authors: Douwe Kiela, Stephen Clark
Institution: University of Cambridge
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W2251885125
