Work

Cross-Modal Similarity Learning via Pairs, Preferences, and Active Supervision

Year: 2015

Type: conference-paper

Abstract: We present a probabilistic framework for learning pairwise similarities between objects belonging to different modalities, such as drugs and proteins, or text and images. Our framework is based on learning a binary code based representation for objects in each modality, and has the following key properties: (i) it can leverage both pairwise as well as easy-to-obtain relative preference based cross-modal constraints, (ii) the probabilistic framework naturally allows querying for the most useful/informative constraints, facilitating an active learning setting (existing methods for cross-modal similarity learning do not have such a mechanism), and (iii) the binary code length is learned from the data. We demonstrate the effectiveness of the proposed approach on two problems that require computing pairwise similarities between cross-modal object pairs: cross-modal link prediction in bipartite graphs, and hashing based cross-modal similarity search.

Source: Proceedings of the AAAI Conference on Artificial Intelligence

DOI: 10.1609/aaai.v29i1.9599

Authors: Yi Zhen, Piyush Rai, Hongyuan Zha, Lawrence Carin

Institutions: Georgia Institute of Technology, Duke University

Language: English

Topic: Advanced Image and Video Retrieval Techniques

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funders: National Science Foundation, U.S. Department of Energy, Defense Advanced Research Projects Agency, Office of Naval Research

Award: III: Small: Exploring Social and Behavioral Contexts for Information Retrieval