Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi-Label Classification
Year: 2021
Type: preprint
Abstract: Multi-label classification (MLC) is a prediction task where each sample can have more than one label. We propose a novel contrastive learning boosted multi-label prediction model based on a Gaussian mixture variational autoencoder (C-GMVAE), which learns a multimodal prior space and employs a contrastive loss. Many existing methods introduce extra complex neural modules like graph neural networks to capture the label correlations, in addition to the prediction modules. We find that by using contrastive learning in the supervised setting, we can exploit label information effectively in a data-driven manner, and learn meaningful feature and label embeddings which capture the label correlations and enhance the predictive power. Our method also adopts the idea of learning and aligning latent spaces for both features and labels. In contrast to previous works based on a unimodal prior, C-GMVAE imposes a Gaussian mixture structure on the latent space, to alleviate the posterior collapse and over-regularization issues. C-GMVAE outperforms existing methods on multiple public datasets and can often match other models' full performance with only 50% of the training data. Furthermore, we show that the learnt embeddings provide insights into the interpretation of label-label interactions.
Source: arXiv (Cornell University)
Authors: Bai, Junwen, Kong, Shufeng, Gomes, Carla P.
Institution: Cornell University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G1743021977, G184040687, G3141641755, Collaborative Research: CompSustNet: Expanding the Horizons of Computational Sustainability, G7452299184, II-EN: Computing research infrastructure for constraint optimization, machine learning, and dynamical models for computational sustainability, G8202728730
OpenAlex ID: W3215060717