Preserving Label Correlation for Multi-label Text Classification by Prototypical Regularizations
Year: 2025
Type: conference-paper
Abstract: Multi-label text classification (MLTC) assigns multiple labels to a sentence, with the key challenge being capturing label correlations. Existing models prioritize leveraging correlations but often overlook overfitting, while plug-and-play regularization methods fail to preserve correlations effectively. In this paper, we distinguish two types of label correlations: explicit co-occurring correlations and implicit semantic correlations, and propose regularizations on prototypical label embeddings for correlation preservation. Specifically, we first generate the prototypical embedding of multiple co-occurred labels as an intermediate. We then apply a prototypical regularization on the distance between the sentence embedding and corresponding prototypical embedding to alleviate the over-alignment issue caused by binary cross entropy loss and facilitate explicit correlation preservation. We finally extend the vanilla Mixup, which solely mixes multi-hot labels, on prototypical embedding mixing to promote implicit correlation preservation. Empirical studies show the effectiveness of our regularization methods.
Authors: Fanshuang Kong, Richong Zhang, Xiaohui Guo, Junfan Chen, Ziqiao Wang
Institutions: Beihang University, Tongji University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Award: G1421742111
OpenAlex ID: W4409670781