Uncertainty-Aware Contrastive Sentence Embedding With Local Context Representation for Text Classification
Year: 2026
Type: conference-paper
Abstract: In real-world applications of natural language processing, it is essential to effectively adapt a pre-trained model to a downstream task.While text classification is undertaken as a downstream task, it is crucial to produce meaningful sentence embedding that is adaptive to the task.In this paper, we explore how to effectively adapt a pre-trained model for extracting meaningful context representations from sentences, and propose an uncertainty-aware contrastive sentence embedding approach that involves addressing language ambiguity and inter-class separability for a text classification task.Specifically, we design an end-to-end strategy for driving the process of learning to transform a word embedding matrix into a contextualized sentence vector and to quantify the representation uncertainty of the sentence, while the word embedding matrix is produced by a pre-trained model without fine-tuning, and a label-wise contrastive learning strategy is designed to enhance intra-class compactness and inter-class separability.The results on public data sets show that a considerable improvement of text classification accuracy is achieved by adopting the proposed approach in comparison with using those state-of-the-art methods 1 .
Authors: Han Liu, Jiaqing Zhan, Zhichao Chen, Qin Zhang
Institution: Shenzhen University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: G2809873295, G4070941038
OpenAlex ID: W7166864989