An Effective Deployment of Diffusion LM for Data Augmentation in Low-Resource Sentiment Classification
Year: 2024
Type: conference-paper
Abstract: Sentiment classification (SC) often suffers from low-resource challenges such as domainspecific contexts, imbalanced label distributions, and few-shot scenarios.The potential of the diffusion language model (LM) for textual data augmentation (DA) remains unexplored, moreover, textual DA methods struggle to balance the diversity and consistency of new samples.Most DA methods either perform logical modifications or rephrase less important tokens in the original sequence with the language model.In the context of SC, strong emotional tokens could act critically on the sentiment of the whole sequence.Therefore, contrary to rephrasing less important context, we propose DiffusionCLS to leverage a diffusion LM to capture in-domain knowledge and generate pseudo samples by reconstructing strong label-related tokens.This approach ensures a balance between consistency and diversity, avoiding the introduction of noise and augmenting crucial features of datasets.Dif-fusionCLS also comprises a Noise-Resistant Training objective to help the model generalize.Experiments demonstrate the effectiveness of our method in various low-resource scenarios including domain-specific and domain-general problems.Ablation studies confirm the effectiveness of our framework's modules, and visualization studies highlight optimal deployment conditions, reinforcing our conclusions.
Authors: Zhuowei Chen, Lianxi Wang, Yuben Wu, Xinfeng Liao, Yujia Tian, Junyang Zhong
Institution: Guangdong University of Foreign Studies
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4404782680