GAN-BERT: Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples
Year: 2020
Type: conference-paper
Abstract: Recent Transformer-based architectures, e.g., BERT, provide impressive results in many Natural Language Processing tasks.However, most of the adopted benchmarks are made of (sometimes hundreds of) thousands of examples.In many real scenarios, obtaining highquality annotated data is expensive and timeconsuming; in contrast, unlabeled examples characterizing the target task can be, in general, easily collected.One promising method to enable semi-supervised learning has been proposed in image processing, based on Semi-Supervised Generative Adversarial Networks.In this paper, we propose GAN-BERT that extends the fine-tuning of BERT-like architectures with unlabeled data in a generative adversarial setting.Experimental results show that the requirement for annotated examples can be drastically reduced (up to only 50-100 annotated examples), still obtaining good performances in several sentence classification tasks.
Institutions: University of Rome Tor Vergata, Amazon (United States)
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W3035454789
