Work

Improving and Simplifying Pattern Exploiting Training

Year: 2021

Type: conference-paper

Abstract: Recently, pre-trained language models (LMs) have achieved strong performance when finetuned on difficult benchmarks like Super-GLUE.However, performance can suffer when there are very few labeled examples available for fine-tuning.Pattern Exploiting Training (PET) is a recent approach that leverages patterns for few-shot learning.However, PET uses task-specific unlabeled data.In this paper, we focus on few shot learning without any unlabeled data and introduce ADAPET, which modifies PET's objective to provide denser supervision during fine-tuning.As a result, ADAPET outperforms PET on Su-perGLUE without any task-specific unlabeled data.

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/v1/2021.emnlp-main.407

Authors: Derek Tam, Rakesh R. Menon, Mohit Bansal, Shashank Srivastava, Colin Raffel

Institutions: University of North Carolina at Chapel Hill, Toyota Technological Institute at Chicago

Language: English

Topic: Domain Adaptation and Few-Shot Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Science Foundation, Defense Advanced Research Projects Agency, Office of Naval Research

Awards: G1562734358, CAREER: Semantic Multi-Task Learning for Generalizable and Interpretable Language Generation, G6671297155, G8876996369