Year: 2021
Type: conference-paper
Abstract: Humans can learn a new language task efficiently with only few examples, by leveraging their knowledge obtained when learning prior tasks.In this paper, we explore whether and how such cross-task generalization ability can be acquired, and further applied to build better few-shot learners across diverse NLP tasks.We introduce CROSSFIT , a problem setup for studying cross-task generalization ability, which standardizes seen/unseen task partitions, data access during different learning stages, and the evaluation protocols.To instantiate different seen/unseen task partitions in CROSS-FIT and facilitate in-depth analysis, we present the NLP Few-shot Gym, a repository of 160 diverse few-shot NLP tasks created from openaccess NLP datasets and converted to a unified text-to-text format.Our analysis reveals that the few-shot learning ability on unseen tasks can be improved via an upstream learning stage using a set of seen tasks.We also observe that the selection of upstream learning tasks can significantly influence few-shot performance on unseen tasks, asking further analysis on task similarity and transferability. 1
Authors: Qinyuan Ye, Bill Yuchen Lin, Xiang Ren
Institution: University of Southern California
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W3152515526