Work

On sensitivity of meta-learning to support data

Year: 2021

Type: preprint

Abstract: Meta-learning algorithms are widely used for few-shot learning. For example, image recognition systems that readily adapt to unseen classes after seeing only a few labeled examples. Despite their success, we show that modern meta-learning algorithms are extremely sensitive to the data used for adaptation, i.e. support data. In particular, we demonstrate the existence of (unaltered, in-distribution, natural) images that, when used for adaptation, yield accuracy as low as 4\% or as high as 95\% on standard few-shot image classification benchmarks. We explain our empirical findings in terms of class margins, which in turn suggests that robust and safe meta-learning requires larger margins than supervised learning.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2110.13953

Authors: Agarwal, Mayank, Yurochkin, Mikhail, Sun, Yuekai

Institutions: IBM (United States), Massachusetts Institute of Technology, University of Michigan

Language: English

Topic: Domain Adaptation and Few-Shot Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: National Science Foundation

Awards: ATD: Algorithmic Threat Detection and Mitigation with Robust Machine Learning, Integrative Analysis on Heterogeneous Datasets with High-Dimensional and Non-Standard Models, A Transfer Learning Approach to Algorithmic Fairness

OpenAlex ID: W3210009341

Cites: 42

Cited by: 7

Related to: 20