Work

Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study

Year: 2018
Type: conference-paper
Abstract: Several recent papers investigate Active Learning (AL) for mitigating the datadependence of deep learning for natural language processing.However, the applicability of AL to real-world problems remains an open question.While in supervised learning, practitioners can try many different methods, evaluating each against a validation set before selecting a model, AL affords no such luxury.Over the course of one AL run, an agent annotates its dataset exhausting its labeling budget.Thus, given a new task, an active learner has no opportunity to compare models and acquisition functions.This paper provides a largescale empirical study of deep active learning, addressing multiple tasks and, for each, multiple datasets, multiple models, and a full suite of acquisition functions.We find that across all settings, Bayesian active learning by disagreement, using uncertainty estimates provided either by Dropout or Bayes-by-Backprop significantly improves over i.i.d.baselines and usually outperforms classic uncertainty sampling.
Language: English
Open Access status: hybrid
OpenAlex ID: W2887340847
FWCI: 7.725
Cites: 33
Cited by: 265
Related to: 9