Year: 2018
Type: conference-paper
Abstract: Training accurate classifiers requires many labels, but each label provides only limited information (one bit for binary classification). In this work, we propose BabbleLabble, a framework for training classifiers in which an annotator provides a natural language explanation for each labeling decision. A semantic parser converts these explanations into programmatic labeling functions that generate noisy labels for an arbitrary amount of unlabeled data, which is used to train a classifier. On three relation extraction tasks, we find that users are able to train classifiers with comparable F1 scores from 5-100× faster by providing explanations instead of just labels. Furthermore, given the inherent imperfection of labeling functions, we find that a simple rule-based semantic parser suffices.
DOI: 10.18653/v1/p18-1175
Institution: Stanford University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funders: National Science Foundation, U.S. Department of Energy, Michael J. Fox Foundation for Parkinson's Research, Gordon and Betty Moore Foundation, Accenture, VMware, National Institutes of Health, Defense Advanced Research Projects Agency, Stanford Graduate School of Business, Office of Naval Research, National Institute of Biomedical Imaging and Bioengineering, Air Force Office of Scientific Research
Awards: Synergy: Collaborative: CPS-Security: End-to-End Security for the Internet of Things, G1635403875, G1868717214, G2935333310, G3014111114, G3461160294, G367213561, G489167276, CAREER: Interactive Training of Semantic Parsers via Paraphrasing, G6671297155, G674549868, G7707571486, Mobility Data Integration to Insight
OpenAlex ID: W2798431140
