How well does active learningactuallywork? Time-based evaluation of cost-reduction strategies for language documentation
Year: 2009
Type: conference-paper
Abstract: Machine involvement has the potential to speed up language documentation. We assess this potential with timed annotation experiments that consider annotator expertise, example selection methods, and suggestions from a machine classifier. We find that better example selection and label suggestions improve efficiency, but effectiveness depends strongly on annotator expertise. Our expert performed best with uncertainty selection, but gained little from suggestions. Our non-expert performed best with random selection and suggestions. The results underscore the importance both of measuring annotation cost reductions with respect to time and of the need for cost-sensitive learning methods that adapt to annotators.
Authors: Jason Baldridge, Alexis Palmer
Institutions: The University of Texas at Austin, Saarland University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: bronze
Funder: National Science Foundation
OpenAlex ID: W2133853925
