The Intelligent Management of Crowd-Powered Machine Learning
Year: 2017
Type: dissertation
Abstract: Artificial intelligence and machine learning power many technologies today, from spam filters to self-driving cars to medical decision assistants.While this revolution has hugely benefited from algorithmic developments, it also could not have occurred without data, which nowadays is frequently procured at massive scale from crowds.Because data is so crucial, a key next step towards truly autonomous agents is the design of better methods for intelligently managing now-ubiquitous crowd-powered data-gathering processes.This dissertation takes this key next step by developing algorithms for the online and dynamic control of these processes.We consider how to gather data for its two primary purposes: training and evaluation.In the first part of the dissertation, we develop algorithms for obtaining data for testing.The most important requirement of testing data is that it must be extremely clean.Thus to deal with noisy human annotations, machine learning practitioners typically rely on careful workflow design and advanced statistical techniques for label aggregation.A common process involves designing and testing multiple crowdsourcing workflows for their tasks, identifying the single best-performing workflow, and then aggregating worker responses from redundant runs of that single workflow.We improve upon this process by building two control models: My advisors, Dan Weld and Mausam, have selflessly dedicated years of their lives to making me a better scientist.They have tried to bestow upon me all of their vast wisdom, and provided me with all the opportunities a PhD student could ever wish for, and they have done all this with infinite patience and understanding.I could not have asked for better advisors.
Author: Christopher H. Lin
Institution: University of Washington
Language: English
Subfield: Computer Science Applications
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2784064574
