Year: 2016
Type: article
Abstract: We study high-dimensional distribution learning in an agnostic setting where an adversary is allowed to arbitrarily corrupt an $\varepsilon$-fraction of the samples. Such questions have a rich history spanning statistics, machine learning, and theoretical computer science. Even in the most basic settings, the only known approaches are either computationally inefficient or lose dimension-dependent factors in their error guarantees. This raises the following question: Is high-dimensional agnostic distribution learning even possible, algorithmically? In this work, we obtain the first computationally efficient algorithms with dimension-independent error guarantees for agnostically learning several fundamental classes of high-dimensional distributions: (1) a single Gaussian, (2) a product distribution on the hypercube, (3) mixtures of two product distributions (under a natural balancedness condition), and (4) mixtures of spherical Gaussians. Our algorithms achieve error that is independent of the dimension, and in many cases scales nearly linearly with the fraction of adversarially corrupted samples. Moreover, we develop a general recipe for detecting and correcting corruptions in high-dimensions that may be applicable to many other problems.
Source: SIAM Journal on Computing
DOI: 10.1137/17m1126680
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funders: National Science Foundation, NEC Foundation of America, University of Southern California, Microsoft Research, Google, Massachusetts Institute of Technology, Johns Hopkins University, University of Edinburgh, Université Pierre et Marie Curie, Simons Institute for the Theory of Computing, University of California Berkeley, Engineering and Physical Sciences Research Council, Office of Naval Research
Awards: CAREER: Algorithmic Aspects of Machine Learning, G6257167855, CAREER: Efficient Algorithms for Learning and Testing Structured Probabilistic Models, G6671297155, CAREER: Towards a Constructive Theory of Networked Interactions, Sublinear Algorithms for Approximating Probability Distributions, G8067955107
OpenAlex ID: W2336954923
