Year: 2024
Type: preprint
Abstract: The seminal work of Linial, Mansour, and Nisan gave a quasipolynomial-time algorithm for learning constant-depth circuits ($\mathsf{AC}^0$) with respect to the uniform distribution on the hypercube. Extending their algorithm to the setting of malicious noise, where both covariates and labels can be adversarially corrupted, has remained open. Here we achieve such a result, inspired by recent work on learning with distribution shift. Our running time essentially matches their algorithm, which is known to be optimal assuming various cryptographic primitives. Our proof uses a simple outlier-removal method combined with Braverman's theorem for fooling constant-depth circuits. We attain the best possible dependence on the noise rate and succeed in the harshest possible noise model (i.e., contamination or so-called "nasty noise").
Source: arXiv (Cornell University)
Authors: Klivans, Adam R., Stavropoulos, Konstantinos, Vasilyan, Arsen
Institution: University of California, Berkeley
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: AF: Small: Sparsity in Local Computation, Collaborative Research: AF: Medium: Foundations of Structured Optimization, AF: Small: Efficient Algorithms for Nonconvex Regression, AF: Large: Collaborative Research: Algebraic Proof Systems, Convexity, and Algorithms, AF: SMALL: Extending the Reach of Distribution Testing via Structure, Foundations of Data Science Institute
OpenAlex ID: W4404354811