Year: 2024
Type: conference-paper
Abstract: A set of high dimensional points X$= \{x_{1},x_{2},\ldots,x_{n}\}\subseteq \mathbb{R}^{d}$in isotropic position is said to be$\delta$-anti concentrated if for every direction$v$, the fraction of points in$X$satisfying$\left|\left\langle x_i, v\right\rangle\right| \leqslant \delta$is at most$O(\delta)$. Motivated by applications to list-decodable learning and clustering, three recent works [7], [44], [71] considered the problem of constructing efficient certificates of anti-concentration in the average case, when the set of points X corresponds to samples from a Gaussian distribution. Their certificates played a crucial role in several subsequent works in algorithmic robust statistics on list-decodable learning and settling the robust learnability of arbitrary Gaussian mixtures. Unlike related efficient certificates of concentration properties that are known for wide class of distri-butions [52], the aforementioned approach has been limited only to rotationally invariant distributions (and their affine transformations) with the only prominent example being Gaussian distributions. This work presents a new (and arguably the most natural) formulation for anti- concentration. Using this formulation, we give quasi-polynomial time verifiable sum-of-squares certificates of anti-concentration that hold for a wide class of non-Gaussian distributions including anti-concentrated bounded product distributions and uniform distributions over$L_{p}$balls (and their affine transformations). Consequently, our method upgrades and extends results in algorithmic robust statistics e.g., list-decodable learning and clustering, to such distributions. As in the case of previous works, our certificates are also obtained via relaxations in the sum-of-squares hierarchy. However, the nature of our argument differs significantly from prior works that formulate anti-concentration as the non-negativity of an explicit polynomial. Our argument constructs a canonical integer program for anti-concentration and analysis a SoS relaxation of it, independent of the intended application. The explicit polynomials appearing in prior works can be seen as specific dual certificates to this program. From a technical standpoint, unlike existing works that explicitly construct sum-of-squares certificates, our argument relies on duality and analyzes a pseudo-expectation on large subsets of the input points that take a small value in some direction. Our analysis uses the method of polynomial reweightings to reduce the problem to analyzing only analytically dense or sparse directions.
Language: English
Subfield: Numerical Analysis
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
Awards: AF: Small: Understanding Expansion Phenomena: Graphical, Hypergraphical, Geometric, and Quantum, AF: Small: Parallels in Approximability of Discrete and Continuous Optimization Problems, CAREER: Beyond Worst-Case Analysis: New Approaches in Approximation Algorithms and Machine Learning, AF: SMALL: Further Investigation of the Sum of Squares Hierarchy, Foundations of Data Science Institute, Small: New Directions in Community Detection, Institute for Data, Econometrics, Algorithms and Learning (IDEAL)
OpenAlex ID: W4404849342