Year: 2024
Type: article
Abstract: . We take a random matrix theory approach to random sketching and show an asymptotic first-order equivalence of the regularized sketched pseudoinverse of a positive semidefinite matrix to a certain evaluation of the resolvent of the same matrix. We focus on real-valued regularization and extend previous results on an asymptotic equivalence of random matrices to the real setting, providing a precise characterization of the equivalence even under negative regularization, including a precise characterization of the smallest nonzero eigenvalue of the sketched matrix, which may be of independent interest. We then further characterize the second-order equivalence of the sketched pseudoinverse. We also apply our results to the analysis of the sketch-and-project method and to sketched ridge regression. Last, we prove that these results generalize to asymptotically free sketching matrices, obtaining the resulting equivalence for orthogonal sketching matrices and comparing our results to several common sketches used in practice.
DOI: 10.1137/22m1530264
Authors: Daniel LeJeune, Pratik Patil, Hamid Javadi, Richard G. Baraniuk, Ryan J. Tibshirani
Language: English
Topic: Matrix Theory and Algorithms
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
Awards: G1523888516, G1717380892, G2167961191, G2308282352, BIGDATA: F: Collaborative Research: Theory and Practice of Randomized Algorithms for Ultra-Large-Scale Signal Processing, G3449126043, G3587679958, Collaborative Research: Computational Photo-Scatterography: Unraveling Scattered Photons for Bio-Imaging, G5121974832, G6098521345, G6706517093, G7840735550, G8006577219, G8252795151, G8280646250, CIF: Small: A Probabilistic Theory of Deep Learning via Spline Operators, G8876996369
OpenAlex ID: W4393319987