Year: 2025
Type: article
Abstract: We consider the classification problem of a high-dimensional mixture of two Gaussians with general covariance matrices. Using the replica method from statistical physics, we investigate the asymptotic behavior of a broad class of regularized convex classifiers in the limit where both the sample size n and the dimension p approach infinity while their ratio α = n / p remains fixed. This approach contrasts with traditional large-sample theory in statistics, which examines asymptotic behavior as n → ∞ with p fixed. A key advantage of this asymptotic regime is that it provides precise quantitative guidelines for designing machine learning systems when both p and n are large but finite. Our focus is on the generalization error and variable selection properties of the estimators. Specifically, based on the distributional limit of the classifier, we construct a de-biased estimator to perform variable selection through an appropriate hypothesis testing procedure. Using L 1-regularized logistic regression as an example, we conduct extensive computational experiments to verify that our analytical findings align with numerical simulations in finite-sized systems. Additionally, we explore the influence of the covariance structure on the performance of the de-biased estimator.
Authors: Hanwen Huang, Peng Zeng
Institutions: Augusta University, Auburn University
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4409546484