Year: 2013
Type: article
Abstract: Summary High dimensional sparse modelling via regularization provides a powerful tool for analysing large-scale data sets and obtaining meaningful interpretable models. The use of non-convex penalty functions shows advantage in selecting important features in high dimensions, but the global optimality of such methods still demands more understanding. We consider sparse regression with a hard thresholding penalty, which we show to give rise to thresholded regression. This approach is motivated by its close connection with L0-regularization, which can be unrealistic to implement in practice but of appealing sampling properties, and its computational advantage. Under some mild regularity conditions allowing possibly exponentially growing dimensionality, we establish the oracle inequalities of the resulting regularized estimator, as the global minimizer, under various prediction and variable selection losses, as well as the oracle risk inequalities of the hard thresholded estimator followed by further L2-regularization. The risk properties exhibit interesting shrinkage effects under both estimation and prediction losses. We identify the optimal choice of the ridge parameter, which is shown to have simultaneous advantages to both the L2-loss and the prediction loss. These new results and phenomena are evidenced by simulation and real data examples.
DOI: 10.1111/rssb.12037
Authors: Zemin Zheng, Yingying Fan, Jinchi Lv
Institution: University of Southern California
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: bronze
Awards: CAREER: High-Dimensional Variable Selection in Nonlinear Models and Classification with Correlated Data, Regularization Methods in High Dimensions with Applications to Functional Data Analysis, Mixed Effects Models and Classification, G6671297155, Variable Selection in High Dimensional Feature Space with Applications to Covariance Matrix Estimation and Functional Data Analysis, CAREER: High Dimensional Variable Selection and Risk Properties
OpenAlex ID: W2040708309