Year: 2011
Type: article
Abstract: We study an instance of high-dimensional inference in which the goal is to estimate a matrix Θ∗∈ℝm1×m2 on the basis of N noisy observations. The unknown matrix Θ∗ is assumed to be either exactly low rank, or “near” low-rank, meaning that it can be well-approximated by a matrix with low rank. We consider a standard M-estimator based on regularization by the nuclear or trace norm over matrices, and analyze its performance under high-dimensional scaling. We define the notion of restricted strong convexity (RSC) for the loss function, and use it to derive nonasymptotic bounds on the Frobenius norm error that hold for a general class of noisy observation models, and apply to both exactly low-rank and approximately low rank matrices. We then illustrate consequences of this general theory for a number of specific matrix models, including low-rank multivariate or multi-task regression, system identification in vector autoregressive processes and recovery of low-rank matrices from random projections. These results involve nonasymptotic random matrix theory to establish that the RSC condition holds, and to determine an appropriate choice of regularization parameter. Simulation results show excellent agreement with the high-dimensional scaling of the error predicted by our theory.
Source: The Annals of Statistics
DOI: 10.1214/10-aos850
Authors: Sahand Negahban, Martin J. Wainwright
Institution: University of California, Berkeley
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Keywords: low-rank matrix estimation, high-dimensional inference, nuclear norm regularization, trace norm regularization, M-estimation, restricted strong convexity, non-asymptotic error bounds, multivariate regression, multi-task regression, vector autoregression, system identification, low-rank matrix recovery, random projections, regularization parameter selection, random matrix theory
Open Access status: bronze
Awards: Inference in high-dimension: statistics, computation and information theory, G5693069798, CAREER: Novel Message-Passing Algorithms for Distributed Computation in Graphical Models: Theory and Applications in Signal Processing, CDI-Type II: Real-time Classification of Massive Time-series Data Streams
OpenAlex ID: W2586353914
