Truncating Singular Values Below the Noise Level
Year: 2026
Type: preprint
Abstract: Let W = S+R, where S is an N × M matrix of fixed rank and R has independent N(0,1/N) entries, with M/N → c ∈ (0,∞), so that ‖R‖ → 1+√c. Let W_T keep the singular values of W above a cutoff T>1+√c and set the others to zero. Each singular value θ>c^(1/4) of S produces a singular value of W near y(θ) = √((θ²+1)(θ²+c))/θ. When the singular values of S are distinct and no θ>c^(1/4) satisfies y(θ) = T, the almost sure limit of ‖W_T-S‖, which the argument of Gavish and Donoho gives from results of Benaych-Georges and Nadakuditi, is the largest of several numbers, one for each singular value of S. We show that the number belonging to a θ>c^(1/4) with y(θ)>T is less than 1+√c, whatever its size. Hence the limit, and the limits of ‖(W_T-S)v‖ and ‖u^T(W_T-S)‖ for the singular vectors u, v of S, are below 1+√c if and only if every singular value θ ≥ 1+√c of S has y(θ)>T. For every cutoff in (1+√c, y(1+√c)) the limit of ‖W_T-S‖ is below 1+√c for every such S, with a bound that does not depend on S; for every cutoff above y(1+√c) some S of rank one gives a limit above 1+√c. The interval, (2,5/2) for square matrices, contains the cutoff that Gavish and Donoho found optimal for the mean squared error. For Gaussian noise this answers a question of Berlyand, Sandier, Shmalo and Zhang on truncating random weight matrices.
Author: Shmalo, Yitzchak
Institution: Hebrew University of Jerusalem
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7236741379
