Get started
Work

The Base-Point Barrier: Certifying Low-Rank Truncation Costs a Full Evaluation Pass

Year: 2026
Type: preprint
Abstract: Deciding what a low-rank truncation costs in accuracy requires evaluating the truncated network, and this paper proves that nothing measured where the network already sits can take the place of that evaluation. Eckart–Young–Mirsky bounds reconstruction error, not accuracy, and the absence of a cheap bound on the accuracy cost has been read as a gap in the literature. It is a property of the problem, and we price the only route that remains. Call a bound base-point cheap when everything it reads is measured where the network already sits: the downstream map's value and derivatives there, to any finite order, which differential calculus calls the jet of the map at that point, together with the spectrum and the data's projections onto the singular directions. We call the resulting obstruction the base-point barrier, and the paper is organized around the dichotomy it produces. Every selection criterion for low-rank truncation either reads only what is measurable where the network already sits, in which case it cannot certify accuracy at any tolerance, or it evaluates the network where the truncation puts it, in which case certifying one retained subset costs a full pass over the evaluation set. There is no third option. Each branch is a theorem. On the first, every valid base-point-cheap bound carries slack at least μ, a quantity counted on the evaluation set and attained by the construction, not merely bounded by it. We exhibit the separating pair twice, once with a C^∞ unit whose jets agree exactly at every order and once with the network's own GELU at a tolerance driven below any prescribed resolution. Of nine published selection criteria, eight lie inside the class, so no reweighting of the singular directions will yield a guarantee about accuracy. On the second, certifying one retained subset to tolerance τ requires at least (A − 2τ)N forward evaluations and N suffice, so the query complexity is Θ(N) and certified pruning over subsets cannot beat exhaustion, while estimation stays cheap at O(τ^−2 log δ^−1) samples. Both are realized inside a public checkpoint. Two networks there have numerically identical base-point jets and bit-identical accuracy of 0.9106, and under the same truncation they separate into 0.9060 and 0.4908. The separation is the value the theorem predicts to ten digits, and the lower figure is the exact accuracy of a constant predictor. The separating network is adversarial by construction, so the theorem bounds what a guarantee can promise and not what a trained network typically does, and the same checkpoint carries the complementary measurement. What follows for practice is a prescription and not a caution. Estimate the accuracy cost and report an interval, since a probabilistic interval costs a number of sampled examples that does not grow with the evaluation set. Do not seek a certificate, since certified pruning over subsets cannot reduce the order of the search. Use the cheap first-order surrogate to order candidates, where it is well calibrated while the downstream is shallow, and not to guarantee that the ordering is safe. And when certainty is required, budget one evaluation per example. The premise itself becomes a diagnostic costing one forward pass. For every direction of every layer it returns whether the construction exists there and a ceiling on the slack it could force, so that a reader can price the barrier on a model of their own. It is measured across 246 weight matrices in five model-and-task combinations, and it orders the expensive outcome at +0.955 over twenty-one layers of three architectures. Pre-print in progress. The file is restricted while the manuscript is under advisor review. Access is granted by share link.
DOI: 10.5281/zenodo.22557173
Language: English
Open Access status: green
OpenAlex ID: W7210271623
Cites:
Cited by:
Related to: