Double Descent, Ensemble Emergence, and Large Model Averaging in High-Dimensional Multimodel Prediction
Year: 2026
Type: preprint
Abstract: This paper investigates the predictive performance of high-dimensional multimodel prediction, where the number of regressors is comparable to the sample size. Leveraging tools from random matrix theory, we derive the exact limiting out-of-sample risk under a nested model setting and comprehensively characterize the risk landscape. This limiting risk helps to reveal two phenomena: simple weighting inherits the double descent trajectory and its associated variance explosion near the interpolation boundary; strategic weighting triggers an ensemble emergence that suppresses the localized risk surge and yields a globally flat risk surface. Building on this limiting risk, we also propose the Large Model Averaging (LaMA) method, in which we consider the discrepancy between in-sample and out-of-sample risks in the high-dimensional regime. Numerical studies and real data applications confirm that LaMA achieves superior predictive accuracy in high-dimensional environments.
Source: arXiv (Cornell University)
Authors: Chen, Ke, Jiang, Dandan, Zhang, Xinyu
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
Awards: G4117264098, G6498450540, G8062590584
OpenAlex ID: W7161139936
Cites:
Cited by:
Related to: