Too Simple, Too Complex, Just Right: A Narrative Review of Overfitting and Underfitting's Literature
Year: 2026
Type: review
Abstract: Overfitting is machine learning's oldest failure and its newest paradox: the model that memorizes the training set fails the world, yet modern networks memorize perfectly and generalize anyway. This article presents a narrative review of the literature of that paradox. The synthesis is organized around three themes: the classical theory, in which Vapnik-Chervonenkis's uniform convergence, Vapnik's statistical learning theory, Geman and colleagues' bias-variance dilemma, and Hastie and colleagues' elements fixed the canon --- capacity, the U-shaped risk curve, and structural risk minimization; the selection and regularization instruments, in which Akaike's information criterion, Schwarz's Bayesian criterion, Breiman's random forests, and dropout's noise injection fixed the practice --- penalizing complexity, averaging models, and randomizing neurons; and the modern revision, in which Zhang and colleagues' rethinking generalization, Belkin's double descent, Wolpert's no-free-lunch, and Breiman's two cultures reopened the canon --- implicit regularization, the second descent, and the modeling cultures' divide. It is concluded that the literature describes a theory's arc --- canonized, practiced, and revised --- whose unity is the generalization question: how much should a model remember, and the modern answer's honesty: the classical U-curve was a chapter, not the book.
Authors: Revista, Zen, IA, 10
Institution: Faculdades Guarulhos
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204754271