Year: 2026
Type: article
Abstract: We view penalized risks through the lens of the calculus of variations. We consider risks comprised of a fitness-term (e.g. MSE) and a gradient-based penalty and search for the minima of such risks. After establishing the Euler-Lagrange field equations as a systematic approach to finding minimizers of risks involving only first derivatives, we proceed to exemplify this approach to the MSE penalized by the integral over the squared $$\ell ^2$$ ℓ 2 -norm of the gradient of the regression function. The minimizer of this risk is given as the solution to a second order inhomogeneous partial differential equation, where the inhomogeneity is given as the conditional expectation of the target variable conditioned on the features. We discuss properties of the field equations and practical implications thereof, which also apply to the classical ridge penalty for linear models, and embed our findings into the existing literature. In particular, we find that we can recover the Rudin-Osher-Fatemi model for image-denoising, if we consider the features as deterministic and evenly distributed. We proceed to derive the minima of the population-MAE and the Cramer Distance using the calculus of variations. Last, we exemplify the usage of the field equations for post-training regularization in a simulation study.
Source: Statistical Papers
Author: Sven Pappert
Institution: TU Dortmund University
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $2,890
Funder: Deutsche Forschungsgemeinschaft
Awards: G1393714528, G2890341565
OpenAlex ID: W4414901670
FWCI:
Cites:
Cited by:
Related to: