Higher-Order Geometric Updates for Levenberg-Marquardt Method via Riemann Normal Coordinates
Year: 2026
Type: preprint
Abstract: Nonlinear least-squares objectives form the foundation of scientific machine-learning tasks, yet even curvature-aware optimizers remain geometrically inconsistent at finite step sizes: Levenberg-Marquardt (LM) derives its direction from local Riemannian metrics but realizes it as a straight parameter update. Here we introduce RNC-LM, which carries the LM direction along a locally constructed curved trajectory in Riemann normal coordinates. A recursive reformulation of the geodesic equation generates arbitrary finite-order corrections while reusing the same damped Gauss-Newton matrix factorization, and curve length is controlled separately from damping. On a reaction-diffusion physics-informed neural network benchmark, RNC-LM reduces relative $L^2$ errors below $8\times10^{-3}$, whereas L-BFGS, LM and LM with geodesic acceleration remain near one. On a large-scale machine-learning potential fitting task with 985,160 configurations, fourth-order RNC-LM reaches a fixed training-error target with a $34\times$ wall-clock speedup over LM. These results establish finite-step geometric realization as a distinct optimizer design principle.
Source: arXiv (Cornell University)
Authors: Liu, Jianing, Zhang, Dong H.
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7167852237