Efficiency and Stability of a New Hybrid Unconstrained Optimization Algorithm with Quasi-Newton Updates and Higher-Order Methods
Year: 2026
Type: article
Abstract: We propose the higher-order quasi-Newton (HOQN) method, a hybrid algorithm for unconstrained optimization that combines Newtonian predictors with higher-order correctors derived from vector extensions of the Traub, Chun, and Ostrowski methods, along with quasi-Newton updates of the inverse Hessian using Broyden–Fletcher–Goldfarb–Shanno (BFGS) or Davidon–Fletcher–Powell (DFP) formulas. We demonstrate that the resulting scheme achieves cubic local convergence order, representing a substantial improvement over the superlinear convergence typical of classical quasi-Newton methods, while maintaining a cost of On2 per iteration. We also analyze variants that incorporate two successive quasi-Newton updates, and show that they retain the same cubic order. Numerical experiments with the benchmark functions of Himmelblau and Freudenstein–Roth confirm the theoretical convergence order and show that the hybrid variants consistently require fewer iterations than BFGS, DFP, and Symmetric Rank-One (SR1). In the case of the Booth function, given its strictly convex quadratic structure, the proposed hybrid methods reach the global minimum in just two iterations and exhibit numerical accuracy superior to that of classical quasi-Newton methods. In addition, limited-memory variants (L-HOQN) are introduced; these are evaluated during the training of a convolutional neural network on the MNIST dataset, where they achieve test accuracies exceeding 99% and outperform L-BFGS and standard stochastic gradient descent (SGD) at all tested learning rates.
Source: Mathematics
DOI: 10.3390/math14101746
Language: English
Subfield: Numerical Analysis
Field: Mathematics
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,137
Award: G2893919860
OpenAlex ID: W7161680674
FWCI:
Cites:
Cited by:
Related to: