Work

Convergence analysis and application for high-order neural networks based on gradient descent learning algorithm via smooth regularization

Year: 2025

Type: article

Abstract: The pi-sigma network (PSN), as a high-order network, has demonstrated its capacity for rapid learning and strong nonlinear processing. This paper proposes a type algorithm, for PSNs using a batch gradient method based on [Formula: see text] regularization. Direct application of [Formula: see text] regularization during network training presents two main drawbacks. There are numerical oscillations and theoretical challenges in computing the gradients at the origin. We then introduced smoothing functions by approximating the [Formula: see text] regularization to overcome these obstacles, resulting in a new gradient descent method based on smoothing [Formula: see text] regularization (GDS[Formula: see text]). Numerical results for the 4-dimensional parity problem and the nonlinear Gabor function problem demonstrate that the GDS[Formula: see text] algorithms perform better than the other four regularization methods in terms of generalization and pruning efficiency. Theoretical analysis and experimental verification strictly prove the monotonicity and strong and weak convergence of the network based on the GDS[Formula: see text] algorithms.

Source: Scientific Reports

DOI: 10.1038/s41598-025-29494-1

Authors: Khidir Shaib Mohamed, Alawia Adam, Yousif Shoaib Mohammed, Yan Xiong

Institutions: Qassim University, University of Science and Technology Liaoning

Language: English

Topic: Neural Networks and Applications

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,690

Funder: Qassim University