Preventing Exploding Gradients in Deep Neural Networks using the AMDCF+CSD Smart Optimizer
Year: 2026
Type: preprint
Abstract: Training deep neural networks on steep loss landscapes (especially in low-precision or quantized environments) often leads to catastrophic gradient explosion, halting the learning process. In this paper, we introduce the AMDCF+CSD (Adaptive Memory-Driven Coupling Framework with Critical Slowing Down) as a novel smart optimizer. By integrating a non-Markovian momentum (memory) with a dynamic safety valve governed by early-warning CSD risk detection, the optimizer autonomously adjusts the learning rate. Simulation on a highly non-linear quartic loss landscape demonstrates that while standard SGD diverges to infinity, the AMDCF optimizer dynamically clamps the gradients and stabilizes the training trajectory. This work provides a robust mathematical solution for AI training under extreme computational constraints and quantized models, with direct applications in high-energy physics data processing.
Author: SAIM, MOHAMMED TAHAR
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225042716