Benchmarking the H. Dilpriya’s Momentum (HDM) Optimizer: A Seeded, Step-by-Step Protocol for Synthetic Quadratic, MNIST and CIFAR-10 Experiments Against Six Optimizers v1
Year: 2026
Type: preprint
Abstract: H. Dilpriya’s Momentum (HDM) is a first-order optimizer for PyTorch that adds a gradient-alignment correction to momentum gradient descent. At every step it measures the cosine similarity between the current and the previous gradient, smooths this signal with an exponential moving average, uses it to weight the momentum accumulation, and scales a correction term whose strength follows a linear warmup and a cosine annealing schedule. This protocol gives a complete, step-by-step procedure for installing the HDM experiment code and repeating its five benchmark experiments against six other optimizers (SGD, SGD with momentum, Adam, RMSprop, DEAM and YellowFin): convergence on synthetic strongly convex quadratics with condition numbers 10, 100 and 1000; image classification on MNIST with a multilayer perceptron; image classification on CIFAR-10 with ResNet-18; an ablation of the HDM components; and an analysis of gradient alignment during training. Every run uses a fixed random seed, so that all optimizers start from the same weights and see the same sequence of mini-batches. The protocol can be run on a local computer or entirely on Kaggle, and it closes with reference results from a complete seeded run, against which a reproduction can be checked.
Author: Janaka Ishan Senarathna
Institution: NSBM Green University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225417694