Work

Machine Learning and Deep Learning Optimization Algorithms for Unconstrained Convex Optimization Problem

Year: 2024

Type: article

Abstract: This paper conducts a thorough comparative analysis of optimization algorithms for an unconstrained convex optimization problem. It contrasts traditional methods like Gradient Descent (GD) and Nesterov Accelerated Gradient (NAG) with modern techniques such as Adaptive Moment Estimation (Adam), Long Short-Term Memory (LSTM) and Multilayer Perceptron (MLP). Through empirical experiments, convergence speed, solution accuracy and robustness, is evaluated providing insights to aid algorithm selection. The convergence dynamics of convex optimization, is explored analyzing classical algorithms and contemporary neural network (NN) methodologies. The study concludes with a comparative assessment of these algorithms performance metrics and their respective strengths and weaknesses.

Source: IEEE Access

DOI: 10.1109/access.2024.3522361

Authors: Kainat Naeem, Amal Bukhari, Ali Daud, Tariq Alsahfi, Bader Alshemaimri, Mousa Alhajlah

Institutions: International Islamic University, Islamabad, University of Jeddah, Rabdan Academy, King Saud University

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $1,995

Funder: King Saud University