Year: 2021
Type: book-chapter
Abstract: Machine learning has enormously contributed to optimization techniques motivating new approaches for optimization algorithms.The optimization approaches in deep learning have broad applicability with the resurgence of novelty starting from Stochastic Gradient Descent to convex and non-convex and derivative-free methods.Selecting an optimizer is a vital choice in deep learning.The optimization algorithm chosen to have convexity principles in its core determines the training speed and final performance predicted by the DL model.The complexity further increases with growing deepness due to hyperparameter tuning, and as the data sets become more significant, they require a better optimizer.In this work, we analyze the most popular optimizers algorithms.The augmenting behaviors are tested on MNIST, AutoEncoder data sets, pointing out similarities, differences, and the likelihood of suitability for a given application.The article focuses on the critical role and pinpoints which one would be a better option.
Source: DLINE eBooks
Authors: Ajeet K. Jain, PVRD Prasad Rao, K. Venkatesh Sharma
Institution: Koneru Lakshmaiah Education Foundation
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4285523955