Year: 2023
Type: article
Abstract: As Adam optimizer’s learning rate decay hyperparameter has recently been deprecated, this journal article focuses not only on providing an alternate optimizer but also on comparing the performance of the said optimizer, AdamW, with the Adam optimizer using a face mask detection model. This study experiments with different weight decay values and finds that a weight decay of 0.00009 with the AdamW optimizer consistently achieves a 98% accuracy rate. Aside from that, this study also discusses the differences between Adam with L2-regularization and AdamW on how the weight decay is decoupled from the Adam optimizer’s gradient-based update that impacts the performance of AdamW. Overall, the study provides insights to those new to AdamW and looking for a starting point in optimizing deep learning models.
Institution: Asia Pacific University of Technology & Innovation
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7124281185