A Study on Different Functionalities and Performances among Different Activation Functions across Different ANNs for Image Classification
Year: 2021
Type: conference-paper
Abstract: Activation functions are very essential in artificial neural networks (ANNs), since they are non-linear functions and they have been proved necessary to implement deep learning. Recently, ReLU is one of the most well-known activation functions; however, several competitors – e.g. LReLU and SWISH – have nowadays been proposed or ‘discovered’. In this paper, the authors perform a detailed comparison of five activation functions over two image classification datasets. We found the overall performances of accuracy rates ranked from the best GELU, RELU, SWISH, SELU, down to Sigmoid. Such the observation would result in the improvement of future image classification via designing new state-of-the-art activation functions.
Authors: Xia Zhang, Di Chang, Weimin Qi, Zhiming Zhan
Institutions: Jianghan University, University of Georgia
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W3119620439
