Swish‐Gumbel: A New Activation Function for Image Classification
Year: 2026
Type: article
Abstract: ABSTRACT In this study, we present Swish‐Gumbel, a new family of activation functions generated by combining the Swish and Gumbel functions through both multiplication and linear composition. We evaluate their performance on several image classification benchmarks, including MNIST, CIFAR‐10, Imagenette, and Beans, using different deep learning architectures. The results show that the multiplicative version consistently outperforms well‐known activation functions such as ReLU, Swish, Mish, and Gumbel, while the linear combination variants provide competitive results particularly on simpler datasets, with performance gains becoming more pronounced for the multiplicative form on complex benchmarks. The strength of the proposed approach lies in its ability to maintain more stable gradients in the positive domain, which improves feature extraction and generalization. Overall, our findings suggest that Swish‐Gumbel can serve as an effective alternative to existing activation functions for image classification, with potential to be applied in other machine learning tasks as well.
DOI: 10.1002/cpe.70822
Authors: Elif Dorukbaşı, Burhan Selçuk, İlker Türker
Institutions: Karabük University, Ankara University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $5,140
OpenAlex ID: W7168267871