Year: 2024
Type: article
Abstract: Deep Neural Networks are widely used to solve a variety of practical problems: language processing, image and video recognition, and others. Such architectures include millions of elementary neurons, used activation functions. Their properties significantly affect the entire network results. The paper examines the problem of increasing Deep Neural Networks learning rate and accuracy through the use of variable shape activation functions, which parameters can be adjusted during network training. As a solution, a modified Satlin activation function is proposed. It has compressive properties, but does not contain asymptotic branches, protecting Deep Neural Network from a vanishing gradient.
Source: Системи обробки інформації
Institution: Kharkiv National University of Radio Electronics
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4402902214