Weight Initialization Methods in Convolutional Neural Networks and Their Impact on Training Efficiency
Year: 2024
Type: article
Abstract: This study analyzes the application effects of different weight initialization methods in convolutional neural networks, focusing on the impact of these methods on training efficiency. Through analysis and derivation from existing studies, we expect significant differences in training speed, stability, and final performance among different initialization methods. Specifically, this study compared random initialization, Xavier initialization, He initialization, LeCun initialization, and Orthogonal initialization. The results show that choosing an appropriate weight initialization method can significantly improve the training efficiency and performance of the model. The main contribution of this article is to systematically compare several common weight initialization methods and propose performance expectations based on existing research to provide a reference for subsequent research. Future research can verify these expected results through actual experiments and explore novel initialization strategies to further improve the training efficiency and performance of the model.
Author: Zhang, Zhenxing
Institution: Belarusian State University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W6930489451
FWCI:
Cites:
Cited by:
Related to: