Get started
Keyword

weight initialization

Description: The method of assigning starting values to a neural network's weights before training, influencing learning dynamics and model performance.
Wikidata ID: wiki/Q130601826
OpenAlex ID: keywords/weight-initialization
Greedy Layer-Wise Training of Deep Networks
2007 · Yoshua Bengio, Pascal Lamblin, et al. · The MIT Press eBooks · 4,839
Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights
1990 · D. Nguyen, B. Widrow · 2022 International Joint Conference on Neural Networks (IJCNN) · 1,375
Exploring Strategies for Training Deep Neural Networks
2009 · Hugo Larochelle, Yoshua Bengio, et al. · 1,109
Predicting Parameters in Deep Learning
2013 · Denil, Misha, Shakibi, Babak, et al. · arXiv (Cornell University) · 852
PDF
Back propagation neural network with adaptive differential evolution algorithm for time series forecasting
2014 · Lin Wang, Yi Zeng, et al. · Expert Systems with Applications · 632
TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation
2018 · Iglovikov, Vladimir, Shvets, Alexey · arXiv (Cornell University) · 551
PDF
Learning Feature Pyramids for Human Pose Estimation
2017 · Wei Yang, Shuang Li, et al. · IEEE International Conference on Computer Vision (ICCV) · 550
PDF
A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
2015 · Le, Quoc V., Jaitly, Navdeep, et al. · arXiv (Cornell University) · 522
PDF
RNN-LSTM: From applications to modeling techniques and beyond—Systematic review
2024 · Safwan Mahmood Al-Selwi, Mohd Fadzil Hassan, et al. · Journal of King Saud University - Computer and Information Sciences · 509
An Efficient Learning Procedure for Deep Boltzmann Machines
2012 · Ruslan Salakhutdinov, Geoffrey Hinton · Neural Computation · 484
Understanding Batch Normalization
2018 · Nils Bjorck, Carla P. Gomes, et al. · neural information processing systems · 471
Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks
2018 · Daniel Povey, Gaofeng Cheng, et al. · Interspeech (USB) · 459
Dying ReLU and Initialization: Theory and Numerical Examples
2020 · Lu Lu, Yeonjong Shin, et al. · Communications in Computational Physics · 456
PDF
Gradient-based Hyperparameter Optimization through Reversible Learning
2015 · Maclaurin, Dougal, Duvenaud, David, et al. · arXiv (Cornell University) · 454
PDF
Train longer, generalize better: closing the generalization gap in large batch training of neural networks
2017 · Hoffer, Elad, Hubara, Itay, et al. · arXiv (Cornell University) · 432
PDF
Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
2020 · Dodge, Jesse, Ilharco, Gabriel, et al. · arXiv (Cornell University) · 397
PDF
All you need is a good init
2015 · Mishkin, Dmytro, Matas, Jiri · arXiv (Cornell University) · 375
PDF
Wind power prediction using deep neural network based meta regression and transfer learning
2017 · Aqsa Saeed Qureshi, Asifullah Khan, et al. · Applied Soft Computing · 368
Unsupervised Pre-training of a Deep LSTM-based Stacked Autoencoder for Multivariate Time Series Forecasting Problems
2019 · Alaa Sagheer, Mostafa Kotb · Scientific Reports · 351
PDF
Understanding Batch Normalization
2018 · Bjorck, Johan, Gomes, Carla, et al. · arXiv (Cornell University) · 328
PDF
Discriminability-Based Transfer between Neural Networks
1992 · Lorien Pratt · 317
A review on weight initialization strategies for neural networks
2021 · Meenal V. Narkhede, Prashant P. Bartakke, et al. · Artificial Intelligence Review · 312
Convergence Analysis of Two-layer Neural Networks with ReLU Activation
2017 · Li, Yuanzhi, Yuan, Yang · arXiv (Cornell University) · 304
PDF
A Comparison of Some Error Estimates for Neural Network Models
1996 · Robert Tibshirani · Neural Computation · 285
Breast cancer histology images classification: Training from scratch or transfer learning?
2018 · Shallu, Rajesh Mehra · ICT Express · 277
PDF
Works count: 2,283
Citation count: 40,210
Year
Topic
Type