The Impact of Neural Network Architecture on Generalization: A Review of Depth, Width, and Regularization Techniques
Year: 2025
Type: article
Abstract: This paper investigates the impact of increasing the depth and width of convolutional neural networks (CNNs) on their generalization performance across image classification tasks. Experiments were conducted using PyTorch on two datasets of varying complexity: MNIST (simple) and CIFAR-10 (complex). A variety of CNN architectures were trained with different depths and widths, and regularization techniques including dropout and L2 weight decay were applied to analyze their effects on overfitting. Results indicate that shallow networks are sufficient for achieving high accuracy on MNIST, while deeper or wider networks yield significant performance gains on CIFAR-10. However, high-capacity models are more prone to overfitting without appropriate regularization. Techniques such as dropout and L2 regularization were found to consistently improve generalization, particularly in deeper architectures. These findings underscore the importance of balancing model complexity and regularization, especially when dealing with datasets of differing size and variability.
Author: Zohaib Muaz
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6930602827
FWCI:
Cites:
Cited by:
Related to: