Reducing Computational Demands in Capsule Net Through Knowledge Distillation and Transfer Learning
Year: 2025
Type: article
Abstract: Capsule networks have emerged as a robust alternative to traditional convolutional neural networks, providing superior performance in recognizing spatial hierarchies and capturing intricate relationships in image data. However, their computational intensity and memory demands present significant challenges, particularly for resource- constrained environments. Addressing this limitation, the proposed study explores the integration of knowledge distillation and transfer learning techniques to enhance the computational efficiency of Capsule Networks without compromising their accuracy. Knowledge distillation compresses the model by transferring learned knowledge from a high-capacity teacher network to a lightweight student network, effectively reducing computational overhead. Transfer learning further minimizes resource demands by leveraging pre-trained models, thus expediting the training process and optimizing performance. Experiments were conducted on the MNIST and CIFAR-10 datasets, with the optimized Capsule Network achieving classification accuracies of 99.1% and 93.7%, respectively, while reducing computational requirements by 45%. The proposed approach demonstrated a significant improvement in training time and memory efficiency, achieving a 40% reduction in model parameters compared to baseline Capsule Network implementations. These results underline the potential of combining knowledge distillation and transfer learning to make advanced architectures like Capsule Networks accessible for real-time and edge applications. Future directions include extending this framework to more complex datasets and applications such as object detection and medical imaging.
Source: ICTACT Journal on Soft Computing
Authors: Vince Paul, Anbu Megelin Star A, Anto Spiritus Kingsly A, Jereesha Mary S.J.
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4409113111