Composing Lightweight Methods on Deep Neural Network: A Survey
Year: 2026
Type: preprint
Abstract: Deep learning has achieved remarkable progress across many domains, largely powered by large-scale neural networks. Their substantial computational demands, however, hinder deployment on resource-constrained edge devices. To address this, lightweight techniques, such as efficient architecture design, pruning, quantization, decomposition, and knowledge distillation, have been explored to reduce the required resources while preserving accuracy. While prior surveys typically investigate these methods one by one, limited attention has been given to how their composition, ordering, and optimization dependencies affect efficiency and accuracy in real-world settings. In this survey, we introduce a taxonomy of lightweight method compositions, organized by which lightweight methods are selected and how those methods are composed and interact, including sequential application, joint optimization, and hybrid approaches. We qualitatively and quantitatively analyze these categories, with particular attention to within-study comparisons, to assess the effectiveness of different compositions in improving efficiency and accuracy, providing guidance for future research in this area and practical deployment of deep neural networks.
Source: Research Square
Authors: Kilho Son, Yunho Jeon
Institutions: Microsoft (United States), Hanbat National University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225815077