Quantitative Effectiveness Assessment and Role Categorization of Individual Units in Convolutional Neural Networks.
Year: 2021
Type: preprint
Abstract: Identifying the roles of individual units is critical for understanding the mechanism of convolutional neural networks (CNNs). However, it is challenging to give the fully automatic and quantitative measures for effectiveness assessment of individual units in CNN. To this end, we propose a novel method for quantitatively clarifying the status and usefulness of single unit of CNN in image classification tasks. The technical substance of our method is ranking the importance of unit for each class in classification based on calculation of specifically defined entropy using algebraic topological tools. It could be implemented totally by machine without any human intervention. Some interesting phenomena including certain kind of phase transition are observed via the evolution of accuracy and loss of network in the successive ablation process of units. All of the network units are divided into four categories according to their performance on training and testing data. The role categorization is excellent startpoint for network construction and simplification. The diverse utility and contribution to the network generalization of units in classification tasks are thoroughly illustrated by extensive experiments on network (VGG) and dataset (ImageNet) with considerable scale. It is easy for our method to have extensional applications on other network models and tasks without essential difficulties.
Source: arXiv (Cornell University)
Authors: Yang Zhao, Hao Zhang
Institution: Tsinghua University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3137179256