PCA-driven Hybrid network design for enabling Intelligence at the Edge.
Year: 2019
Type: preprint
Abstract: The recent advent of IOT has increased the demand for enabling AI-based edgecomputing in several applications including healthcare monitoring systems,
autonomous vehicles etc. This has necessitated the search for efficient
implementations of neural networks in terms of both computation and storage.
Although extreme quantization has proven to be a powerful tool to achieve
significant compression over full-precision networks, it can result in
significant degradation in performance for complex image classification tasks.
In this work, we propose a Principal Component Analysis (PCA) driven
methodology to design mixed-precision, hybrid networks. Unlike standard
practices of using PCA for dimensionality reduction, we leverage PCA to
identify significant layers in a binary network which contribute relevant
transformations on the input data by increasing the number of significant
dimensions. Subsequently, we propose Hybrid-Net, a network with increased
bit-precision of the weights and activations of the significant layers in a
binary network. We show that the proposed Hybrid-Net achieves over 10%
improvement in classification accuracy over binary networks such as XNOR-Net
for ResNet and VGG architectures on CIFAR-100 and ImageNet datasets while still
achieving upto 94% of the energy-efficiency of XNOR-Nets. The proposed design
methodology allows us to move closer to the accuracy of standard full-precision
networks by keeping more than half of the network binary. This work
demonstrates an effective, one-shot methodology for designing hybrid,
mixed-precision networks which significantly improve the classification
performance of binary networks while attaining remarkable compression. The
proposed hybrid networks further the feasibility of using highly compressed
neural networks for energy-efficient neural computing in IOT-based edge
devices.
Source: arXiv (Cornell University)
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2948091815
