Work

Similarities in Neuron Vectors and The Implications to CNN Inferences

Year: 2018
Type: report
Abstract: This technical report presents deep reuse, a method for speeding up CNN inferences by detecting and exploiting deep reusable computations on the fly.It empirically reveals the massive similarities among neuron vectors in activation maps, both within CNN inferences on an input and across inputs.It gives an in-depth study on how to effectively turn the similarities into beneficial computation reuse to speed up CNN inferences.The investigation covers various factors, ranging from the clustering methods for similarity detection, to clustering scopes, similarity metrics, and neuron vector granularities.The insights help create deep reuse.As an on-line method, deep reuse is easy to apply and adaptive to each CNN (compressed or not), and its input.Using no special hardware support or CNN model changes, this method speeds up inferences by 1.77-2X (up to 4.3X layer-wise) on the fly with virtually no (<0.0005)loss in accuracy.
Language: English
Open Access status: green
OpenAlex ID: W2948603148
Cites: 21
Cited by: 2
Related to: 10