Enabling Spike-based Backpropagation in State-of-the-art Deep Neural Network Architectures
Year: 2019
Type: preprint
Abstract: Spiking Neural Networks (SNNs) has recently emerged as a prominent neuralcomputing paradigm. However, the typical shallow spiking network architectures
have limited capacity for expressing complex representations, while training a
very deep spiking network have not been successful so far. Diverse methods have
been proposed to get around this issue such as converting off-line trained deep
Artificial Neural Networks (ANNs) to SNNs. However, ANN-to-SNN conversion
scheme fails to capture the temporal dynamics of a spiking system. On the other
hand, it is still a difficult problem to directly train deep SNNs using input
spike events due to the discontinuous and non-differentiable nature of the
spike signals. To overcome this problem, we propose using differentiable (but
approximate) activation for Leaky Integrate-and-Fire (LIF) spiking neurons to
train deep convolutional SNNs with input spike events using spike-based
backpropagation algorithm. Our experiments show the effectiveness of the
proposed spike-based learning strategy on state-of-the-art deep networks (VGG
and Residual architectures) by achieving the best classification accuracies in
MNIST, SVHN and CIFAR-10 datasets compared to other SNNs trained with
spike-based learning. Moreover, we analyze sparse event-driven computations to
demonstrate the efficacy of proposed SNN training method for inference
operation in the spiking domain.
Source: arXiv (Cornell University)
Authors: Chankyu Lee, Syed Shakib Sarwar, Kaushik Roy
Language: English
Subfield: Electrical and Electronic Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2923485687