Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network Based on Analog Resistive Synapse
Year: 2017
Type: article
Abstract: Asymmetric nonlinear weight update is considered as one of the major obstacles for realizing hardware neural networks based on analog resistive synapses, because it significantly compromises the online training capability. This paper provides new solutions to this critical issue through co-optimization with the hardware-applicable deep-learning algorithms. New insights on engineering activation functions and a threshold weight update scheme effectively suppress the undesirable training noise induced by inaccurate weight update. We successfully trained a two-layer perceptron network online and improved the classification accuracy of MNIST handwritten digit data set to 87.8%/94.8% by using 6-/8-b analog synapses, respectively, with extremely high asymmetric nonlinearity.
Authors: Chih-Cheng Chang, Pin-Chun Chen, Teyuh Chou, I-Ting Wang, Boris Hudec, Che-Chia Chang, Chia-Ming Tsai, Tian-Sheuan Chang, Tuo-Hung Hou
Institution: National Yang Ming Chiao Tung University
Language: English
Subfield: Electrical and Electronic Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
Awards: Development of Low-Power Electronic Synaptic Device and Application of Neuromorphic Computing Accelerator( I ), International Center of Excellence for Advanced Heterogeneous Integration for Green Electronics Research( V ), Lego Semiconductor Core Technology for Intelligent Electronics Platform( I )
OpenAlex ID: W2767433132
