Work

Deep Learning without Weight Transport

Year: 2019
Type: preprint
Abstract: Current algorithms for deep learning probably cannot run in the brain because they rely on weight transport, where forward-path neurons transmit their synaptic weights to a feedback path, in a way that is likely impossible biologically. An algorithm called feedback alignment achieves deep learning without weight transport by using random feedback weights, but it performs poorly on hard visual-recognition tasks. Here we describe two mechanisms - a neural circuit called a weight mirror and a modification of an algorithm proposed by Kolen and Pollack in 1994 - both of which let the feedback path learn appropriate synaptic weights quickly and accurately even in large networks, without weight transport or complex wiring.Tested on the ImageNet visual-recognition task, these mechanisms outperform both feedback alignment and the newer sign-symmetry method, and nearly match backprop, the standard algorithm of deep learning, which uses weight transport.
Authors: Akrout, Mohamed, Wilson, Collin, Humphreys, Peter C., Lillicrap, Timothy, Tweed, Douglas
Language: English
Open Access status: green
OpenAlex ID: W2935958002
Cites: 32
Cited by: 201
Related to: 9