Work

Efficiency of Local Learning Rules in Threshold-Linear Associative Networks

Year: 2021

Type: article

Abstract: We derive the Gardner storage capacity for associative networks of threshold linear units, and show that with Hebbian learning they can operate closer to such Gardner bound than binary networks, and even surpass it. This is largely achieved through a sparsification of the retrieved patterns, which we analyze for theoretical and empirical distributions of activity. As reaching the optimal capacity via nonlocal learning rules like back propagation requires slow and neurally implausible training procedures, our results indicate that one-shot self-organized Hebbian learning can be just as efficient.

Source: Physical Review Letters

DOI: 10.1103/physrevlett.126.018301

Authors: Francesca Schönsberg, Yasser Roudi, Alessandro Treves

Institutions: Scuola Internazionale Superiore di Studi Avanzati, Norwegian University of Science and Technology, Kavli Institute for Systems Neuroscience

Language: English

Topic: Neural Networks and Applications

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $3,750

Funders: Kavli Foundation, Human Frontier Science Program, Norges Forskningsråd, H2020 Marie Skłodowska-Curie Actions