Work

Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models

Year: 2022

Type: article

Abstract: . We derive all these memory models as instances of our general framework with differing similarity and separation functions. We extend the mathematical framework of Krotov & Hopfield (2020) to express general associative memory models using neural network dynamics with local computation, and derive a general energy function that is a Lyapunov function of the dynamics. Finally, using our framework, we empirically investigate the capacity of using different similarity functions for these associative memory models, beyond the dot product similarity measure, and demonstrate empirically that Euclidean or Manhattan distance similarity metrics perform substantially better in practice on many tasks, enabling a more robust retrieval and higher memory capacity than existing models.

Source: PubMed

DOI: 10.48550/arxiv.2202.04557

Authors: Millidge, Beren, Salvatori, Tommaso, Song, Yuhang, Lukasiewicz, Thomas, Bogacz, Rafal

Institutions: University of Oxford, Brain Network Dynamics Unit, TU Wien

Language: English

Topic: Neural Networks and Applications

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green