Learning to imagine: generative models of memory construction and consolidation
Year: 2024
Type: dissertation
Abstract: Episodic memory is the (re)construction of an experience rather than the retrieval of a copy; memories involve schema-based predictions, show classic patterns of distortion, and share neural substrates with imagination. Brains need to make predictions to survive, and to achieve this must extract statistical structure from experience. Generative neural networks provide a mechanism for learning this by ‘prediction error’ minimisation. I explore how the brain develops generative models through memory consolidation, how these models reconstruct experiences during memory ‘retrieval’, and how they support other cognitive functions. \n \nFirst I present a computational model in which episodic memories are initially encoded in the hippocampus (a modern Hopfield network), then replayed to train a neocortical generative network (variational autoencoder) to (re)create sensory experiences via latent variable representations. Using images, I simulate how this generative network supports episodic memory, semantic memory, imagination, and inference. The network can reconstruct scenes from partial inputs according to learned schemas (which produces gist-based distortions) and imagine novel scenes consistent with those schemas. I also show how unique and predictable elements of memories could be stored and reconstructed by efficiently combining both hippocampal and neocortical systems, optimising the use of limited hippocampal storage. \n \nI then extend the model to sequential stimuli, with the generative networks trained not only to reconstruct their own inputs, but to predict the next input during replay. I apply this model to statistical learning, relational inference, and planning tasks, consider memory distortions in narratives, and explore ‘retrieval augmented generation’ as a model of hippocampal-neocortical interaction during recall. Finally, I address the question of continual learning, and suggest that generative replay may stabilise existing memories as new ones are assimilated into the generative model. \n \nIn conclusion, I explore how replayed memories update a generative, or predictive, model of the world, which supports multiple cognitive functions.
Author: Spens, Eleanor
Language: English
Topic: Memory and Neural Mechanisms
Subfield: Cognitive Neuroscience
Field: Neuroscience
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7039266717