Mechanistic models of reward based learning and decision making for clinically motivated problems
Year: 2018
Type: dissertation
Abstract: Mechanistic models of learning and decision making can help test specific hypotheses about observed behavior and brain function. This thesis presents a framework for integrating computational models of adaptive intelligence systems such as Reinforcement Learning and Bayesian Learning algorithms to address clinically motivated problems. In order to provide a comprehensive evaluation, Functional Magnetic Resonance Imaging (fMRI) data were analyzed with behavioral and connectivity models. In this work, some of the most widely used reinforcement learning algorithms in neuroimaging and psychological studies were evaluated with simulations to understand the behavior of agents under different model parameters and strategies. The models were then tested on a large empirical dataset, and the prediction errors that were derived from the winning model informed the general linear model for fMRI data analysis. Reinforcement learning models were able to capture differences in the function of dopaminergic brain regions and associated behavior in individuals with different genotype. It was further proposed that integrating learning algorithms in effective connectivity models can provide a complementary framework for studying altered brain network dynamics. This was achieved by constructing bilinear and nonlinear dynamic causal models of brain regions involved in reward and prediction error processing. Finally, hierarchical Bayesian models were implemented to model an agent's learning behavior in a complex, volatile environment. A parallel learning system approach was developed for learning and combining multiple cues by adopting the Hierarchical Gaussian Filter and the precision weighted response model pair. Simulations of parameter recovery suggest that this approach can be used for learning and combining different sources of information. Further, the proposed model was tested on a real dataset and compared against alternative models including an optimal Bayesian learner. This method allowed us to identify individual sub-processes involved in learning from social cues that differ according to the level of autistic traits. We further propose that the experimental and modeling approach presented here can contribute to mechanistic formulations of many psychiatric disorders.
Author: Meltem Sevgi
Institution: Technische Universität Ilmenau
Language: English
Subfield: Cognitive Neuroscience
Field: Neuroscience
Domain: Life Sciences
Keywords: reward learning, reinforcement learning, Bayesian learning, decision-making, prediction error, genotype differences, effective connectivity, dynamic causal modeling, Bayesian hierarchical model, hierarchical Gaussian filter, cue combination, social learning, autistic traits, mental illness
Open Access status: green
OpenAlex ID: W2911610344
