Compute to learn: Neural implementation of computations underlying associative learning and decision making
Year: 2016
Type: dissertation
Abstract: MotivationIn 1950, Alan Turing published a paper to suggest a game, the "imitation game".This paper provided an objective measure of how the question of "can machines think?" could be addressed (Turing, 1950).This was, in a way, the beginning of artificial intelligence, although the term was invented in 1956, two years after Turing's untimely and sad death.Just a few months before that, John Cade, an Australian psychiatrist, serendipitously discovered the sedating effects of lithium carbonate in controlling psychotic excitement (Cade, 1949).This was the beginning of a decade of success for psychiatry in which new medications for several psychiatric disorders were discovered.65 years later, building on recent developments in artificial intelligence, computational neuroscientists and computer scientists have joined the efforts of psychiatrists to understand mental disorders.Consequently, new questions are arising: can machines simulate mental disorders?If they can, can we discover new treatments for mental disorders by simulating their effects on those machines? On computational psychiatryPsychiatry is at a crossroads (Insel et al., 2010; Sahakian et al., 2010;Hyman, 2012;Robbins et al., 2012).Patients are typically diagnosed in late phases of their illness, when diseases are already chronic and relapsing.Diagnosis based on descriptive symptoms is often challenging due to complexity and heterogeneity of the psychiatric symptoms and the high chance of co-morbidity among disorders (McHugh, 2005).Consequently, often there are neither necessary nor sufficient conditions for defining a particular disease (Robbins et al., 2012).Furthermore, most available medications lack proper biological bases and might have severe side effects (Insel et al., 2010).In the last two decade, cognitive and systems neurosciences have advanced our understanding of neurobiological bases of brain disorders using state of the art neuroimaging and preclinical tools.This approach can help psychiatry to go beyond descriptive subjective symptoms reported by patients and Chapter 1 independent of the variational distribution.Therefore, any change in the negative free energy with respect to the variational distribution necessarily reflects a similar change in the divergence in opposite direction as their sum is independent of changes in the variational distribution.From these three features, we conclude that the negative free energy can be maximized with respect to parameters of the variational distribution, q(s), which necessarily reduces the divergence (as their sum is independent of q(s)) and provides an approximation for the posterior, p(s | o,a).Therefore, by choosing "appropriate" variational distributions, the animal can solve the perception problem (inferring environmental state based on sensory observations).This variational approximation also provides a promising approach to efficiently approximate the intractable integral in Equation 3, arising in computing expected value of state-action pairs.Specifically, by replacing p(s | o,a) with q(s) in Equation 3, one could compute an approximate expected value.The limitations is that the variational distribution, q(s), should be chosen in a way to also simplify computing the integral in Equation 3given the form of value function, V(s,a). Active inference frameworkThe active inference framework extends the variational treatment of perception to the domain of action selection (Friston and Stephan, 2007;Friston, 2013).To explain this framework, we should see the whole process of perception and action in an iterative manner: animals infer the state of the world based on their sensory input (perception) and sample the world (i.e.change their sensory observations) given the inferred state of the world (action).Therefore, every action is followed by a perception process because the sensory observations have been changed.For example, eye movements sample the environment and thereby change the visual inputs, which then could change our perception of the world.From this perspective, action is intimately linked with perception.Now suppose that the animal has approximated the current state of the world by evaluating the variational distribution, q(s).We already mentioned that -F (q(s),a) is an approximation of (log-) sensory evidence, p(o | a), which is the animal's ability to predict its own sensory inputs given the action.If we assume that a goal of the animal is to be able to better predict its own sensory observations, then one way to change the sensory input is to make actions that increase p(o | a).Since free energy is a tractable approximation of this quantity, animal could minimize the free energy with respect to action, a, which necessarily maximizes predictability of sensory observations, p(o | a).This simple situation can give us some hints about how priors might corrupt inference.Given a fixed internal model, inference can be corrupted due to "bad" priors in two ways: 1) The prior is too wide resulting in a too complex model that is able to make a wide variety of predictions.Such a model is probably too slow and Modern systems-and cognitive-neuroscience theories of addiction indicate that addiction arises as a progressive transition from goal-directed voluntary drugseeking to maladaptive habitual drug-taking behavior (Everitt et al., 2008).In terms of neural control, there is a corresponding transition from the affective and cognitive loops of corticostriatal circuitry to the sensorimotor corticostriatal loop.This transition is mediated by dopaminergic spiraling connections between the
Author: Payam Piray
Institution: Radboud University Nijmegen
Language: English
Topic: Memory and Neural Mechanisms
Subfield: Cognitive Neuroscience
Field: Neuroscience
Domain: Life Sciences
Keywords: computational psychiatry, associative learning, decision-making, variational free energy, variational inference, variational distribution, artificial intelligence, mental illness, brain disorders, Turing test, lithium carbonate, history of psychiatry, psychiatric diagnosis, psychiatric medication, variational autoencoder
Open Access status: green
OpenAlex ID: W2563536771
