Work

“PyTDL”: A versatile temporal difference learning algorithm to simulate behavior process of decision making and cognitive learning

Year: 2024

Type: article

Abstract: Humans and animals excel at learning complex tasks through reward-based feedback, dynamically adjusting value expectations and choices based on past experiences to optimize outcomes. However, understanding the hidden cognitive components driving these behaviors remains challenging. Neuroscientists use the Temporal Difference (TD) learning model to estimate cognitive elements like value representation and prediction error during learning and decision-making processes. However, traditional TD algorithms fall short in diverse and dynamic tasks due to their fixed patterns. We present PyTDL, a Python-based modular framework that enables customizable value updating functions and decision policies, effectively simulating dynamic, non-linear cognitive processes. PyTDL's utility was demonstrated by modeling the decision-making processes of animals in two cognitive tasks under uncertain conditions. As open-source software, PyTDL offers a user-friendly GUI and APIs, empowering researchers to tailor models for specific tasks, align computational models with empirical data, and advance the understanding of brain learning and decision-making in complex environments.

Source: iScience

DOI: 10.1016/j.isci.2024.111600

Authors: Qiyun Wu, Xiaodan Yang, Kaishu Wang, Min Zhu, Jiejunyi Liang, Yunyun Han

Institutions: Huazhong University of Science and Technology, Tongji Hospital

Language: English

Topic: Neural and Behavioral Psychology Studies

Subfield: Cognitive Neuroscience

Field: Neuroscience

Domain: Life Sciences

Open Access status: gold

Estimated APC paid: $3,150

Funders: National Natural Science Foundation of China, Government of Jiangsu Province, Natural Science Foundation of Hubei Province