Promoting the Emergence of Cooperation in Networked Self-interested Agent societies: the Roles of Reinforcement Social Learning and Fixed-strategy Agents
Year: 2019
Type: dissertation
Abstract: Cooperation plays a very important role in the development of society and the processes of evolution. However, understanding how to sustain and promote cooperative behaviour among selfish individuals is a central challenge in biology and social science. The emergence of cooperative behaviours among self-interested agents is also an essential research problem in computer science. Current research mainly focuses on designing a mechanism that can largely promote the cooperative behaviours of agents that face social dilemmas. Agents usually are constructed in particular network topologies due to some network structures ability to promote cooperation. The traditional learning models, such as replicate dynamics, fail to emerge to a high level of cooperation quickly when starting with a small initial fraction of cooperators or even lead to the extinction of cooperative behaviours under small-world networks. The most successful of these rules are based on a special combination of game model, network structure and the initial fraction of cooperators. Moreover, traditional learning-based update rules simply copy others' strategies but ignore the fact that humans learn from trial and error. In this thesis, we consider N-player iterated Prisoner's Dilemma (PD) games as the interaction game model, in which agents have only two strategies, either cooperate or defect. In each round, agents gain a total payoff by interacting with all their neighbours. Reinforcement learning is experience-based learning and it has been regarded as a powerful method for studying the behaviour of agents in multi agent systems. Thus, we propose new strategy update rules based on reinforcement learning, especially Q-learning, in the context of individual learning (IL) and social learning (SL). We introduce two learning strategies, IL refers to learning from ones own self and SL refers to learning by imitating others' behaviour through communication or observation. We also investigate the aggregated algorithms by combining IL and SL in order to design a more reasonable mechanism and better understand the behaviour of agents in the emergence of cooperation. Furthermore, we propose the concept of a role model, which is defined as an unconditional cooperator (UC) that corporate with different strategy update rules in order to facilitate the emergence of cooperation. An unconditional cooperator denotes a type of cooperator that will never change their strategy. Finally, we consider the location or influence of agents to further judge whether this can further promote the emergence of cooperation in MASs. We conducted a large number of experiments to evaluate these conditions by considering various parameters. For examples, we consider a set of popular network models, such as scale-free and small-world networks, including some real-world networks. The result reveals that strategy update rules based on reinforcement learning and social learning while considering the influence of agents can largely promote the emergence of cooperation.
Author: Zhou, Kaisheng
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W2990270435