Year: 2024
Type: preprint
Abstract: We study security threats to Markov games due to information asymmetry and misinformation. We consider an attacker player who can spread misinformation about its reward function to influence the robust victim player's behavior. Given a fixed fake reward function, we derive the victim's policy under worst-case rationality and present polynomial-time algorithms to compute the attacker's optimal worst-case policy based on linear programming and backward induction. Then, we provide an efficient inception ("planting an idea in someone's mind") attack algorithm to find the optimal fake reward function within a restricted set of reward functions with dominant strategies. Importantly, our methods exploit the universal assumption of rationality to compute attacks efficiently. Thus, our work exposes a security vulnerability arising from standard game assumptions under misinformation.
Source: arXiv (Cornell University)
Institution: University of Wisconsin–Madison
Language: English
Topic: Game Theory and Applications
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
Awards: G2000969370, Digitally Inoculating Viewers Against Visual Misinformation With a Perceptual Training, SLES: Foundations of Safety-Aware Learning in the Wild, CNS Core: Medium: Resource Constrained Reinforcement Learning for Computing Systems, TRIPODS: Institute for Foundations of Data Science, CAREER: Embracing Local Minima and Nonsmoothness in Nonconvex Statistical Estimation: From Structures to Algorithms, CAREER: Stochasticity and Resilience in Reinforcement Learning: From Single to Multiple Agents, FMitF: Collaborative Research: Formal Methods for Machine Learning System Design
OpenAlex ID: W4400064663