Work

An Efficient Solution to s-Rectangular Robust Markov Decision Processes

Year: 2023

Type: preprint

Abstract: We present an efficient robust value iteration for \texttt{s}-rectangular robust Markov Decision Processes (MDPs) with a time complexity comparable to standard (non-robust) MDPs which is significantly faster than any existing method. We do so by deriving the optimal robust Bellman operator in concrete forms using our $L_p$ water filling lemma. We unveil the exact form of the optimal policies, which turn out to be novel threshold policies with the probability of playing an action proportional to its advantage.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2301.13642

Authors: Kumar, Navdeep, Levy, Kfir, Wang, Kaixin, Mannor, Shie

Institutions: Technion – Israel Institute of Technology, National University of Singapore

Language: English

Topic: Reinforcement Learning in Robotics

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W4318908150

Cites:

Cited by:

Related to: 10