Work

Optimizing the CVaR via Sampling

Year: 2015

Type: article

Abstract: Conditional Value at Risk (CVaR) is a prominent risk measure that is being used extensively in various domains. We develop a new formula for the gradient of the CVaR in the form of a conditional expectation. Based on this formula, we propose a novel sampling-based estimator for the gradient of the CVaR, in the spirit of the likelihood-ratio method. We analyze the bias of the estimator, and prove the convergence of a corresponding stochastic gradient descent algorithm to a local CVaR optimum. Our method allows to consider CVaR optimization in new domains. As an example, we consider a reinforcement learning application, and learn a risk-sensitive controller for the game of Tetris.

Source: AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI))

DOI: 10.1609/aaai.v29i1.9561

Authors: Tamar, Aviv, Glassner, Yonatan, Mannor, Shie

Institution: Technion – Israel Institute of Technology

Language: English

Topic: Risk and Portfolio Optimization

Subfield: Management Science and Operations Research

Field: Decision Sciences

Domain: Social Sciences

Open Access status: green

Funders: European Commission, FP7 Ideas: European Research Council

Awards: G3207356259, Scaling Up Reinforcement Learning: Structure Learning, Skill Acquisition, and Reward Shaping"", G896728979