Algorithm Selection for Reinforcement Learning
Year: 2017
Type: preprint
Abstract: This paper formalises the problem of online algorithm selection in thecontext of Reinforcement Learning. The setup is as follows: given an episodic
task and a finite number of off-policy RL algorithms, a meta-algorithm has to
decide which RL algorithm is in control during the next episode so as to
maximize the expected return. The article presents a novel meta-algorithm,
called Epochal Stochastic Bandit Algorithm Selection (ESBAS). Its principle is
to freeze the policy updates at each epoch, and to leave a rebooted stochastic
bandit in charge of the algorithm selection. Under some assumptions, a thorough
theoretical analysis demonstrates its near-optimality considering the
structural sampling budget limitations. ESBAS is first empirically validated on
a dialogue task under 32 algorithm configurations. Another experiment on a
fruit collection task shows that ESBAS can successfully be adapted to a true
online setting where algorithms update their policies after each transition.
Source: arXiv (Cornell University)
Authors: Romain Laroche, Raphaël Féraud
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2721507161