Work

Stochastic Action Value Gradients

Year: 2017
Type: preprint
Abstract: Stochastic off-policy actor-critic methods rely on approximating the policy gradient using the first term of the complete off-policy theorem. As noted by Degris et al (2012), the other term in the theorem, the stochastic action-gradient term, is difficult to estimate under the incremental off-policy setting. Using a compatible action-value function, we propose a method of estimating this term such that, similar to the first term, it also follows the approximate policy gradient.
Language: English
Open Access status: green
OpenAlex ID: W2592619258
Cites: 15
Cited by:
Related to: 10