Generalized Gradient Estimation for Variational Autoencoders and Reinforcement Learning
Year: 2025
Type: article
Abstract: This work presents a generalized gradient estimator that optimizes expectations involving known or black-box functions for discrete and continuous random variables. We synthesize and extend standard methods for constructing gradient estimators, offering a framework that incurs minimal computational overhead. Our proposed approach demonstrates effectiveness in variational autoencoders and introduces a straightforward extension to reinforcement learning, accommodating discrete and continuous action settings. Experiment-tal results reveal improved training performance and sample efficiency, highlighting the utility of our estimator in various domains. Future applications include training models with complex attention mechanisms, continuous latent-variable models with non-differentiable likelihoods, and integrating our method with existing variance-reduction techniques and optimization methods in reinforcement learning.
Author: Hyacinthe Hamon -
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4408102587