Year: 2025
Type: article
Abstract: Reinforcement learning (RL) methods increasingly underpin modern ranking, recommendation, and generative systems, yet challenges of bias, variance, safety, and sample inefficiency hinder their deployment [Gupta et al., 2024a]. This dissertation advances contextual bandits and RL to address these challenges in two critical domains: learning-to-rank and text-to-image diffusion models. The first part develops safe counterfactual learning-to-rank (CLTR) methods that leverage user interaction logs to learn improved ranking policies without online experimentation. Conventional inverse propensity scoring produces high-variance estimates [Gupta et al., 2024a], rendering learned policies unsafe to deploy. We introduce an exposure-based generalization bound that guarantees new policies perform at least as well as the logging policy with high confidence [Gupta et al., 2023]. Recognizing that guarantees depend on user model assumptions, we further develop a robust safety framework maintaining reliability even when real-world behavior deviates from assumed click models [Gupta et al., 2024c]. These methods substantially reduce unsafe initial warm-up periods in deployment ultimately making CLTR safe for deployment. The second part addresses variance reduction and sample efficiency. For contextual bandits, we propose a unified framework that yields closed-form, variance-optimal baseline corrections for off-policy evaluation and optimization [Gupta et al., 2024b]. For diffusion models, we introduce Leave-One-Out PPO (LOOP), combining REINFORCE's computational simplicity with PPO's sample efficiency to improve aesthetic quality, and prompt instruction following while requiring fewer input samples [Gupta et al., 2025]. In summary, this thesis contributes theoretical guarantees, algorithms, and empirical findings that make RL methods safer, more reliable, and more efficient for ranking systems and diffusion models, supporting broader deployment in user-facing applications. Awarded by: University of Amsterdam, Amsterdam, The Netherlands on 13 October 2025. Supervised by: Maarten de Rijke, Harrie Oosterhuis. Available at: https://pure.uva.nl/ws/files/257088241/Thesis.pdf.
Source: ACM SIGIR Forum
Author: Shashank Gupta
Institution: University of Amsterdam
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7133570163
FWCI:
Cites:
Cited by:
Related to: