Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning
Year: 2025
Type: article
Abstract: We prove a nonasymptotic central limit theorem (CLT) for vector-valued martingale differences using Stein’s method, and we use Poisson’s equation to extend the result to functions of Markov chains. We then show that these results can be applied to establish a nonasymptotic CLT for temporal difference learning with averaging. Funding: This work was supported by National Science Foundation [Grants CNS 23-12714, CCF 22-07547, and CNS 21-06801] and Air Force Office of Scientific Research [Grant FA9550-24-1-0002].
Author: R. Srikant
Institution: University of Illinois Urbana-Champaign
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
Awards: G1104240869, G1523888516, HDR TRIPODS: Illinois Institute for Data Science and Dynamical Systems (iDS2), G4244981204, NeTS: Medium: SLATE: Service Layer Traffic Engineering, G6098521345, Collaborative Research: CIF: Small: Nonasymptotic Analysis for Stochastic Networks and Systems: Foundations and Applications, Collaborative Research: CNS Core: Medium: Foundations and Scalable Algorithms for Personalized and Collaborative Virtual Reality Over Wireless Networks, G8876996369
OpenAlex ID: W4414783466