Deep Learning for Markov Chains: Lyapunov Functions, Poisson's Equation, and Stationary Distributions
Year: 2025
Type: preprint
Abstract: Lyapunov functions are fundamental to establishing the stability of Markovian models, yet their construction typically demands substantial creativity and analytical effort. In this paper, we show that deep learning can automate this process by training neural networks to satisfy integral equations derived from first-transition analysis. Beyond stability analysis, our approach can be adapted to solve Poisson's equation and estimate stationary distributions. While neural networks are inherently function approximators on compact domains, it turns out that our approach remains effective when applied to Markov chains on non-compact state spaces. We demonstrate the effectiveness of this methodology through several examples from queueing theory and beyond.
Source: arXiv (Cornell University)
Authors: Qu, Yanlin, Blanchet, Jose, Glynn, Peter
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7077973588
Cites:
Cited by:
Related to: