Certified Nash Equilibrium Learning for Nonzero-Sum Jump-Diffusion Games: Policy Iteration and Off-Policy Q-Learning
Year: 2026
Type: preprint
Abstract: This paper establishes two model-free reinforcement learning methods for infinite-horizon N-player linear-quadratic nonzero-sum stochastic differential games whose drift, Brownian diffusion, and Poisson jump amplitudes depend on all players' controls. Control dependence in the diffusion and jump terms couples the players' simultaneous feedback updates. A verifiable certificate ensures that the partially model-free policy iteration map is contractive on an invariant closed ball centered at an admissible initial feedback profile. All feedback iterates are mean-square stabilizing, and the algorithm achieves local geometric convergence of feedback gains and value matrices to those associated with a closed-loop Nash equilibrium. The term certified refers to a verifiable convergence certificate that guarantees mean-square stability of all policy iterates and local geometric convergence to a closed-loop Nash equilibrium. To implement the policy updates from data, the off-policy Q-learning algorithm reuses the same behavior-policy data across iterations without requiring the system coefficients or jump intensity. Under quantitative persistence of excitation and a common initialization, exact population regression reproduces the policy-iteration sequence. The same local convergence guarantee therefore applies to off-policy Q-learning at the population level. Simulation results demonstrate the effectiveness of the two proposed algorithms.
Source: arXiv (Cornell University)
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
Awards: G213983658, G6752647633, G7049880175
OpenAlex ID: W7235631419
