Work

Undiscounted Semi-Markov Decision Processes with Countably Infinite Action Spaces

Year: 2026

Type: article

Abstract: In this article we study semi-Markov decision processes (SMDPs) where the pay-off criterion is limiting ratio average, generally known as undiscounted pay-off. Here we consider the action space of the decision maker to be possibly countably infinite. However, we do not put any restriction on the reward function. We prove the existence of a near-optimal or ϵ-optimal strategy of the decision maker which turns out to be a deterministic semi-stationary. An efficient algorithm is discussed to compute a near-optimal pure semi-stationary strategy for such SMDP model. Also under some standard ergodicity conditions, we propose an optimality equation of these SMDP models.

Source: Algorithms

DOI: 10.3390/a19070529

Authors: Kushal Guha Bakshi, Sagnik Sinha, Ramakant Bhardwaj, Purvee Bhardwaj, Satyendra Narayan

Institutions: Jadavpur University, Algoma University

Language: English

Topic: Adaptive Dynamic Programming Control

Subfield: Computational Theory and Mathematics

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $1,930

OpenAlex ID: W7166738227

FWCI:

Cites:

Cited by:

Related to: