Extreme Occupation Measures in Markov Decision Processes with an Absorbing State

Piunovskiy, Alexey; Zhang, Yi

doi:10.1137/23m1572398

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Other2

Article1

Relationship

Self Cite1

Independent2

Authors

Journals

Cited by 3 publications

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

On the Continuity of the Projection Mapping from Strategic Measures to Occupation Measures in Absorbing Markov Decision Processes

Piunovskiy,

Zhang

2024

Appl Math Optim

Self Cite

View full text Add to dashboard Cite

In this paper, we prove the following assertion for an absorbing Markov decision process (MDP) with the given initial distribution, which is also assumed to be semi-continuous: the continuity of the projection mapping from the space of strategic measures to the space of occupation measures, both endowed with their weak topologies, is equivalent to the MDP model being uniformly absorbing. An example demonstrates, among other interesting scenarios, that for an absorbing (but not uniformly absorbing) semi-continuous MDP with the given initial distribution, the space of occupation measures can fail to be compact in the weak topology.

show abstract