Unbounded dynamic programming via the Q-transform

An Approximation Approach to Dynamic Programming With Unbounded Returns

“…We compare our findings in this paper with the most recent literature on unbounded returns with time-additive aggregators of the form Ma et al (2022) present an innovative approach to unbounded returns based on the idea of a 𝑄-transform. Rincón-Zapatero (2024) studies unbounded returns under minimal assumptions by means of a contraction-type method.…”

“…Another method followed by Le Van and Morhaim (2002), Le Van and Vailakis (2005), Kamihigashi (2014) and Wiszniewska-Matyszkiel and Singh (2021) abandons the contraction approach and looks directly for solutions to the Bellman's equation in a suitable space of functions satisfying a sort of transversality condition. Finally, a recent paper by Ma et al (2022) exploits a transformation of Bellman's operator, along with boundedness of the expected reward, to turn unbounded into bounded programs so that conventional contraction techniques apply.…”

“…The idea in Ma et al (2022) consists in transforming the aggregator to obtain, under additional restrictions, bounded returns. Their translation can be understood as…”

Do not Blame Bellman: It Is Koopmans' Fault

“…Jaśkiewisz and Nowak (2011) and Matkowski and Nowak (2011) presented a systematic study of all these approaches under uncertainty. Finally, a recent paper by Ma, Stachurski, and Akira Toda (2022) exploited a transformation of the Bellman operator, along with boundedness of the expected reward, to turn unbounded into bounded programs, so that conventional contraction techniques apply.…”

Fifty years of mathematical growth theory: Classical topics and new trends

“…In the paper [85] the author proposed a new approach to solving dynamic programs with unbounded rewards, based on Q-transforms which have their root in the machine learning algorithms. To give rough explanation, the author take a simple Bellman Equation of the following type…”