Deterministic Discounted Markov Decision Processes with Fuzzy Rewards/Costs

Abstract

The article concerns a study of infinite-horizon deterministic Markov decision processes (MDPs) for which the fuzzy environment will be presented through considering these MDPs with both fuzzy rewards and fuzzy costs. Specifically, these rewards and costs will be assumed of a suitable trapezoidal type. For both classes of MDPs, i.e., MDPs with fuzzy rewards and MDPs with fuzzy costs, the fuzzy total discounted function will be taken into account as the objective function, and the corresponding optimal decision problems will be considered with respect to the max order of the fuzzy numbers. For each optimal decision problem, the optimal policy and the optimal value function are related and obtained as a solution of a convenient standard MDP (i.e., a standard MDP is an MDP with a non-fuzzy reward function or a non-fuzzy cost function). Moreover, an economic growth model (EGM), a deterministic version of the linear-quadratic model (LQM), and an optimal consumption model (OCM) in order to clarify the theory presented are given, and it is remarked that these models have uncountable state spaces, and the corresponding non-fuzzy version of both the EGM and the OCM has an unbounded reward function, and the corresponding non-fuzzy version of the LQM has an unbounded cost function. KEYWORDSdeterministic Markov decision process; discounted criterion; fuzzy reward; fuzzy cost; trapezoidal fuzzy number T his article deals with the extension to the fuzzy context [1,2] of the class of the infinite-horizon deterministic discounted Markov decision processes (MDPs), which are sequential decision models theoretically interesting and highly applicable mainly in economics [3−13] . The deterministic MDPs which will be extended have spaces of states and decisions, Borel spaces, (possibly) noncompact restriction sets, and both cases will be considered: MDPs with rewards as well as MDPs with costs [4,5,14] , in this paper, an MDP of this class will be referred to as a standard MDP. Moreover, for the standard MDPs considered here, there are well-known conditions for the existence of optimal policies [4,14,15] , conditions that are supposed to hold and permit to take general enough MDPs into account; for instance, it is possible to consider MDPs with uncountable state spaces. This last mentioned kind is important, because the technique commonly used to solve the deterministic MDPs is the Euler's Equation [10,16,17] , which involves the derivative of the value function defined in a suitable open set.