Dynamic Programming on Reduced Models and Its Evaluation through Its Application to Elevator Operation Problems

Abstract

: In this paper, we present a modified dynamic programming (DP) method. The method is basically the same as the value iteration method (VI), a representative DP method, except the preprocess of a system's state transition model for reducing its complexity, and is called the dynamic programming on reduced models (DPRM). That reduction is achieved by imaginarily considering causes of the probabilistic behavior of a system, and then cutting off some causes with low occurring probabilities. In computational illustrations, VI, DPRM, and the real-time Q-learning method (RTQ) are applied to elevator operation problems, which can be modeled by using Markov decision processes. The results show that DPRM can compute quasi-optimal value functions which bring more effective allocations of elevators than value functions by RTQ in less computational times than VI. This characteristic is notable when the traffic pattern is complicated.

Intermittently Proving Dynamic Programming to Solve Infinite MDPs on GPUs

“…These illustrations displayed that the IPDP is about 35, 707/469. 4 76 and 1, 029, 120/22, 685 45 times faster than VI on a CPU for those problems, respectively. Furthermore, as expected, it was also displayed that IPDP is about 975.4/469.4 2 and 146, 829/22, 685 6 times faster than VI on a GPU.…”

“…Furthermore, we believe that the elemental cause which disturbs that system can be evinced with its occurring probability. Such cause must be discrete and finite, and is called situational input [4] w ∈ W where W denotes the situational input space. By representing the occurring probability of w as P (w), the state transition probability is formalized as follows [3]:…”

“…The state transition function can be represented as a binary-vector function by representing state, decision, and situational input as binary vectors, and formalizing the progress of each state variable as a binary function [4].…”

Optimal Periods for Probing Convergence of Infinite-stage Dynamic Programmings on GPUs

“…Furthermore, we believe that the elemental fountain which disturbs that system can be evinced with its occurring probability. Such fountain must be discrete and finite in computers, and is called situational input [6]. A situational input is denoted w ∈ W, where W is the situational input space.…”

“…The state transition function can be represented as a binary vector-valued function by representing state, decision, and situational input as binary vectors, and formalizing the progress of each state variable as a binary function [6]. Thus, it is basically possible, and may be promising if |W| is small, to formalize state transition probabilities of a reproducible system.…”