New prioritized value iteration for Markov decision processes

Abstract

Abstract:The problem of solving large Markov decision processes accurately and quickly is challenging. Since the computational effort incurred is considerable, current research focuses on finding superior acceleration techniques. For instance, the convergence properties of current solution methods depend, to a great extent, on the order of backup operations. On one hand, algorithms such as topological sorting are able to find good orderings but their overhead is usually high. On the other hand, shortest path methods, such as Dijkstra's algorithm which is based on priority queues, have been applied successfully to the solution of deterministic shortest-path Markov decision processes. Here, we propose an improved value iteration algorithm based on Dijkstra's algorithm for solving shortest path Markov decision processes. The experimental results on a stochastic shortest-path problem show the feasibility of our approach.

Survey of Scientific Programming Techniques for the Management of Data-Intensive Engineering Environments

“…Euler models Scientific computation [104] Statistical methods [105] [ 106,154] Graph theory Automata [124] Graph/complex network analysis [37,[108][109][110][111][112][113][114][115][116][117] Engineering methods…”

Simplificación de los procesos de decisión de Markov mediante reglamentación de acciones y priorización de estados

“…Este enfoque aplica una reglamentación de acciones y un nuevo enfoque de priorización de estados en el algoritmo de iteración de valor. Al enfoque propuesto se le denominó IPVI (Improved Prioritized Value Iteration) [Garcia-Hernandez, 2012a]. Para verificar la robustez del enfoque propuesto, en este capítulo se presenta su implementación en el simulador de planificación de movimientos robóticos (SPRM) de Reyes et al [Reyes, 2006b], en una tarea compleja de ruta estocástica más corta, la cual a continuación se describe.…”

“…− el clásico algoritmo de iteración de valor (VI) [Puterman, 1994], − el algoritmo de iteración de valor con reglamentación de acciones (ARVI) [Garcia-Hernandez, 2009 [Dibangoye, 2008], − el algoritmo propuesto en esta tesis (IPVI) [Garcia-Hernandez, 2012a].…”