Adaptive aggregation methods for infinite horizon dynamic programming

Bertsekas, Dimitri P.; Castañón, David A.

doi:10.1109/9.24227

Cited by 140 publications

(100 citation statements)

References 18 publications

Supporting

Mentioning

100

Contrasting

Order By: Relevance

“…Bertsekas and Castafion (1989) developed an adaptive aggregation scheme for use with the policy iteration algorithm. Rather than relying on feature extraction, this approach automatically and adaptively aggregates states during the course of an algorithm based on probability transition matrices under greedy policies.…”

Section: Related Workmentioning

confidence: 99%

Feature-based methods for large scale dynamic programming

1996

View full text Add to dashboard Cite

Abstract. We develop a methodological framework and present a few different ways in which dynamic programming and compact representations can be combined to solve large scale stochastic control problems. In particular, we develop algorithms that employ two types of feature-based compact representations; that is, representations that involve feature extraction and a relatively simple approximation architecture. We prove the convergence of these algorithms and provide bounds on the approximation error. As an example, one of these algorithms is used to generate a strategy for the game of Tetris. Furthermore, we provide a counterexample illustrating the difficulties of integrating compact representations with dynamic programming, which exemplifies the shortcomings of certain simple approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

Feature-based methods for large scale dynamic programming

1996

View full text Add to dashboard Cite

show abstract

“…The aggregation is done in order to reduce the size of the state space. Some algorithms of this type include those in Hinderer (1978), Mendelssohn (1982), Bean et al (1987) (which is for deterministic dynamic programs only), and Bertsekas and Castanon (1989). Morin (1978) is a general survey paper of the older literature.…”

Section: Hierarchical Aggregation For Problems With Large Attribute Smentioning

confidence: 99%

The Dynamic Assignment Problem

Spivey

Powell

2004

Transportation Science

106

View full text Add to dashboard Cite

T here has been considerable recent interest in the dynamic vehicle routing problem, but the complexities of this problem class have generally restricted research to myopic models. In this paper, we address the simpler dynamic assignment problem, where a resource (container, vehicle, or driver) can serve only one task at a time. We propose a very general class of dynamic assignment models, and propose an adaptive, nonmyopic algorithm that involves iteratively solving sequences of assignment problems no larger than what would be required of a myopic model. We consider problems where the attribute space of future resources and tasks is small enough to be enumerated, and propose a hierarchical aggregation strategy for problems where the attribute spaces are too large to be enumerated. Finally, we use the formulation to also test the value of advance information, which offers a more realistic estimate over studies that use purely myopic models. The problem of dynamically assigning resources to tasks over time arises in a number of applications in transportation. In the field of freight transportation, truckload motor carriers, railroads, and shipping companies all have to manage fleets of containers (trucks, boxcars, and intermodal containers) that move one load at a time, with orders arriving continuously over time. In the passenger arena, taxi companies and companies that manage fleets of business jets have to assign vehicles (taxicabs or jets) to move customers from one location to the next. It is common to assume that the arrival of customer demands is random (e.g., known only through a probability distribution) over time, but it may also be the case that the vehicles become available in a random way. Finally, each assignment of a resource to a task generates a contribution to profits, which may also be random.We refer to the problem of dynamically assigning resources to tasks as a dynamic assignment problem. In general, it may be possible to assign a resource to a sequence of two or more tasks at the same time, but we focus on problems where we assign a resource to one task at a time. We assume that resources and tasks are each characterized by a set of possibly unique attributes, where the contribution generated by an assignment will depend on the attributes of the resource and task. Resources do not have to be used and tasks do not all have to be covered, although there can be a cost for holding either one.The dynamic assignment problem is a fundamental problem in routing and scheduling. It is a special case of the dynamic vehicle routing problem, without the complexities of in-vehicle consolidation. For this reason, it provides a natural framework for modeling the dynamic information processes and comparing myopic models with those that exploit distributional information about the future. It is common practice, for example, to model dynamic vehicle routing problems using myopic models, which ignore any forecasts of the future based on currently available data. These problems are themselves quite difficult be...

show abstract

“…The problem with aggregation is that the right level changes with the number of times you observe a set of states. Some authors have suggested changing the level of aggregation with the number of iterations (Bertsekas and Castanon (1989), Luus (2000)). …”

Section: Multilevel Aggregationmentioning

confidence: 99%

What you should know about approximate dynamic programming

Powell

2009

Naval Research Logistics

144

View full text Add to dashboard Cite

Approximate dynamic programming (ADP) is a broad umbrella for a modeling and algorithmic strategy for solving problems that are sometimes large and complex, and are usually (but not always) stochastic. It is most often presented as a method for overcoming the classic curse of dimensionality that is well‐known to plague the use of Bellman's equation. For many problems, there are actually up to three curses of dimensionality. But the richer message of approximate dynamic programming is learning what to learn, and how to learn it, to make better decisions over time. This article provides a brief review of approximate dynamic programming, without intending to be a complete tutorial. Instead, our goal is to provide a broader perspective of ADP and how it should be approached from the perspective of different problem classes. © 2009 Wiley Periodicals, Inc. Naval Research Logistics 2009

show abstract

Adaptive aggregation methods for infinite horizon dynamic programming

Cited by 140 publications

References 18 publications

Feature-based methods for large scale dynamic programming

Feature-based methods for large scale dynamic programming

The Dynamic Assignment Problem

What you should know about approximate dynamic programming

Contact Info

Product

Resources

About