An Approximate Dynamic Programming Approach to Decentralized Control of Stochastic Systems

Cogill, Randy; Rotkowitz, Michael; Roy, Benjamin Van; Lall, Sanjay

doi:10.1007/11664550_13

Cited by 34 publications

(61 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many different domains where these problems arise have been studied by researchers in recent years. Examples include coordination of space exploration rovers [51], load balancing for decentralized queues [11], coordinated helicopter flights [39,49], multi-access broadcast channels [29], and sensor network management [27]. In all these problems, multiple decision makers jointly control a process, but cannot share all of their information in every time step.…”

Section: Motivationmentioning

confidence: 99%

“…The algorithm developed by Cogill et al [11] generalizes this algorithm by extending it to decentralized problems. Their approach is the first to use centralized solution techniques to tackle a decentralized problem.…”

Section: Approximate Dynamic Programmingmentioning

confidence: 99%

“…5.3.2 then presents the approximate linear programming algorithm for decentralized problems. In presenting the formal models and algorithms, some notation from [11] has been changed to facilitate a comparison with the DEC-POMDP model.…”

Section: Approximate Dynamic Programmingmentioning

confidence: 99%

“…In their paper, Cogill et al [11] prove two theorems which relate the approximation error between Q * andQ to the difference in the achieved value, i.e. the difference between J * and J .…”

Section: Approximate Dynamic Programming For Decentralized Problemsmentioning

confidence: 99%

See 3 more Smart Citations

Formal models and algorithms for decentralized decision making under uncertainty

Seuken

Zilberstein

2008

Auton Agent Multi-Agent Syst

138

104

View full text Add to dashboard Cite

Over the last 5 years, the AI community has shown considerable interest in decentralized control of multiple decision makers or "agents" under uncertainty. This problem arises in many application domains, such as multi-robot coordination, manufacturing, information gathering, and load balancing. Such problems must be treated as decentralized decision problems because each agent may have different partial information about the other agents and about the state of the world. It has been shown that these problems are significantly harder than their centralized counterparts, requiring new formal models and algorithms to be developed. Rapid progress in recent years has produced a number of different frameworks, complexity results, and planning algorithms. The objectives of this paper are to provide a comprehensive overview of these results, to compare and contrast the existing frameworks, and to provide a deeper understanding of their relationships with one another, their strengths, and their weaknesses. While we focus on cooperative systems, we do point out important connections with game-theoretic approaches. We analyze five different formal frameworks, three different optimal algorithms, as well as a series of approximation techniques. The paper provides interesting insights into the structure of decentralized problems, the expressiveness of the various models, and the relative advantages and limitations of the different solution techniques. A better understanding of these issues will facilitate further progress in the field and help resolve several open problems that we identify.

show abstract

Section: Motivationmentioning

confidence: 99%

Section: Approximate Dynamic Programmingmentioning

confidence: 99%

Section: Approximate Dynamic Programmingmentioning

confidence: 99%

“…In their paper, Cogill et al [11] prove two theorems which relate the approximation error between Q * andQ to the difference in the achieved value, i.e. the difference between J * and J .…”

Section: Approximate Dynamic Programming For Decentralized Problemsmentioning

confidence: 99%

See 2 more Smart Citations

Formal models and algorithms for decentralized decision making under uncertainty

Seuken

Zilberstein

2008

Auton Agent Multi-Agent Syst

138

104

View full text Add to dashboard Cite

show abstract

“…Many of the earlier applications were motivating, but not deployed, while some of the newer work has been deployed on various platforms. Applications include multi-robot coordination in the form of space exploration rovers [79], helicopter flights [18], foraging [24] and navigation [71], [80], [81], load balancing for decentralized queues [82], network congestion control [83], [84], network routing [85], wireless networking [61] as well as sensor networks for target tracking [21], [22] and weather phenomena [23]. There is also an application of Dec-POMDPs to a real-time strategy video game.…”

Section: Applications and Learningmentioning

confidence: 99%

Decentralized control of partially observable Markov decision processes

Amato¹,

Chowdhary

Geramifard

et al. 2013

52nd IEEE Conference on Decision and Control

View full text Add to dashboard Cite

Abstract-Markov decision processes (MDPs) are often used to model sequential decision problems involving uncertainty under the assumption of centralized control. However, many large, distributed systems do not permit centralized control due to communication limitations (such as cost, latency or corruption). This paper surveys recent work on decentralized control of MDPs in which control of each agent depends on a partial view of the world. We focus on a general framework where there may be uncertainty about the state of the environment, represented as a decentralized partially observable MDP (Dec-POMDP), but consider a number of subclasses with different assumptions about uncertainty and agent independence. In these models, a shared objective function is used, but plans of action must be based on a partial view of the environment. We describe the frameworks, along with the complexity of optimal control and important properties. We also provide an overview of exact and approximate solution methods as well as relevant applications. This survey provides an introduction to what has become an active area of research on these models and their solutions.

show abstract

Feature Selection for Neuro‐Dynamic Programming

Huang

Chen

Mehta

et al. 2012

Reinforcement Learning and Approximate Dynamic Programming for Feedback Control

View full text Add to dashboard Cite

Neuro-Dynamic Programming encompasses techniques from both reinforcement learning and approximate dynamic programming. Feature selection refers to the choice of basis that defines the function class that is required in the application of these techniques.This chapter reviews two popular approaches to neuro-dynamic programming, TDlearning and Q-learning. The main goal of the chapter is to demonstrate how insight from idealized models can be used as a guide for feature selection for these algorithms. Several approaches are surveyed, including fluid and diffusion models, and the application of idealized models arising from mean-field game approximations. The theory is illustrated with several examples.

show abstract

An Approximate Dynamic Programming Approach to Decentralized Control of Stochastic Systems

Cited by 34 publications

References 14 publications

Formal models and algorithms for decentralized decision making under uncertainty

Formal models and algorithms for decentralized decision making under uncertainty

Decentralized control of partially observable Markov decision processes

Feature Selection for Neuro‐Dynamic Programming

Contact Info

Product

Resources

About