Contribution to nonserial dynamic programming

Bertelè, Umberto; Brioschi, Francesco

doi:10.1016/0022-247x(69)90030-4

Cited by 34 publications

(34 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This technique reduces structured LPs with exponentially many constraints to equivalent, polynomially-sized ones. This decomposition follows a procedure analogous to variable elimination that applies both to additively structured value functions (Bertele & Brioschi, 1972) and to value functions that also exploit context-specific structure (Zhang & Poole, 1999). Using these basic operations, our planning algorithms can be implemented efficiently, even though the size of the state space grows exponentially in the number of variables.…”

Section: Factored Mdpsmentioning

confidence: 99%

“…We can maximize such a function, F w , without enumerating every state using non-serial dynamic programming (Bertele & Brioschi, 1972). The idea is virtually identical to variable elimination in a Bayesian network.…”

Section: Maximizing Over the State Spacementioning

confidence: 99%

“…Computing the optimal elimination order is an NP-hard problem (Arnborg, Corneil, & Proskurowski, 1987) and elimination orders yielding low induced tree width do not exist for some problems. These issues have been confronted successfully for a large variety of practical problems in the Bayesian network community, which has benefited from a large variety of good heuristics which have been developed for the variable elimination ordering problem (Bertele & Brioschi, 1972;Kjaerulff, 1990;Reed, 1992;Becker & Geiger, 2001). …”

Section: Example 42 Assumementioning

confidence: 99%

“…In Section 4.2.1, we showed that the maximization of a linear combination of table-based functions with restricted scope can be performed efficiently using non-serial dynamic programming (Bertele & Brioschi, 1972), or variable elimination. To exploit structure in rules, we use an algorithm similar to variable elimination in a Bayesian network with context-specific independence (Zhang & Poole, 1999).…”

Section: Rule-based Maximization Over the State Spacementioning

confidence: 99%

See 3 more Smart Citations

Efficient Solution Algorithms for Factored MDPs

Guestrin¹,

Koller²,

Parr³

et al. 2003

jair

280

268

View full text Add to dashboard Cite

This paper addresses the problem of planning under uncertainty in large Markov Decision Processes (MDPs). Factored MDPs represent a complex state space using state variables and the transition model using a dynamic Bayesian network. This representation often allows an exponential reduction in the representation size of structured MDPs, but the complexity of exact solution algorithms for such MDPs can grow exponentially in the representation size. In this paper, we present two approximate solution algorithms that exploit structure in factored MDPs. Both use an approximate value function represented as a linear combination of basis functions, where each basis function involves only a small subset of the domain variables. A key contribution of this paper is that it shows how the basic operations of both algorithms can be performed efficiently in closed form, by exploiting both additive and context-specific structure in a factored MDP. A central element of our algorithms is a novel linear program decomposition technique, analogous to variable elimination in Bayesian networks, which reduces an exponentially large LP to a provably equivalent, polynomial-sized one. One algorithm uses approximate linear programming, and the second approximate dynamic programming. Our dynamic programming algorithm is novel in that it uses an approximation based on max-norm, a technique that more directly minimizes the terms that appear in error bounds for approximate MDP algorithms. We provide experimental results on problems with over 10 40 states, demonstrating a promising indication of the scalability of our approach, and compare our algorithm to an existing state-of-the-art approach, showing, in some problems, exponential gains in computation time.

show abstract

Section: Factored Mdpsmentioning

confidence: 99%

Section: Maximizing Over the State Spacementioning

confidence: 99%

Section: Example 42 Assumementioning

confidence: 99%

Section: Rule-based Maximization Over the State Spacementioning

confidence: 99%

See 2 more Smart Citations

Efficient Solution Algorithms for Factored MDPs

Guestrin¹,

Koller²,

Parr³

et al. 2003

jair

280

268

View full text Add to dashboard Cite

show abstract

“…The solution of the secondary optimization problem nécessitâtes graph theoretical considérations. The works in this field up to now are [1 5 2,3,4]. This paper introduces parametrization in nonserial dynamic programming.…”

Section: Introductionmentioning

confidence: 99%