Discounted near-optimal control of general continuous-action nonlinear systems using optimistic planning

“…A box is refined into 3 subboxes by splitting the interval of some dimension k into 3 equal-length pieces, see Figure 1, left. Buşoniu et al (2016b). )…”

Section: Background On Opcmentioning

confidence: 99%

“…We describe in Section 4 the impact of these assumptions in PWA systems, see Buşoniu et al (2016b) for further discussion.…”

Section: Background On Opcmentioning

confidence: 99%

“…Optimistic planning for continuous actions (OPC) (Buşoniu et al, 2016b) is geared towards general nonlinear systems and maximization of cumulative rewards, so we present it in that setting first. Later, we explain how it can be adapted to solve the problem in Section 2.…”

Section: Background On Opcmentioning

confidence: 99%

“…Optimistic methods have their roots in reinforcement learning (Sutton and Barto, 1998) and bandit theory (Auer et al, 2002). In particular, we use our continuous-action optimistic planner (OPC) from (Buşoniu et al, 2016b), since it is the most suitable for PWA systems. As for the entire optimistic family, the performance guarantees of OPC place computation in a tight relationship with the near-optimality of the returned sequence.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Near-optimal control with adaptive receding horizon for discrete-time piecewise affine systems

Buşoniu

Schutter

2017

IFAC-PapersOnLine

View full text Add to dashboard Cite

Abstract:We consider the infinite-horizon optimal control of discrete-time, Lipschitz continuous piecewise affine systems with a single input. Stage costs are discounted, bounded, and use a 1 or ∞-norm. Rather than using the usual fixed-horizon approach from model-predictive control, we tailor an adaptive-horizon method called optimistic planning for continuous actions (OPC) to solve the piecewise affine control problem in receding horizon. The main advantage is the ability to solve problems requiring arbitrarily long horizons. Furthermore, we introduce a novel extension that provides guarantees on the closed-loop performance, by reusing data ("learning") across different steps. This extension is general and works for a large class of nonlinear dynamics. In experiments with piecewise affine systems, OPC improves performance compared to a fixedhorizon approach, while the data-reuse approach yields further improvements.

show abstract