Finite-Horizon Optimal State-Feedback Control of Nonlinear Stochastic Systems Based on a Minimum Principle

Deisenroth, Marc Peter; Ohtsuka, Toshiyuki; Weissel, Florian; Brunn, Dietrich; Hanebeck, Uwe D.

doi:10.1109/mfi.2006.265616

Cited by 6 publications

(8 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This obviously leads to insufficient solutions especially for highly nonlinear systems and reward functions. In [5], an extension of the deterministic reward function by a term considering the noise is presented. In [12], an approach for infinite horizon optimal control is presented, where a continuous state space is discretized by means of a radial-basis-function network.…”

Section: {Weissel|marcohuber|uwehanebeck}@ieeeorgmentioning

confidence: 99%

Efficient Control of Nonlinear Noise-Corrupted Systems Using a Novel Model Predictive Control Framework

Weissel

Huber

Hanebeck

2007

2007 American Control Conference

Self Cite

View full text Add to dashboard Cite

Abstract-Model identification and measurement acquisition is always to some degree uncertain. Therefore, a framework for Nonlinear Model Predictive Control (NMPC) is proposed that explicitly considers the noise influence on nonlinear dynamic systems with continuous state spaces and a finite set of control inputs in order to significantly increase the control quality. Integral parts of NMPC are the prediction of system states over a finite horizon as well as the problem specific modeling of reward functions. For achieving an efficient and also accurate state prediction, the introduced framework uses transition densities approximated by means of axis-aligned Gaussian mixtures. The representation power of Gaussian mixtures is also used to model versatile reward functions. Thus, together with the prediction technique a closed-form calculation of the optimization problems arising from NMPC is possible. Additionally, an efficient algorithm for calculating an approximate value function of the corresponding optimal control problem employing dynamic programming is presented. Thus, the value function can be calculated off-line, which reduces the on-line computational burden significantly and also permits the use of long optimization horizons. The capabilities of the framework and especially the benefits that can be gained by incorporating the noise in the controller are illustrated by the example of a two-wheeled differential-drive mobile robot following a given path.

show abstract

Section: {Weissel|marcohuber|uwehanebeck}@ieeeorgmentioning

confidence: 99%

Efficient Control of Nonlinear Noise-Corrupted Systems Using a Novel Model Predictive Control Framework

Weissel

Huber

Hanebeck

2007

2007 American Control Conference

Self Cite

View full text Add to dashboard Cite

show abstract

“…is employed. Using this result, an initial solution to the original optimal control problem for a finite decision-making horizon is found as described in [15]. There, the value function (5) of the stochastic dynamic programming algorithm is approximated by means of Taylor series expansion up to second order to simplify the problem.…”

Section: A Initial Solutionmentioning

confidence: 99%

“…The arising nonlinear equations are solved numerically by means of a continuation method [16]. In [15] the continuation consists of transforming an initial linear system into the original nonlinear system, while the solution to the corresponding (non)linear equation system is being traced. This procedure yields a good candidate for the sequence of optimal state feedbacks of the simplified problem.…”

Section: A Initial Solutionmentioning

confidence: 99%

“…A common assumption in model-based control is the twice differentiability of the value function J k in the Bellman equation 5in the dynamic programming algorithm. For instance, higher-order Taylor series approximation of J k requires at least second-order derivatives [6], [15]. To maintain this property, interpolating polynomials of at least third degree are required.…”

Section: Value Function Approximationmentioning

confidence: 99%

See 1 more Smart Citation

Online-computation approach to optimal control of noise-affected nonlinear systems with continuous state and control spaces

Deisenroth

Weissel

Ohtsuka

et al. 2007

2007 European Control Conference (ECC)

Self Cite

View full text Add to dashboard Cite

A novel online-computation approach to optimal control of nonlinear, noise-affected systems with continuous state and control spaces is presented. In the proposed algorithm, system noise is explicitly incorporated into the control decision. This leads to superior results compared to state-of-the-art nonlinear controllers that neglect this influence. The solution of an optimal nonlinear controller for a corresponding deterministic system is employed to find a meaningful state space restriction. This restriction is obtained by means of approximate state prediction using the noisy system equation. Within this constrained state space, an optimal closed-loop solution for a finite decisionmaking horizon (prediction horizon) is determined within an adaptively restricted optimization space. Interleaving stochastic dynamic programming and value function approximation yields a solution to the considered optimal control problem. The enhanced performance of the proposed discrete-time controller is illustrated by means of a scalar example system. Nonlinear model predictive control is applied to address approximate treatment of infinite-horizon problems by the finite-horizon controller.

show abstract

“…In (Deisenroth et al, 2006) an extension of the deterministic cost function by a term considering the noise is presented. In (Nikovski and Brand, 2003) an approach for infinite horizon optimal control is presented, where a continuous state space is discretized by means of a radialbasis-function network.…”

Section: Introductionmentioning

confidence: 99%

A Closed-Form Model Predictive Control Framework for Nonlinear Noise-Corrupted Systems

Weissel¹,

Huber²,

Hanebeck³

2007

Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics

View full text Add to dashboard Cite

Abstract:In this paper, a framework for Nonlinear Model Predictive Control (NMPC) that explicitly incorporates the noise influence on systems with continuous state spaces is introduced. By the incorporation of noise, which results from uncertainties during model identification and the measurement process, the quality of control can be significantly increased. Since NMPC requires the prediction of system states over a certain horizon, an efficient state prediction technique for nonlinear noise-affected systems is required. This is achieved by using transition densities approximated by axis-aligned Gaussian mixtures together with methods to reduce the computational burden. A versatile cost function representation also employing Gaussian mixtures provides an increased freedom of modeling. Combining the prediction technique with this value function representation allows closed-form calculation of the necessary optimization problems arising from NMPC. The capabilities of the framework and especially the benefits that can be gained by considering the noise in the controller are illustrated by the example of a mobile robot following a given path.

show abstract

Finite-Horizon Optimal State-Feedback Control of Nonlinear Stochastic Systems Based on a Minimum Principle

Cited by 6 publications

References 5 publications

Efficient Control of Nonlinear Noise-Corrupted Systems Using a Novel Model Predictive Control Framework

Efficient Control of Nonlinear Noise-Corrupted Systems Using a Novel Model Predictive Control Framework

Online-computation approach to optimal control of noise-affected nonlinear systems with continuous state and control spaces

A Closed-Form Model Predictive Control Framework for Nonlinear Noise-Corrupted Systems

Contact Info

Product

Resources

About