Stochastic Nonlinear Model Predictive Control with State Estimation by Incorporation of the Unscented Kalman Filter

Bradford, Eric; Imsland, Lars

doi:10.48550/arxiv.1709.01201

Cited by 3 publications

(4 citation statements)

References 17 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The parameters for all test cases are the cross-entropy results for the condition with process noise variance 0.01. The resulting mean values for the MCTS hyperparameters were µ = [22,5,12,27]. The planning horizon of MPC was selected to match the optimized MCTS depth of 12.…”

Section: Cross-entropy For Tuning Solver Parametersmentioning

confidence: 99%

Estimation and control using sampling‐based Bayesian reinforcement learning

Slade

Sunberg

Kochenderfer

2020

IET cyber-phys. syst.

View full text Add to dashboard Cite

Real-world autonomous systems operate under uncertainty about both their pose and dynamics. Autonomous control systems must simultaneously perform estimation and control tasks to maintain robustness to changing dynamics or modeling errors. However, information gathering actions often conflict with optimal actions for reaching control objectives, requiring a trade-off between exploration and exploitation. The specific problem setting considered here is for discrete-time nonlinear systems, with process noise, input-constraints, and parameter uncertainty. This article frames this problem as a Bayes-adaptive Markov decision process and solves it online using Monte Carlo tree search with an unscented Kalman filter to account for process noise and parameter uncertainty. This method is compared with certainty equivalent model predictive control and a tree search method that approximates the QMDP solution, providing insight into when information gathering is useful. Discrete time simulations characterize performance over a range of process noise and bounds on unknown parameters. An offline optimization method is used to select the Monte Carlo tree search parameters without hand-tuning. In lieu of recursive feasibility guarantees, a probabilistic bounding heuristic is offered that increases the probability of keeping the state within a desired region.

show abstract

Section: Cross-entropy For Tuning Solver Parametersmentioning

confidence: 99%

Estimation and control using sampling‐based Bayesian reinforcement learning

Slade

Sunberg

Kochenderfer

2020

IET cyber-phys. syst.

View full text Add to dashboard Cite

show abstract

“…Although many of the methods above focus on robustness, they do not incorporate uncertainty over the parameters of the transition function. In [5], this is accounted for by using a SNMPC with an Unscented Kalman Filter to propagate the uncertainty over the state-space. However, this method requires an optimisation with chance constraints to be solved online and, to keep the problem feasible, the variance of the trajectories has to be artificially constrained.…”

Section: Related Workmentioning

confidence: 99%

“…However, control tasks in reinforcement learning are usually more complex and therefore less suitable to linearisation, motivating the use of non-linear models [4]. Another motivation for more complex models is the ability to use of more expressive constraints, even if not directly involved in the physical process, such as economic criteria [5]. Despite its vast application in the linear case, the use of MPC in non-linear systems continues to be an increasingly active area of research in control theory [2], [3].…”

Section: Introductionmentioning

confidence: 99%

DISCO: Double Likelihood-free Inference Stochastic Control

Barcelos¹,

Oliveira²,

Possas³

et al. 2020

Preprint

View full text Add to dashboard Cite

Accurate simulation of complex physical systems enables the development, testing, and certification of control strategies before they are deployed into the real systems. As simulators become more advanced, the analytical tractability of the differential equations and associated numerical solvers incorporated in the simulations diminishes, making them difficult to analyse. A potential solution is the use of probabilistic inference to assess the uncertainty of the simulation parameters given real observations of the system. Unfortunately the likelihood function required for inference is generally expensive to compute or totally intractable. In this paper we propose to leverage the power of modern simulators and recent techniques in Bayesian statistics for likelihood-free inference to design a control framework that is efficient and robust with respect to the uncertainty over simulation parameters. The posterior distribution over simulation parameters is propagated through a potentially non-analytical model of the system with the unscented transform, and a variant of the information theoretical model predictive control. This approach provides a more efficient way to evaluate trajectory roll outs than Monte Carlo sampling, reducing the online computation burden. Experiments show that the controller proposed attained superior performance and robustness on classical control and robotics tasks when compared to models not accounting for the uncertainty over model parameters.

show abstract

“…A disadvantage of the PCE-based SNMPC is that the computational cost scales exponentially with the number of uncertain parameters. 23,39 To address these concerns, two tractable approximations to the SNMPC problems with the incorporation of unscented Kalman filter (UKF) 40 and Gaussian process (GP) 40 were proposed. Taking advantage of the UKF which estimates and propagates the entire conditional distribution of the states over the prediction horizon by taking into account state estimation errors, the UKF-SNMPC reformulates the model cost and constraint functions by using the mean and covariance information, which yields a tractable control framework for handling nonlinear stochastic dynamic optimization (SDO) problems.…”

Section: Introductionmentioning

confidence: 99%

A model‐and data‐driven predictive control approach for tracking of stochastic nonlinear systems using Gaussian processes

2023

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

Nonlinear model predictive control (NMPC) is one of the few control methods that can handle complex nonlinear systems with multi‐objectives and various constraints. However, the performance of NMPC highly depends on the model accuracy and the deterministic solutions may suffer from conservatism, for example, robust MPC only considers the worst‐case scenario, which yields the NMPC not working efficiently in uncertain stochastic cases. To address these issues, a model‐and data‐driven predictive control approach using Gaussian processes (GP‐MDPC) is synthesized in this paper, which copes with the tracking control problems of stochastic nonlinear systems subject to model uncertainties and chance constraints. Because GP has high flexibility to capture complex unknown functions and it inherently handles measurement noise, GP models are employed to approximate the unknowns, the predictions and uncertainty quantification provided by the GPs are then exploited to propagate the uncertainties through the nonlinear model and to formulate a finite‐horizon stochastic optimal control problem (FH‐SOCP). Specifically, given the GP models, closed‐loop simulations are executed offline to generate Monte Carlo samples, from which the back‐offs for constraint tightening are calculated iteratively. The tightened constraints then guarantee the satisfaction of chance constraints online. A tractable GP‐MDPC framework using back‐offs for handling nonlinear chance constrained tracking control problems is yielded, whose advantages include fast online evaluation, consideration of closed‐loop behaviour, and achievable trade‐off between conservatism and constraint violation. Comparisons are carried out to verify the effectiveness and superiority of the proposed GP‐MDPC scheme with back‐offs.

show abstract

Stochastic Nonlinear Model Predictive Control with State Estimation by Incorporation of the Unscented Kalman Filter

Cited by 3 publications

References 17 publications

Estimation and control using sampling‐based Bayesian reinforcement learning

Estimation and control using sampling‐based Bayesian reinforcement learning

DISCO: Double Likelihood-free Inference Stochastic Control

A model‐and data‐driven predictive control approach for tracking of stochastic nonlinear systems using Gaussian processes

Contact Info

Product

Resources

About