Probabilistic DHP adaptive critic for nonlinear stochastic control systems

Abstract

Following the recently developed algorithms for fully probabilistic control design for general dynamic stochastic systems [15], [18], this paper presents the solution to the probabilistic dual heuristic programming (DHP) adaptive critic method [15] and randomized control algorithm for stochastic nonlinear dynamical systems. The purpose of the randomized control input design is to make the joint probability density function of the closed loop system as close as possible to a predetermined ideal joint probability density function. This paper completes the previous work [15], [18] by formulating and solving the fully probabilistic control design problem on the more general case of nonlinear stochastic discrete time systems. A simulated example is used to demonstrate the use of the algorithm and encouraging results have been obtained. Index Termsnonlinear stochastic systems; fully probabilistic design; nonlinear randomized control input design; adaptive critic.

Fully Probabilistic Design for Stochastic Discrete System with Multiplicative Noise

“…Based on the Fully Probabilistic Design (FPD) [13]- [15], the control law c * (u k−1 |x k−1 ) which minimises the performance index (14) takes the following form,…”

“…The quadratic distance has been used in many papers to characterise the distance between the actual pdf and the desire pdf [10]- [12]. As many distributions can be expressed by an exponential function, Kullback-Leibler divergence (KLD) can be considered as a convenient scale to characterise the distance between two distributions [13]- [15]. Therefore, Fully Probabilistic Design (FPD) ( [13]- [16]), inspired by Bayesian approach to adaptive control design [17], which is proposed based on KLD, is applied in this paper.…”

“…However, the computational loads are quite heavy since the evaluation of the randomised optimal controller involves multivariate integration steps which need to be calculated by backward recursion. To solve that, [14], [18] provides a probabilistic Dual Heuristic Programming (DHP) adaptive critic method, which applies a critic network to circumvent the requirement for explicitly evaluating the optimal value function and enormously reduced computational requirements.…”

Generalised Probabilistic Control Design for Uncertain Stochastic Control Systems

“…The minimisation of the cost-to-go function (3) with respect to control law, c(u t | x t−1 ) is shown in [8,9] to be given by…”

Towards probabilistic synchronisation of local controllers

“…The pdf of optimal controller, c * (u t |x t ), minimizing the cost-to-go function (6) is determined by the following backward recursion, see e.g. Herzallah & Kárný (2011);Herzallah (2013)…”