Wide-sense adaptive dual control for nonlinear stochastic systems

Untitled

“…Open-loop feedback control and the present algorithm also share the unfortunate characteristic that the agent does not take account of the possibility of future use of the knowledge acquired. This possibility, which is at the heart of the tradeoff between exploration and exploitation, is incorporated into a more sophisticated and complicated method called wide-sense dual adaptive control (Tse, Bar-Shalom & Meier, 1973;) which also has a counterpart in the sort of problems we are considering here. We are currently testing an algorithm in which the agent knows that it is will perform repeated trials in the same maze, and knows that if it finds a transition to be open on one trial then it has the expectation that it will be able to use the transition in succeeding trials, which gives the agent an incentive to learn.…”

“…For example, consider the quadratic regulator problem of Tse, Bar-Shalom and Meier (1973) which has imperfect state information and a nonlinear transition function. Their system used a second order model that maintained only the mean and variance, which were updated using an extended Kalman filter in the light of information from the world.…”

Adaptive Control for Stochastic Systems with Unknown Parameters and Input Delay

“…The control problem is then reduced to finding the control sequence ( ), , ( ) u k u k N d  that minimize the performance index (7) subject to equation (5). Theoretically, the optimal solution of control could be obtained by solving a dynamic programming.…”

“…The first ones try to solve the Bellman equation by using some approximated solutions of dynamic programming [5] [6]. In the second class, the optimization problem is reformulated by modifying cost functions [7] [8] [9].…”

Receding Horizon-Based Dual Control Strategy for Pinpoint Planetary Landing

“…15,16) The use of standard certainty equivalence control techniques and various other approximation techniques is limited due to the inherent complexity of the problem, including the trade-off between efficient control and reliable estimation. [17][18][19][20][21] In this paper, a dual control algorithm considering such a trade-off is derived. To improve estimation accuracy, it incorporates the cost incurred by the system uncertainty into the performance index.…”