Adaptive dynamic programming

Murray, J.; Cox, C.; Lendaris, G.G.; Saeks, R.

doi:10.1109/tsmcc.2002.801727

Cited by 589 publications

(253 citation statements)

References 17 publications

Supporting

Mentioning

251

Contrasting

Unclassified

Order By: Relevance

“…Adaptive Critic Designs (ACDs), developed by Werbos, perform optimization based on approximate dynamic programming using neural networks [17][18]. The ACDs series includes Heuristic Dynamic Programming (HDP), Dual Heuristic Programming (DHP) and Globalized Dual Heuristic Programming (GDHP) [19].…”

Section: Adhdp Algorithmmentioning

confidence: 99%

Design of an Approximate Dynamic Programming based neural controller for Smart Home Energy Management

Dasht-e-Bayaz¹,

Moghaddam²,

Asgharian³

2017

Communications on Advanced Computational Science with Applicati

View full text Add to dashboard Cite

Demand Side Management (DSM) is the control of consumer demand for energy via different techniques such as financial incentives. This technology has become inevitable in the new smart grid infrastructure. In this study, a DSM scheme, a novel smart home energy management system, is proposed. The goal, defined in terms of cost, is to manage the home energy system according to time-varying prices in a way that energy demand from grid is reduced as much as possible or it is moved to off-peak times. The proposed scheme takes advantage of local energy generation, energy storage unit and schedulable load. Our offline scheme uses an Adaptive Dynamic Programming (ADP) based algorithm to solve the energy management problem and optimally schedule the battery and load operations in a given time horizon. We also use PSO method to solve the mentioned problem. The results obtained by PSO are used as an element of comparison. Simulation results show that the ADP algorithm can reduce costs with respect to PSO due to better decision making ability.

show abstract

Section: Adhdp Algorithmmentioning

confidence: 99%

Design of an Approximate Dynamic Programming based neural controller for Smart Home Energy Management

Dasht-e-Bayaz¹,

Moghaddam²,

Asgharian³

2017

Communications on Advanced Computational Science with Applicati

View full text Add to dashboard Cite

show abstract

“…al. used trajectories to provide estimates of values of a set of initial states [19]. A number of efforts have been made to use collections of trajectories to represent policies [3,20,6,7,21,22,23,24,25,26,27].…”

Section: Related Workmentioning

confidence: 99%

Trajectory-Based Dynamic Programming

Atkeson

Liu

2013

Cognitive Systems Monographs

View full text Add to dashboard Cite

This paper reviews a variety of ways to use trajectory optimization to accelerate dynamic programming. Dynamic programming provides a way to design globally optimal control laws for nonlinear systems. However, the curse of dimensionality, the exponential dependence of space and computation resources needed on the dimensionality of the state and control, limits the application of dynamic programming in practice. We explore trajectory-based dynamic programming, which combines many local optimizations to accelerate the global optimization of dynamic programming.What is Dynamic Programming? Dynamic programming provides a way to find globally optimal control laws (policies), u = u(x), which give the appropriate action u for any state x [1,2]. Dynamic programming takes as input a one step cost (a.k.a. "reward" or "loss") function and the dynamics of the problem to be optimized. This paper focuses on offline planning of nonlinear control laws for control problems with continuous states and actions, deterministic time invariant discrete time dynamics x k+1 = f(x k , u k ), and a time invariant one step cost function L(x, u), so we use discrete time dynamic programming. We are focusing on steady state policies and thus an infinite time horizon.One approach to dynamic programming is to approximate the value function V (x) (the optimal total future cost from each state , u))) at sampled states x j until the value function estimates have converged. Typically the value function and control law are represented on a regular grid. Some type of interpolation is used to approximate these functions within each grid cell. If each dimension of the state and action is represented with a resolution R, and the dimensionality of the state is d x and that of the action is d u , the computational cost of the conventional approach is proportional to R d x × R d u and the memory cost is proportional to R d x . This is known as the Curse of Dimensionality [1].An example problem: We use one link pendulum swingup as an example problem to provide the reader with a visualizable example of a nonlinear control law and corresponding value function. In one link pendulum swingup a motor at the base of the pendulum swings a rigid arm from the downward stable equilibrium to the upright unstable equilibrium and balances the arm there (Fig. 1). What makes this challenging is that a one step cost function penalizes the amount of torque used and the deviation of the current position from the goal. The controller must try to minimize the total cost of the trajectory. The one step cost function for this example is a weighted sum of the squared position errors (θ: difference between current angle and the goal angle) and the squared torques τ: L(x, u) = 0.1θ 2 + τ 2 where 0.1 weights the position error relative to the torque penalty. There are no costs associated with the joint velocity. The uniform density link has a mass of 1kg, length of 1m, and width of 0.1m. Because the dynamics and cost function are time invariant, there is a steady state control law and...

show abstract

“…The convergence and optimality of ADP without time-varying parameter are proven [10][14] [15]. But ADP can not be applied directly to HEO as there are time-varying parameters within HEO.…”

Section: Introductionmentioning

confidence: 99%

A Stochastic Optimal Control Method of Home Energy Based on Adaptive Dynamic Programming

Liu¹,

Zhang²,

Chen³

2016

Proceedings of the 2016 7th International Conference on Mechatronics, Control and Materials (ICMCM 2016)

View full text Add to dashboard Cite

Abstract. Home energy optimization (HEO) is an infinite horizon optimization problem, only adaptive dynamic programming (ADP) can solve infinite horizon optimization problem. But due to the existence of time-varying parameters, ADP can not converge to the optimal and can not be applied directly to the optimal control of home energy. According to the periodic characteristics of the time-varying parameters, a stochastic optimal control method is proposed in the paper based on ADP. The influence of time-varying parameters in the iteration of ADP is eliminated by the proposed method. Simulation results show the performance of the proposed method.

show abstract

Adaptive dynamic programming

Cited by 589 publications

References 17 publications

Design of an Approximate Dynamic Programming based neural controller for Smart Home Energy Management

Design of an Approximate Dynamic Programming based neural controller for Smart Home Energy Management

Trajectory-Based Dynamic Programming

A Stochastic Optimal Control Method of Home Energy Based on Adaptive Dynamic Programming

Contact Info

Product

Resources

About