An Improved DDPG and Its Application Based on the Double-Layer BP Neural Network

Zhang, Mingli; Zhang, Yijie; Gao, Zhengjie; He, Xiaolong

doi:10.1109/access.2020.3020590

Cited by 28 publications

(14 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to the salt cavern exploitation foundation and previous experience of underground gas storage in Jintan, salt caverns are selected to be the UHS formation in the proposed IHES. To simulate the storage process and injection/withdrawal cycles of UHS, the hydrodynamic model of UHS can be denoted by (31).…”

Section: Case Descriptionmentioning

confidence: 99%

“…Deep deterministic policy gradient (DDPG) is a kind of machine learning algorithm for optimisation problems with model‐free structures [30]. Based on the system response data, the controlled systems can achieve adaptive updating, and control strategies can be adaptively optimised [31]. DDPG has been successfully applied to achieve adaptively optimal control in various fields, including Internet‐of‐Things networks [32], energy harvesting [33], and hybrid electric bus [34].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Data‐driven optimal scheduling for underground space based integrated hydrogen energy system

Qin

Jiang

et al. 2022

IET Renewable Power Gen

View full text Add to dashboard Cite

Integrated hydrogen energy systems (IHESs) have attracted extensive attention in mitigating climate problems. As a kind of large‐scale hydrogen storage device, underground hydrogen storage (UHS) can be introduced into IHES to balance the seasonal energy mismatch, while bringing challenges to optimal operation of IHES due to the complex geological structure and uncertain hydrodynamics. To address this problem, a deep deterministic policy gradient (DDPG)‐based optimal scheduling method for underground space based IHES is proposed. The energy management problem is formulated as a Markov decision process to characterize the interaction between environmental states and policy. Based on DDPG theory, the actor‐critic structure is applied to approximate deterministic policy and actor‐value function. Through policy iteration and actor‐critic network training, the operation of UHS and other energy conversion devices can be adaptively optimised, which is driven by real‐time response data instead of accurate system models. Finally, the effectiveness of the proposed optimal scheduling method and the benefits of underground space are verified through time‐domain simulations.

show abstract

Section: Case Descriptionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Data‐driven optimal scheduling for underground space based integrated hydrogen energy system

Qin

Jiang

et al. 2022

IET Renewable Power Gen

View full text Add to dashboard Cite

show abstract

“…Although DQN has had a strong influence in great dimensional problems, the search space may remain low-key. In this case the Deep Deterministic Policy Gradients (DDPG) [13] off-policy algorithm may be used.…”

Section: Markovian Decision-making Process Modelingmentioning

confidence: 99%

Discrete Event Modeling and Simulation for Reinforcement Learning System Design

Capocchi¹,

Santucci²

2022

Information

View full text Add to dashboard Cite

Discrete event modeling and simulation and reinforcement learning are two frameworks suited for cyberphysical system design, which, when combined, can give powerful tools for system optimization or decision making process for example. This paper describes how discrete event modeling and simulation could be integrated into reinforcement learning concepts and tools in order to assist in the realization of reinforcement learning systems, more specially considering the temporal, hierarchical, and multi-agent aspects. An overview of these different improvements are given based on the implementation of the Q-Learning reinforcement learning algorithm in the framework of the Discrete Event system Specification (DEVS) and System Entity Structure (SES) formalisms.

show abstract

“…Mnih et al [7] proposed the concept of two-layer BP neural network and hence improved the DDPG algorithm. Te search efciency of BP network was improved by using Armijo-Goldstein-based criterion and BFGS method [8]. Nikishin et al [9] reduced the infuence of noise on the gradient by averaging methods under the premise of random weights.…”

Section: Introductionmentioning

confidence: 99%

Network Architecture for Optimizing Deep Deterministic Policy Gradient Algorithms

Zhang¹,

Zhang

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The traditional Deep Deterministic Policy Gradient (DDPG) algorithm has been widely used in continuous action spaces, but it still suffers from the problems of easily falling into local optima and large error fluctuations. Aiming at these deficiencies, this paper proposes a dual-actor-dual-critic DDPG algorithm (DN-DDPG). First, on the basis of the original actor-critic network architecture of the algorithm, a critic network is added to assist the training, and the smallest Q value of the two critic networks is taken as the estimated value of the action in each update. Reduce the probability of local optimal phenomenon; then, introduce the idea of dual-actor network to alleviate the underestimation of value generated by dual-evaluator network, and select the action with the greatest value in the two-actor networks to update to stabilize the training of the algorithm process. Finally, the improved method is validated on four continuous action tasks provided by MuJoCo, and the results show that the improved method can reduce the fluctuation range of error and improve the cumulative return compared with the classical algorithm.

show abstract

An Improved DDPG and Its Application Based on the Double-Layer BP Neural Network

Cited by 28 publications

References 19 publications

Data‐driven optimal scheduling for underground space based integrated hydrogen energy system

Data‐driven optimal scheduling for underground space based integrated hydrogen energy system

Discrete Event Modeling and Simulation for Reinforcement Learning System Design

Network Architecture for Optimizing Deep Deterministic Policy Gradient Algorithms

Contact Info

Product

Resources

About