Smart Grid Optimization by Deep Reinforcement Learning over Discrete and Continuous Action Space

Sogabe, Tomah; Malla, Dinesh Bahadur; Takayama, Shota; Shin, Seiichi; Sakamoto, Katsuyoshi; Yamaguchi, Koichi; Singh, Thakur Praveen; Sogabe, Masaru; Hirata, Toshiki; Okada, Yoshitaka

doi:10.1109/pvsc.2018.8547862

Cited by 21 publications

(12 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The popularity of deep learning has been considerably increased over recent years leading many studies to focus on finding solutions for energy-based problems with the use of deep architectures of common ML methods [30]. Specifically, deep reinforcement learning has been employed for simulating energy savings and demand response in buildings [31], as well as in the optimization of the energy demand and supply of smart grids [32]. Deep artificial neural networks have been employed on load prediction of a district cooling system combined with physics-based TES modelling [33], whereas in [34] deep recurrent neural networks, that offer great performance in time-dependent problems, were used for the prediction of the heating demand in commercial buildings.…”

Section: Introductionmentioning

confidence: 99%

A Hybrid Bimodal LSTM Architecture for Cascading Thermal Energy Storage Modelling

et al. 2022

View full text Add to dashboard Cite

Modelling of thermal energy storage (TES) systems is a complex process that requires the development of sophisticated computational tools for numerical simulation and optimization. Until recently, most modelling approaches relied on analytical methods based on equations of the physical processes that govern TES systems’ operations, producing high-accuracy and interpretable results. The present study tackles the problem of modelling the temperature dynamics of a TES plant by exploring the advantages and limitations of an alternative data-driven approach. A hybrid bimodal LSTM (H2M-LSTM) architecture is proposed to model the temperature dynamics of different TES components, by utilizing multiple temperature readings in both forward and bidirectional fashion for fine-tuning the predictions. Initially, a selection of methods was employed to model the temperature dynamics of individual components of the TES system. Subsequently, a novel cascading modelling framework was realised to provide an integrated holistic modelling solution that takes into account the results of the individual modelling components. The cascading framework was built in a hierarchical structure that considers the interrelationships between the integrated energy components leading to seamless modelling of whole operation as a single system. The performance of the proposed H2M-LSTM was compared against a variety of well-known machine learning algorithms through an extensive experimental analysis. The efficacy of the proposed energy framework was demonstrated in comparison to the modelling performance of the individual components, by utilizing three prediction performance indicators. The findings of the present study offer: (i) insights on the low-error performance of tailor-made LSTM architectures fitting the TES modelling problem, (ii) deeper knowledge of the behaviour of integral energy frameworks operating in fine timescales and (iii) an alternative approach that enables the real-time or semi-real time deployment of TES modelling tools facilitating their use in real-world settings.

show abstract

Section: Introductionmentioning

confidence: 99%

A Hybrid Bimodal LSTM Architecture for Cascading Thermal Energy Storage Modelling

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In [ 18 ], the author compares DQN and deep deterministic actor-critic (DDAC) algorithms to show the importance of using continuous actions in this control problem. These algorithms have one shortcoming, that is, automatic driving is a continuous control problem, but these algorithms can only be applied to discrete problems, so the control quantity must be discretized, and this process will inevitably lead to the inability to deal with the surrounding environment dynamic factors in the system and imprecise control [ 19 , 20 , 21 ].…”

Section: Introductionmentioning

confidence: 99%

Lane Following Method Based on Improved DDPG Algorithm

Zhang

et al. 2021

Sensors

View full text Add to dashboard Cite

In an autonomous vehicle, the lane following algorithm is an important component, which is a basic function of autonomous driving. However, the existing lane following system has a few shortcomings: first, the control method it adopts requires an accurate system model, and different vehicles have different parameters, which needs a lot of parameter calibration work. The second is that it may fail on road sections where the lateral acceleration requirements of vehicles are large, such as large curves. Third, its decision-making system is defined based on rules, which has disadvantages: it is difficult to formulate; human subjective factors cannot guarantee objectivity; coverage is difficult to guarantee. In recent years, the deep deterministic policy gradient (DDPG) algorithm has been widely used in the field of autonomous driving due to its strong nonlinear fitting ability and generalization performance. However, the DDPG algorithm has overestimated state action values and large cumulative errors, low training efficiency and other issues. Therefore, this paper improves the DDPG algorithm based on the double critic networks and priority experience replay mechanism. Then this paper proposes a lane following method based on this algorithm. Experiment shows that the algorithm can achieve excellent following results under various road conditions.

show abstract

“…In recent years, several successful studies have been published on the use of advanced RL methods for optimal control of microgrids based on deep Q-networks (DQN) [55,56], Monte-Carlo tree search (MCTS) [57], deep policy gradient [58], batch RL [59], multi-agent RL [60], etc. Part of the research is devoted to comparing the effectiveness of the RL methods (capable of giving quick, but approximate solutions) with traditional optimization methods, for example, mixed-integer linear programming (MILP) [61,62].…”

Section: Introductionmentioning

confidence: 99%

Optimal Operation Control of PV-Biomass Gasifier-Diesel-Hybrid Systems Using Reinforcement Learning Techniques

et al. 2020

View full text Add to dashboard Cite

The importance of efficient utilization of biomass as renewable energy in terms of global warming and resource shortages are well known and documented. Biomass gasification is a promising power technology especially for decentralized energy systems. Decisive progress has been made in the gasification technologies development during the last decade. This paper deals with the control and optimization problems for an isolated microgrid combining the renewable energy sources (solar energy and biomass gasification) with a diesel power plant. The control problem of an isolated microgrid is formulated as a Markov decision process and we studied how reinforcement learning can be employed to address this problem to minimize the total system cost. The most economic microgrid configuration was found, and it uses biomass gasification units with an internal combustion engine operating both in single-fuel mode (producer gas) and in dual-fuel mode (diesel fuel and producer gas).

show abstract

Smart Grid Optimization by Deep Reinforcement Learning over Discrete and Continuous Action Space

Cited by 21 publications

References 5 publications

A Hybrid Bimodal LSTM Architecture for Cascading Thermal Energy Storage Modelling

A Hybrid Bimodal LSTM Architecture for Cascading Thermal Energy Storage Modelling

Lane Following Method Based on Improved DDPG Algorithm

Optimal Operation Control of PV-Biomass Gasifier-Diesel-Hybrid Systems Using Reinforcement Learning Techniques

Contact Info

Product

Resources

About