Scheduling Fast Local Rule-Based Controllers for Optimal Operation of Energy Storage

Hoog, Julian de; Abdulla, Khalid; Kolluri, Ramachandra Rao; Karki, Paras

doi:10.1145/3208903.3208917

Cited by 12 publications

(5 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One approach to solving the optimal scheduling of energy storage problem is to set it up as a standard optimisation problem [5]. Optimisation problems can be solved using linear programming, quadratic programming or mixed-integer linear programming (MILP).…”

Section: Optimisation Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Optimizing a domestic battery and solar photovoltaic system with deep reinforcement learning

Kell¹,

McGough²,

Forshaw³

2021

Preprint

View full text Add to dashboard Cite

A lowering in the cost of batteries and solar PV systems has led to a high uptake of solar battery home systems. In this work, we use the deep deterministic policy gradient algorithm to optimise the charging and discharging behaviour of a battery within such a system. Our approach outputs a continuous action space when it charges and discharges the battery, and can function well in a stochastic environment. We show good performance of this algorithm by lowering the expenditure of a single household on electricity to almost $1AUD for large batteries across selected weeks within a year.

show abstract

Section: Optimisation Methodsmentioning

confidence: 99%

“…Another common approach is to calculate the charging schedule using stochastic dynamic programming. These methods allow non-linear models to be incorporated into the dynamic program's cost function [5]. However, dynamic programming models require a discretisation of the action space.…”

Section: Optimisation Methodsmentioning

confidence: 99%

Optimizing a domestic battery and solar photovoltaic system with deep reinforcement learning

Kell¹,

McGough²,

Forshaw³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In order to enable efficient optimal control towards such multiverse and dynamically changing frameworks, conventional Rule-Based Control techniques (RBC) are being traditionally deployed. However, Rule-Based Control (RBC) portrays an oversimplified control scheme that primarily utilizes a set of predetermined rules in order to generate control decisions [10][11][12][13][14][15]. To this end, even if RBC control schemes may be effective in certain situations, they are not particularly efficient for the optimization of energy systems: (a) RBC practices are not adequate to offer efficient control towards where complex system dynamics and changing environmental conditions emerge.…”

Section: Control Strategies For Energy Systemsmentioning

confidence: 99%

Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications

Vamvakas,

Michailidis,

Korkas

et al. 2023

Energies

View full text Add to dashboard Cite

With the rise in electricity, gas and oil prices and the persistently high levels of carbon emissions, there is an increasing demand for effective energy management in energy systems, including electrical grids. Recent literature exhibits large potential for optimizing the behavior of such systems towards energy performance, reducing peak loads and exploiting environmentally friendly ways for energy production. However, the primary challenge relies on the optimization of such systems, which introduces significant complexities since they present quite dynamic behavior. Such cyberphysical frameworks usually integrate multiple interconnected components such as power plants, transmission lines, distribution networks and various types of energy-storage systems, while the behavior of these components is affected by various external factors such as user individual requirements, weather conditions, energy demand and market prices. Consequently, traditional optimal control approaches—such as Rule-Based Control (RBC)—prove inadequate to deal with the diverse dynamics which define the behavior of such complicated frameworks. Moreover, even sophisticated techniques—such as Model Predictive Control (MPC)—showcase model-related limitations that hinder the applicability of an optimal control scheme. To this end, AI model-free techniques such as Reinforcement Learning (RL) offer a fruitful potential for embedding efficient optimal control in cases of energy systems. Recent studies present promising results in various fields of engineering, indicating that RL frameworks may prove the key element for delivering efficient optimal control in smart buildings, electric vehicle charging and smart grid applications. The current paper provides a comprehensive review of RL implementations in energy systems frameworks—such as Renewable Energy Sources (RESs), Building Energy-Management Systems (BEMSs) and Electric Vehicle Charging Stations (EVCSs)—illustrating the benefits and the opportunities of such approaches. The work examines more than 80 highly cited papers focusing on recent RL research applications—between 2015 and 2023—and analyzes the model-free RL potential as regards the energy systems’ control optimization in the future.

show abstract

“…According to different load scenarios, a rule-based online energy storage battery management approach is designed to deal with power emergencies [15]. A rule-based local controller to reduce computational complexity and meet the needs of rapid response was proposed [16]. An approach based on the Laypunov method to optimize the charging and discharging of energy storage batteries has been proposed to reduce electricity costs.…”

Section: Introductionmentioning

confidence: 99%

Improved Deep Q-Network for User-Side Battery Energy Storage Charging and Discharging Strategy in Industrial Parks

Chen

Jiang

et al. 2021

Entropy

View full text Add to dashboard Cite

Battery energy storage technology is an important part of the industrial parks to ensure the stable power supply, and its rough charging and discharging mode is difficult to meet the application requirements of energy saving, emission reduction, cost reduction, and efficiency increase. As a classic method of deep reinforcement learning, the deep Q-network is widely used to solve the problem of user-side battery energy storage charging and discharging. In some scenarios, its performance has reached the level of human expert. However, the updating of storage priority in experience memory often lags behind updating of Q-network parameters. In response to the need for lean management of battery charging and discharging, this paper proposes an improved deep Q-network to update the priority of sequence samples and the training performance of deep neural network, which reduces the cost of charging and discharging action and energy consumption in the park. The proposed method considers factors such as real-time electricity price, battery status, and time. The energy consumption state, charging and discharging behavior, reward function, and neural network structure are designed to meet the flexible scheduling of charging and discharging strategies, and can finally realize the optimization of battery energy storage benefits. The proposed method can solve the problem of priority update lag, and improve the utilization efficiency and learning performance of the experience pool samples. The paper selects electricity price data from the United States and some regions of China for simulation experiments. Experimental results show that compared with the traditional algorithm, the proposed approach can achieve better performance in both electricity price systems, thereby greatly reducing the cost of battery energy storage and providing a stronger guarantee for the safe and stable operation of battery energy storage systems in industrial parks.

show abstract

Scheduling Fast Local Rule-Based Controllers for Optimal Operation of Energy Storage

Cited by 12 publications

References 3 publications

Optimizing a domestic battery and solar photovoltaic system with deep reinforcement learning

Optimizing a domestic battery and solar photovoltaic system with deep reinforcement learning

Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications

Improved Deep Q-Network for User-Side Battery Energy Storage Charging and Discharging Strategy in Industrial Parks

Contact Info

Product

Resources

About