Reinforcement Learning-Based Microgrid Energy Trading With a Reduced Power Plant Schedule

Lu, Xiaozhen; Xiao, Xingyu; Xiao, Liang; Dai, Canhuang; Peng, Mugen; Poor, H. Vincent

doi:10.1109/jiot.2019.2941498

Cited by 91 publications

(37 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The agent manages the activities of the storage devices with a goal to maximize demand side cost savings. Other research in this direction are presented in [79], [80]. However, from the perspective of efficiency of the grid operations, Ren et al focused on the use of RL for load balancing in smart grids [81].…”

Section: E Reinforcement Learning Applications In Cpsmentioning

confidence: 99%

Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS

Olowononi

Rawat

Liu

2021

IEEE Commun. Surv. Tutorials

143

View full text Add to dashboard Cite

Cyber Physical Systems (CPS) are characterized by their ability to integrate the physical and information or cyber worlds. Their deployment in critical infrastructure have demonstrated a potential to transform the world. However, harnessing this potential is limited by their critical nature and the far reaching effects of cyber attacks on human, infrastructure and the environment. An attraction for cyber concerns in CPS rises from the process of sending information from sensors to actuators over the wireless communication medium, thereby widening the attack surface. Traditionally, CPS security has been investigated from the perspective of preventing intruders from gaining access to the system using cryptography and other access control techniques. Most research work have therefore focused on the detection of attacks in CPS. However, in a world of increasing adversaries, it is becoming more difficult to totally prevent CPS from adversarial attacks, hence the need to focus on making CPS resilient. Resilient CPS are designed to withstand disruptions and remain functional despite the operation of adversaries. One of the dominant methodologies explored for building resilient CPS is dependent on machine learning (ML) algorithms. However, rising from recent research in adversarial ML, we posit that ML algorithms for securing CPS must themselves be resilient. This paper is therefore aimed at comprehensively surveying the interactions between resilient CPS using ML and resilient ML when applied in CPS. The paper concludes with a number of research trends and promising future research directions. Furthermore, with this paper, readers can have a thorough understanding of recent advances on ML-based security and securing ML for CPS and countermeasures, as well as research trends in this active research area.

show abstract

Section: E Reinforcement Learning Applications In Cpsmentioning

confidence: 99%

Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS

Olowononi

Rawat

Liu

2021

IEEE Commun. Surv. Tutorials

143

View full text Add to dashboard Cite

show abstract

“…For a battery behind meter systems, the essential variables to solve the problem include the DG power output, the load profile, the measured battery storage system (BSS) instantaneous state of charge (SoC) and the forecasted day-ahead grid tariff profile. The costs considered include the grid power purchase cost, the cost of degradation of the BSS as defined in [28], [29], [39] and the cost of power purchase from auxiliary sources such as vehicleto-microgrid (V2M) [6], [40], [41]. In some cases, the grid tariff is constant like in [41] and in others, a stochastic tariff is considered as in [42], [43].…”

Section: Mathematical Formulationsmentioning

confidence: 99%

“…The authors used convolutional neural networks to learn a general policy for scheduling the storage under unpredictable demand and generation environment. Lu et al, [29] used the DQN strategy for energy trading between a microgrid and a power plant and achieved a 22. 3% improvement in self-consumption of the MG generated power.…”

Section: E Deep Q-networkmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement Learning Techniques for Optimal Power Control in Grid-Connected Microgrids: A Comprehensive Review

Arwa

Folly

2020

IEEE Access

View full text Add to dashboard Cite

The utility grids are undergoing several upgrades. Distributed generators that are supplied by intermittent renewable energy sources (RES) are being connected to the grids. As RES generations get cheaper, more customers are opting for peer-to-peer energy interchanges through the smart metering infrastructure. Consequently, power management in grid-tied RES-based microgrids has become a challenging task. This paper reviews the applications of reinforcement learning (RL) algorithms in managing power in grid-tide microgrids. Unlike other optimization methods such as numerical and soft computing techniques, RL does not require an accurate model of the optimization environment in order to arrive at an optimal solution. In this paper, various challenges associated with the control of power in grid-tied microgrids are described. The application of RL techniques in addressing those challenges is reviewed critically. This review identifies the need to improve and scale multi-agent RL methods to enable seamless distributed power dispatch among interconnected microgrids. Finally, the paper gives directions for future research, e.g., the hybridization of intrinsic and extrinsic reward schemes, the use of transfer learning to improve the learning outcomes of RL in complex power systems environments and the deployment of priority-based experience replay in post-disaster microgrid power flow control. INDEX TERMS Electric vehicle charging station, energy management, Markov Decision Process, microgrid, reinforcement learning.

show abstract

“…Deep reinforcement learning is used in [29] to reach the policy that provides the commerce of energy between microgrids and attained 12.7% reduction in power plant scheduled for the proposed system model. The state is chosen to describe the current battery level, the predicted production of renewable energy, and the forecasted demand.…”

Section: Related Workmentioning

confidence: 99%

MARLA-SG: Multi-Agent Reinforcement Learning Algorithm for Efficient Demand Response in Smart Grid

et al. 2020

View full text Add to dashboard Cite

The population is sharply growing in the last decade, resulting in non-potential power requests in dense urban areas, especially with the traditional power grid where the system is not compatible with the infrequent changes. Smart grids have shown strong potential to effectively mitigate and smooth power consumption curves to avoid shortages by adjusting and forecasting the cost function in real-time in response to consumption fluctuations to achieve the desired objectives. The main challenge for the smart grid designers is to reduce the cost and Peak to Average Ratio (PAR) while maintaining the desired satisfaction level. This paper presents the development and evaluation of a Multi-Agent Reinforcement Learning Algorithm for efficient demand response in Smart Grid (MARLA-SG). Also, it shows a simple and flexible way of choosing state elements to reduce the possible number of states, regardless of the device type, range of operation, and maximum allowable delay. It also produces a simple way to represent the reward function regardless of the used cost function. SARSA (State-Action-Reward-State-Action) and Qlearning schemes are used and attained PAR reduction of 9.6%, 12.16%, and an average cost reduction of 10.2%, 7.8%, respectively.

show abstract

Reinforcement Learning-Based Microgrid Energy Trading With a Reduced Power Plant Schedule

Cited by 91 publications

References 28 publications

Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS

Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS

Reinforcement Learning Techniques for Optimal Power Control in Grid-Connected Microgrids: A Comprehensive Review

MARLA-SG: Multi-Agent Reinforcement Learning Algorithm for Efficient Demand Response in Smart Grid

Contact Info

Product

Resources

About