Deep Reinforcement Learning-Based Energy Storage Arbitrage With Accurate Lithium-Ion Battery Degradation Model

Cao, Jun; Harrold, Dan; Fan, Zhong; Morstyn, Thomas; Healey, David; Li, Kang

doi:10.1109/tsg.2020.2986333

Cited by 215 publications

(104 citation statements)

References 23 publications

(36 reference statements)

Supporting

Mentioning

104

Contrasting

Order By: Relevance

“…Hence, we remove the dependence on the model-based EKF approach. The authors of [29] also highlighted the effectiveness of reinforcement learning in designing an optimal control policy to reduce transmission losses.…”

Section: Related Workmentioning

confidence: 99%

Learning to Calibrate Battery Models in Real-Time with Deep Reinforcement Learning

et al. 2021

View full text Add to dashboard Cite

Lithium-ion (Li-I) batteries have recently become pervasive and are used in many physical assets. For the effective management of the batteries, reliable predictions of the end-of-discharge (EOD) and end-of-life (EOL) are essential. Many detailed electrochemical models have been developed for the batteries. Their parameters are calibrated before they are taken into operation and are typically not re-calibrated during operation. However, the degradation of batteries increases the reality gap between the computational models and the physical systems and leads to inaccurate predictions of EOD/EOL. The current calibration approaches are either computationally expensive (model-based calibration) or require large amounts of ground truth data for degradation parameters (supervised data-driven calibration). This is often infeasible for many practical applications. In this paper, we introduce a reinforcement learning-based framework for reliably inferring calibration parameters of battery models in real time. Most importantly, the proposed methodology does not need any labeled data samples of observations and the ground truth parameters. The experimental results demonstrate that our framework is capable of inferring the model parameters in real time with better accuracy compared to approaches based on unscented Kalman filters. Furthermore, our results show better generalizability than supervised learning approaches even though our methodology does not rely on ground truth information during training.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning to Calibrate Battery Models in Real-Time with Deep Reinforcement Learning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…al. [28], is to add future prices to the state. Thus, for the FQI-based controller no additional adaptations are needed to the state, but every training day the rewards in F need to be recalculated.…”

Section: B Belpex Pricementioning

confidence: 99%

Domain Randomization for Demand Response of an Electric Water Heater

Peirelinck

Hermans

Spiessens

2021

IEEE Trans. Smart Grid

View full text Add to dashboard Cite

Thermostatically Controlled Loads (TCLs) provide a source of demand flexibility, and are often considered a good source for Demand Response (DR) applications. Due to their heterogeneity, and as such a lack of dynamics models, Reinforcement Learning (RL) is often used to exploit this flexibility. Unfortunately, RL requires exploratory interaction with the TCL, resulting in a period of potential discomfort for the users. We present an approach to reduce this exploratory time by pretraining the RL-agent. Domain randomization is used to facilitate knowledge transfer. We evaluate the pre-training potential in a DR energy arbitrage scenario with an Electric Water Heater (EWH). Our experiments show that a priori knowledge about EWH dynamics can be used to initialize and improve the control policy. In our experiments, pre-training attributes to 8.8 % additional cost savings, compared to starting from scratch.

show abstract

“…Model-free RL ConvLSTM+DQN 2020 [142] Energy storage arbitrage A battery energy arbitrage is modelled to learn optimal control policy for storage charging/discharging strategy via a battery degradation mode ConvLSTM for price prediction DDQN 2020 [143] Battery management The optimal operation of the community battery is controlled by the double deep Q-learning method.…”

Section: Milpmentioning

confidence: 99%

AI-powered Energy Internet Towards Carbon Neutrality: Challenges and Opportunities

Li¹

2021

Preprint

View full text Add to dashboard Cite

From self-driving vehicles, voice recognition based virtual digital assistants, smart thermostats to recommendation systems, Artificial Intelligence (AI) is becoming a crucial part of the carbon neutral society that has drawn considerable interest from energy supply firms, startups, technology developers, financial institutions, national governments and the academic community. The emergence of AI initiates numerous opportunities to transform energy industry to AI-powered smart system which can revolutionize traditional approaches of creativity thinking, strategical operation, and solution seeking, especially for accelerating carbon neutrality of our society. This survey provides a comprehensive overview of fundamental principles that underpin applications of big data analysis in Energy Internet (EI), such as smart energy supply and consumption, smart health and Fintech. Next, we focus on intelligent decision-making for the energy industry and inform the state-of-the-art by thoroughly reviewing the literature. Subsequently, cybersecurity issues for AI system related to EI are discussed with recent advancements from vulnerability analysis of AI system to differential privacy and to blockchain based security technology. To our knowledge, this is one of the first academic, peer-reviewed works to provide a systematic review of AI applications for EI research and initiatives in terms of big data analysis, intelligent decision-making and AI related cybersecurity These initiatives were systematically classified into different groups according to the field of application, methodology and contribution Afterwards, potential challenges, limitations for existing research and opportunities for future directions are discussed, ranging from emerging explainable AI, to localized multi-energy marketplaces, self-driving electric vehicle charging and e-mobility. This paper can help us understand how to build smart cities and critical infrastructure for a climate-changed world towards the UN’s sustainable development goals.

show abstract

Deep Reinforcement Learning-Based Energy Storage Arbitrage With Accurate Lithium-Ion Battery Degradation Model

Cited by 215 publications

References 23 publications

Learning to Calibrate Battery Models in Real-Time with Deep Reinforcement Learning

Learning to Calibrate Battery Models in Real-Time with Deep Reinforcement Learning

Domain Randomization for Demand Response of an Electric Water Heater

AI-powered Energy Internet Towards Carbon Neutrality: Challenges and Opportunities

Contact Info

Product

Resources

About