A Multi-Agent Reinforcement Learning Framework for Lithium-ion Battery Scheduling Problems

Abstract

This paper presents a reinforcement learning framework for solving battery scheduling problems in order to extend the lifetime of batteries used in electrical vehicles (EVs), cellular phones, and embedded systems. Battery pack lifetime has often been the limiting factor in many of today’s smart systems, from mobile devices and wireless sensor networks to EVs. Smart charge-discharge scheduling of battery packs is essential to obtain super linear gain of overall system lifetime, due to the recovery effect and nonlinearity in the battery characteristics. Additionally, smart scheduling has also been shown to be beneficial for optimizing the system’s thermal profile and minimizing chances of irreversible battery damage. The recent rapidly-growing community and development infrastructure have added deep reinforcement learning (DRL) to the available tools for designing battery management systems. Through leveraging the representation powers of deep neural networks and the flexibility and versatility of reinforcement learning, DRL offers a powerful solution to both roofline analysis and real-world deployment on complicated use cases. This work presents a DRL-based battery scheduling framework to solve battery scheduling problems, with high flexibility to fit various battery models and application scenarios. Through the discussion of this framework, comparisons have also been made between conventional heuristics-based methods and DRL. The experiments demonstrate that DRL-based scheduling framework achieves battery lifetime comparable to the best weighted-k round-robin (kRR) heuristic scheduling algorithm. In the meantime, the framework offers much greater flexibility in accommodating a wide range of battery models and use cases, including thermal control and imbalanced battery.

Integrated energy scheduling and routing for a network of mobile prosumers

“…All the other parameters have the same settings as parameters in [42,111]. The prosumers' electricity consumption rates for mobility are adopted from [112]. All the parameter settings are provided in Appendix A.…”

Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

“…A minority of works considers the impact of temperature on aging and degradation. Sui & Song [170] consider a battery pack and propose an intelligent controller to select between batteries to avoid overheating caused by excessively frequent charging and discharging of any single battery. Li et al [19] go further and consider diverse 'high energy' and 'high power' battery packs [100].…”

A Prescriptive Dirichlet Power Allocation Policy with Deep Reinforcement Learning

“…For example, taking the economic and environmental impacts into account and prescribing maintenance operation to improve the efficiency of aircraft maintenance [34]. For batteries, optimal charging schedules have been proposed to prolong the remaining useful life (RUL) [49]. Prescriptive operation provides a very promising and urgently required research direction for the operation of industrial applications due to the increasing complexity and increasing requirements of complex industrial assets [55,3].…”