Reinforcement Learning Based Modulation for Balancing Capacitor Voltage and Thermal Stress to Enhance Current Capability of MMCs

Jung, Jae Hwan; Hosseini, Ehsan Shah; Ko, Youngjong; Fernández‐Ramírez, Luis M.

doi:10.1109/pedg54999.2022.9923188

Cited by 6 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Decentralized controllers have already proven to be an effective alternative for governing complex power systems [22,23]. They are based on the premise that numerous controllers can work independently, based on local measurements.…”

Section: Methodsmentioning

confidence: 99%

Multi-Agent Reinforcement Learning-Based Decentralized Controller for Battery Modular Multilevel Inverter Systems

et al. 2023

View full text Add to dashboard Cite

The battery-based multilevel inverter has grown in popularity due to its ability to boost a system’s safety while increasing the effective battery life. Nevertheless, the system’s high degree of freedom, induced by a large number of switches, provides difficulties. In the past, central computation systems that needed extensive communication between the master and the slave module on each cell were presented as a solution for running such a system. However, because of the enormous number of slaves, the bus system created a bottleneck during operation. As an alternative to conventional multilevel inverter systems, which rely on a master–slave architecture for communication, decentralized controllers represent a feasible solution for communication capacity constraints. These controllers operate autonomously, depending on local measurements and decision-making. With this approach, it is possible to reduce the load on the bus system by approximately 90 percent and to enable a balanced state of charge throughout the system with an absolute maximum standard deviation of 1.1×10−5. This strategy results in a more reliable and versatile multilevel inverter system, while the load on the bus system is reduced and more precise switching instructions are enabled.

show abstract

Section: Methodsmentioning

confidence: 99%

Multi-Agent Reinforcement Learning-Based Decentralized Controller for Battery Modular Multilevel Inverter Systems

et al. 2023

View full text Add to dashboard Cite

show abstract

“…In the literature, it has been discussed to balance the voltage of the capacitor and the thermal stress [6] of a modular multilevel converter (MMC) or to optimize the efficiency of the DC-DC converter of the dual active bridge (DAB) [7,8] using Reinforcement Learning (RL) algorithms. Furthermore, the potential of different algorithms in the field of RL have been introduced for the controlling and optimization of reconfigurable batteries.…”

Section: Introductionmentioning

confidence: 99%

Controlling Algorithm of Reconfigurable Battery for State of Charge Balancing using Amortized Q-Learning

Karnehm,

Bliemetsrieder,

Pohlmann

et al. 2024

Preprint

View full text Add to dashboard Cite

Towards smart batteries for electric vehicles (EVs) smart algorithms to control battery packs, mainly reconfigurable batteries, have to be developed. This work proposes a reinforcement learning (RL) algorithm to balance the State of Charge (SoC) of reconfigurable batteries based on the topologies half-bridge and battery modular multilevel management (BM3). As RL algorithm, Amortized Q-learning (AQL) is implemented, which allows enourmous numbers of possible configurations of the reconfigurable battery to be controlled, as well as the combination of classical controlling approach and machine learning methods. This enables safety mechanisms in control. As a neural network of the AQL a Feedforward Neuronal Network (FNN) is implemented consisting of three hidden layers. The experimental evaluation using a 12-cell hybrid cascaded multilevel converter illustrates the applicability of the method to balance the SoC and maintain the balanced state during discharge. The evaluation shows a 20.3% slower balancing compared to the classical. Nevertheless, AQL shows great potential to be applied for multiobjective optimizations as an applicable RL algorithm for control in power electronics.

show abstract

“…In the literature, different Reinforcement Learning (RL) algorithms have been discussed to optimize the system operation parameters. For instance, balancing the voltage of the capacitor and the thermal stress [6] of a modular multilevel converter (MMC) or the optimization of the efficiency of the DC-DC converter of the dual active bridge (DAB) [7,8].…”

Section: Introductionmentioning

confidence: 99%

Controlling Algorithm of Reconfigurable Battery for State of Charge Balancing Using Amortized Q-Learning

Karnehm,

Bliemetsrieder,

Pohlmann

et al. 2024

Batteries

View full text Add to dashboard Cite

In the context of the electrification of the mobility sector, smart algorithms have to be developed to control battery packs. Smart and reconfigurable batteries are a promising alternative to conventional battery packs and offer new possibilities for operation and condition monitoring. This work proposes a reinforcement learning (RL) algorithm to balance the State of Charge (SoC) of reconfigurable batteries based on the topologies half-bridge and battery modular multilevel management (BM3). As an RL algorithm, Amortized Q-learning (AQL) is implemented, which enables the control of enormous numbers of possible configurations of the reconfigurable battery as well as the combination of classical controlling approaches and machine learning methods. This enhances the safety mechanisms during control. As a neural network of the AQL, a Feedforward Neuronal Network (FNN) is implemented consisting of three hidden layers. The experimental evaluation using a 12-cell hybrid cascaded multilevel converter illustrates the applicability of the method to balance the SoC and maintain the balanced state during discharge. The evaluation shows a 20.3% slower balancing process compared to a conventional approach. Nevertheless, AQL shows great potential for multiobjective optimizations and can be applied as an RL algorithm for control in power electronics.

show abstract

Reinforcement Learning Based Modulation for Balancing Capacitor Voltage and Thermal Stress to Enhance Current Capability of MMCs

Cited by 6 publications

References 13 publications

Multi-Agent Reinforcement Learning-Based Decentralized Controller for Battery Modular Multilevel Inverter Systems

Multi-Agent Reinforcement Learning-Based Decentralized Controller for Battery Modular Multilevel Inverter Systems

Controlling Algorithm of Reconfigurable Battery for State of Charge Balancing using Amortized Q-Learning

Controlling Algorithm of Reconfigurable Battery for State of Charge Balancing Using Amortized Q-Learning

Contact Info

Product

Resources

About