Transferring Online Reinforcement Learning for Electric Motor Control From Simulation to Real-World Experiments

Book, Gerrit; Traue, Arne; Balakrishna, Praneeth; Brosch, Anian; Schenke, Maximilian; Hanke, Sören; Kirchgässner, Wilhelm; Wallscheid, Oliver

doi:10.1109/ojpel.2021.3065877

Cited by 36 publications

(29 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the existing work carried out in this field, the research team of the Paderborn University [233] is at the forefront exploring the boundary and tackle the unsolved problems to make this deep RL-enabled data-driven motor control approach a competitive alternative to classical methods [67]- [73]. To start with, a simulative proof-of-concept of the current control in a PMSM drive is presented in [67], which successfully validates the basic design architecture shown in Fig.…”

Section: A Workflow From Simulation To the Test Benchmentioning

confidence: 72%

“…More recently, another important step is accomplished towards introducing RL in the embedded control of physical motor drives, which involves the complete workflow transferring an RL controller from offline simulation to online training and inference on real motor drive systems, as illustrated in Fig. 8(b) [73]. The hardware implementation is carried out by running automated and exported C-code on commercial rapid control prototyping systems -dSPACE MicroLabBox and DS1006MC.…”

Section: A Workflow From Simulation To the Test Benchmentioning

confidence: 99%

“…In addition to embedded control systems, commercial rapid control prototyping systems have also been used in deploying deep learning-based motor control algorithms. Such systems include the dSPACE MicroLabBox and DS1006MC [73], which implement a deep deterministic policy gradient algorithm that learns the current control policy for a PM motor. It is further envisioned in [73] that the presented training scheme can also be implemented on typical SoC embedded hardware, and an implementation in typical industrial applications will also be possible for low-cost applications in the future.…”

Section: Othersmentioning

confidence: 99%

See 2 more Smart Citations

Artificial Intelligence in Electric Machine Drives: Advances and Trends

Zhang¹

2021

Preprint

View full text Add to dashboard Cite

This review paper systematically summarizes the existing literature on applying classical AI techniques and advanced deep learning algorithms to electric machine drives. It is anticipated that with the rapid progress in deep learning models and embedded hardware platforms, AI-based data-driven approaches will become increasingly popular for the automated high-performance control of electric machines. Additionally, this paper also provides some outlook towards promoting its widespread application in the industry, such as implementing advanced RL algorithms with good domain adaptation and transfer learning capabilities and deploying them onto low-cost SoC FPGA devices.

show abstract

Section: A Workflow From Simulation To the Test Benchmentioning

confidence: 72%

Section: A Workflow From Simulation To the Test Benchmentioning

confidence: 99%

Section: Othersmentioning

confidence: 99%

See 1 more Smart Citation

Artificial Intelligence in Electric Machine Drives: Advances and Trends

Zhang¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…An RL-based attempt for the latter scenario has been presented in [20], where the remaining challenge of stationary control error has not been addressed. The transfer from simulation to real-world experiment was delivered in [21], which underlines that RL control is making its path into practical application. In the corresponding article, the RL agent lacks behind concerning the steady-state error compared to a classical PI controller but outperforms it in terms of total demand distortion.…”

Section: A State Of the Artmentioning

confidence: 99%

Steady-State Error Compensation in Reference Tracking and Disturbance Rejection Problems for Reinforcement Learning-Based Control

Weber¹,

Schenke²,

Wallscheid³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Reinforcement learning (RL) is a promising, upcoming topic in automatic control applications. Where classical control approaches require a priori system knowledge, data-driven control approaches like RL allow a modelfree controller design procedure, rendering them emergent techniques for systems with changing plant structures and varying parameters. While it was already shown in various applications that the transient control behavior for complex systems can be sufficiently handled by RL, the challenge of non-vanishing steady-state control errors remains, which arises from the usage of control policy approximations and finite training times. To overcome this issue, an integral action state augmentation (IASA) for actor-critic-based RL controllers is introduced that mimics an integrating feedback, which is inspired by the delta-input formulation within model predictive control. This augmentation does not require any expert knowledge, leaving the approach model free. As a result, the RL controller learns how to suppress steady-state control deviations much more effectively. Two exemplary applications from the domain of electrical energy engineering validate the benefit of the developed method both for reference tracking and disturbance rejection. In comparison to a standard deep deterministic policy gradient (DDPG) setup, the suggested IASA extension allows to reduce the steady-state error by up to 52 % within the considered validation scenarios.

show abstract

“…While first promising CCS-RL approaches for the current control problem of drives are already available [18][19] [20], additional control tasks, such as torque or speed control, and learning FCS approaches have not yet been investigated. The question therefore arises if learning, model-free controllers can also be successfully applied to further, challenging control tasks in drive and power electronic applications.…”

Section: A State-of-the-artmentioning

confidence: 99%