A Deep Q-Learning Direct Torque Controller for Permanent Magnet Synchronous Motors

Schenke, Maximilian; Wallscheid, Oliver

doi:10.1109/ojies.2021.3075521

Cited by 35 publications

(22 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The original definition of reward from [9] handles all aforementioned operation specifications with exception of the voltage boundary, whose consideration is hardly possible without availability of any model. Therefore, a data-driven prediction model is presented in the following section.…”

Section: Control Designmentioning

confidence: 99%

“…2: Graphical depiction of the proposed reward function gradient according to Tab. II (oriented at [9]), please note that regions B S , D S and E S correspond to safeguard activation and are not actually entered prediction solely on the basis of o k and a k . This means that o k must incorporate all necessary information about the plant state that render the control agent capable of such a prediction while simultaneously avoiding redundant features.…”

Section: Control Designmentioning

confidence: 99%

“…The employed MicroLabBox system by dSPACE permits inference of the DQN (i.e., function evaluation of qθ (o k , a k )) either by means of its CPU or by utilization of the built-in FPGA. As indicated in [9], sensible ANNs for the given task grow to a size where implementation of the ANN inference on the CPU is not feasible anymore to yield a result within the given sampling time of T s . Here it is more reasonable to make use of the FPGA's parallelization potential to allow adherence to the real-time constraint.…”

Section: B Dqn Inferencementioning

confidence: 99%

“…III. The real-world behavior of the described motor setup does, naturally, not fit the ideal assumptions that have been featured in [9]. The rotational vibration capability of the drive train and the limited bandwidth of common torque sensors complicate the learning task even further and must be dealt with to enable model-independent learning.…”

Section: B Dqn Inferencementioning

confidence: 99%

See 3 more Smart Citations

Finite-Set Direct Torque Control via Edge Computing-Assisted Safe Reinforcement Learning for a Permanent Magnet Synchronous Motor

Schenke¹,

Haucke-Korber²,

Wallscheid³

2023

Preprint

View full text Add to dashboard Cite

<p>Advances in the field of reinforcement learning (RL)-based drive control allow formulation of holistic optimization goals for the data-driven training phase. The resulting controllers feature efficient drive operation without the necessity of an a priori known plant model but, so far, conduction of the corresponding training phase in real-world drive systems has been applied only sparsely due to safety concerns. This contribution targets the challenging problem of self-learning torque control for a permanent magnet synchronous motor assuming a finite control set (FCS), i.e., the direct selection of switching actions instead of a modulator-based setup. In order to allow a secure and effective online training with real-world drive systems, the RL controller is monitored by a safeguarding algorithm that prevents application of unsafe switching actions, e.g., such that result in overcurrent. The accruing amount of measurement data is handled with the use of an edge computing pipeline to outsource the RL training from the embedded control hardware. The inference of the utilized artificial neural network in hard real time is realized with the use of a reconfigurable FPGA architecture. The resulting RL-based algorithm is able to learn a torque control policy in just ten minutes which has been validated during comprehensive real-world experiments.</p>

show abstract

Section: Control Designmentioning

confidence: 99%

Section: Control Designmentioning

confidence: 99%

Section: B Dqn Inferencementioning

confidence: 99%

Section: B Dqn Inferencementioning

confidence: 99%

See 2 more Smart Citations

Finite-Set Direct Torque Control via Edge Computing-Assisted Safe Reinforcement Learning for a Permanent Magnet Synchronous Motor

Schenke¹,

Haucke-Korber²,

Wallscheid³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Experience replay is utilized to break the correlations between the samples, which prevents the agent from overfitting to the sequence of training. In [33], unlike the conventional control method, the complex nonlinear model of the PMSM does not need to be known. A DQN-based controller is proposed to control the direct torque of the PMSM; however, DQN can only deal with discrete and low-dimension action spaces.…”

Section: Introductionmentioning

confidence: 99%

PMSM Speed Control Based on Particle Swarm Optimization and Deep Deterministic Policy Gradient under Load Disturbance

et al. 2021

View full text Add to dashboard Cite

Proportional integral-based particle swarm optimization (PSO) and deep deterministic policy gradient (DDPG) algorithms are applied to a permanent-magnet synchronous motor to track speed control. The proposed methods, based on notebooks, can deal with time delay challenges, imprecise mathematical models, and unknown disturbance loads. First, a system identification method is used to obtain an approximate model of the motor. The load and speed estimation equations can be determined using the model. By adding the estimation equations, the PSO algorithm can determine the sub-optimized parameters of the proportional-integral controller using the predicted speed response; however, the computational time and consistency challenges of the PSO algorithm are extremely dependent on the number of particles and iterations. Hence, an online-learning method, DDPG, combined with the PSO algorithm is proposed to improve the speed control performance. Finally, the proposed methods are implemented on a real platform, and the experimental results are presented and discussed.

show abstract

Fault‐resilient control of parallel PV inverters using multi‐agent twin‐delayed deep deterministic policy gradient approach

Malik,

Haque,

Kurukuru

et al. 2023

Circuit Theory & Apps

View full text Add to dashboard Cite

SummaryGrid‐tied photovoltaic system (GTPS) are widely favored due to their inherent benefits. The interface of parallel power electronic converters in these systems is rapidly increasing. The inverter in GTPS has a very important role in power conversion, transfer, and control. However, their challenging working conditions contribute to susceptibility to power switching device failures. Traditional fault diagnosis and tolerant approaches exhibit limitations in control efficiency, model dependency, and slow recovery from fault. This study proposes a multi‐agent twin delayed deep deterministic policy gradient (MATD3PG) configuration for intelligent parallel inverter control, fault diagnosis, and fault‐tolerant operation in GTPS structure. By leveraging the multi‐agent reinforcement learning (RL) framework, an optimal control of the parallel inverter can be achieved, encompassing fault‐tolerant operation using MATLAB Simulink/PLECS. The major advantage the given technique offers is that it carries out optimum fault‐tolerant operation without causing the system derating. Experimental findings demonstrate that the proposed fault‐tolerant model based on RL traditional methods is able to ensure continuous supply to the connected loads even during fault events. Further, the transition time from fault occurrence to recovery is found to be 6 ms, which is quite less compared with the fault‐tolerant techniques presented in literature. Through real‐time fault diagnosis‐based results, the proposed approach ensures precise tracking of reference currents, quicker response times, uninterrupted supply, and smooth transition to the post‐fault operation mode.

show abstract

A Deep Q-Learning Direct Torque Controller for Permanent Magnet Synchronous Motors

Cited by 35 publications

References 24 publications

Finite-Set Direct Torque Control via Edge Computing-Assisted Safe Reinforcement Learning for a Permanent Magnet Synchronous Motor

Finite-Set Direct Torque Control via Edge Computing-Assisted Safe Reinforcement Learning for a Permanent Magnet Synchronous Motor

PMSM Speed Control Based on Particle Swarm Optimization and Deep Deterministic Policy Gradient under Load Disturbance

Fault‐resilient control of parallel PV inverters using multi‐agent twin‐delayed deep deterministic policy gradient approach

Contact Info

Product

Resources

About