Steady-State Error Compensation for Reinforcement Learning-Based Control of Power Electronic Systems

Weber, Daniel; Schenke, Maximilian; Wallscheid, Oliver

doi:10.1109/access.2023.3297274

Cited by 6 publications

(3 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…dλ FL (t) denotes the derivative of F x,FL (t) to λ FL (t). The continuous-time state space representation, Equation (32), is discretized with a sampling time.…”

Section: Internal Prediction Modelmentioning

confidence: 99%

“…To precisely anticipate changes in braking torque within the predictive horizon, an algorithm [47] is proposed to compensate for the state variable () xt in Equation (32). This algorithm enhances the robustness of the eMPC controller against parameter changes.…”

Section: Online State Optimization Based On Feedback Error Compensationmentioning

confidence: 99%

“…To substantially utilize information collected during vehicle braking, aiming to reduce the adverse impact of state error on the control effect, recent research on optimizing system state variables has introduced adaptive optimization methods [30,31] and state error compensation techniques [32][33][34]. Adaptive optimization methods, such as model reference adaptive control (MRAC) [35], direct adaptive control (DAC) [36], and fuzzy adaptive control (FAC) [37], allow for the automatic adjustment of controller parameters in response to changes in system status and parameters, thereby enhancing control performance.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

RBS and ABS Coordinated Control Strategy Based on Explicit Model Predictive Control

Chu,

Li,

Guo

et al. 2024

Sensors

View full text Add to dashboard Cite

During the braking process of electric vehicles, both the regenerative braking system (RBS) and anti-lock braking system (ABS) modulate the hydraulic braking force, leading to control conflict that impacts the effectiveness and real-time capability of coordinated control. Aiming to enhance the coordinated control effectiveness of RBS and ABS within the electro-hydraulic composite braking system, this paper proposes a coordinated control strategy based on explicit model predictive control (eMPC-CCS). Initially, a comprehensive braking control framework is established, combining offline adaptive control law generation, online optimized control law application, and state compensation to effectively coordinate braking force through the electro-hydraulic system. During offline processing, eMPC generates a real-time-oriented state feedback control law based on real-world micro trip segments, improving the adaptiveness of the braking strategy across different driving conditions. In the online implementation, the developed three-dimensional eMPC control laws, corresponding to current driving conditions, are invoked, thereby enhancing the potential for real-time braking strategy implementation. Moreover, the state error compensator is integrated into eMPC-CCS, yielding a state gain matrix that optimizes the vehicle braking status and ensures robustness across diverse braking conditions. Lastly, simulation evaluation and hardware-in-the-loop (HIL) testing manifest that the proposed eMPC-CCS effectively coordinates the regenerative and hydraulic braking systems, outperforming other CCSs in terms of braking energy recovery and real-time capability.

show abstract

“…dλ FL (t) denotes the derivative of F x,FL (t) to λ FL (t). The continuous-time state space representation, Equation (32), is discretized with a sampling time.…”

Section: Internal Prediction Modelmentioning

confidence: 99%

Section: Online State Optimization Based On Feedback Error Compensationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

RBS and ABS Coordinated Control Strategy Based on Explicit Model Predictive Control

Chu,

Li,

Guo

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

Application of data‐driven methods in power systems analysis and control

Bertozzi,

Chamorro,

Gomez‐Diaz

et al. 2023

IET Energy Syst Integration

View full text Add to dashboard Cite

The increasing integration of variable renewable energy resources through power electronics has brought about substantial changes in the structure and dynamics of modern power systems. In response to these transformations, there has been a surge in the development of tools and algorithms leveraging real‐time computational power to enhance system operation and stability. Data‐driven methods have emerged as practical approaches for extracting reliable representations from non‐linear system data, enabling the identification of dynamics and system parameters essential for analysing stability and ensuring reliable operation. This study provides a comprehensive review of recent contributions in the literature concerning the application of data‐driven identification, analysis, and control methods in various aspects of power system operation. Specifically, the focus is on frequency support, power oscillation detection, and damping, which play crucial roles in maintaining grid stability. By discussing the challenges posed by parametric uncertainties, load and source variability, and reduced system inertia, this review sheds light on the opportunities for future research endeavours.

show abstract

Empowering Dynamic Active and Reactive Power Control: A Deep Reinforcement Learning Controller for Three-Phase Grid-Connected Electric Vehicles

Ahmadian,

Sedghisigarchi,

Gadh

2024

IEEE Access

View full text Add to dashboard Cite

Advancing bidirectional voltage source inverters in the on-board chargers of electric vehicles is vital for smart grid electric power management. These electric vehicle inverters play a crucial role in actively contributing to active/reactive power flow control in vehicle-to-grid and grid-to-vehicle applications. In this research, a proof-of-concept, model-free reinforcement learning-based agent/controller is introduced that eliminates the need for system identification or linearization techniques for control design. The goal is to dynamically control user-specified active and reactive electric vehicle on-board charger power outputs in all four quadrants for three-phase grid-connected electric vehicle voltage source inverters. A Twin-Delayed Deep Deterministic Policy Gradient deep-reinforcement learning-based controller was designed (1) to smoothly learn continuous control variables and to generate deterministic actions from the learned control policy, (2) to avoid exhaustive system identification techniques associated with nonlinearities involved in the on-board charger of electric vehicles, and (3) to ultimately track continuous active and reactive powers across all four-quadrant operation modes, ensuring safe and minimal steady-state errors. In this paper, a Markov Decision Process framework was established, encompassing the environment, actions, states, and a unique reward function. The minimal steady state error achieved in active and reactive power reference tracking across all four quadrant operation modes substantiates the trained agent's capacity to sustain tracking accuracy and stability across a variety of operational settings. This research could potentially pave the way for any type of electric vehicle to actively play a role in smart grid management, especially through vehicle-to-grid and grid-to-vehicle capabilities.INDEX TERMS EV (Electric Vehicle), OBC (On-Board Charger), VSI (Voltage Source Inverter), RL (Reinforcement Learning).

show abstract

Steady-State Error Compensation for Reinforcement Learning-Based Control of Power Electronic Systems

Cited by 6 publications

References 45 publications

RBS and ABS Coordinated Control Strategy Based on Explicit Model Predictive Control

RBS and ABS Coordinated Control Strategy Based on Explicit Model Predictive Control

Application of data‐driven methods in power systems analysis and control

Empowering Dynamic Active and Reactive Power Control: A Deep Reinforcement Learning Controller for Three-Phase Grid-Connected Electric Vehicles

Contact Info

Product

Resources

About