A model free controller based on reinforcement learning for active steering system with uncertainties

Zhao, Jintao; Cheng, Shuo; Zhang, Lipeng; Li, Mingcong; Zhang, Zhihuang

doi:10.1177/0954407021994416

Cited by 3 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The literature provides the control of various actuators inside the powertrain, such as an active turbocharger or an exhaust gas recirculation (EGR) valve [30,31,33] of a combustion engine or the inverter of an electric motor [34]. Furthermore, applications outside the powertrain, such as an active steering system [19] or a semi-active suspension [35] control, can be found in the literature. Here, specialized simulations are utilized to model the specific system that contains the controlled component.…”

Section: Introductionmentioning

confidence: 99%

Cloud-Based Reinforcement Learning in Automotive Control Function Development

Koch,

Roeser,

Badalian

et al. 2023

Vehicles

View full text Add to dashboard Cite

Automotive control functions are becoming increasingly complex and their development is becoming more and more elaborate, leading to a strong need for automated solutions within the development process. Here, reinforcement learning offers a significant potential for function development to generate optimized control functions in an automated manner. Despite its successful deployment in a variety of control tasks, there is still a lack of standard tooling solutions for function development based on reinforcement learning in the automotive industry. To address this gap, we present a flexible framework that couples the conventional development process with an open-source reinforcement learning library. It features modular, physical models for relevant vehicle components, a co-simulation with a microscopic traffic simulation to generate realistic scenarios, and enables distributed and parallelized training. We demonstrate the effectiveness of our proposed method in a feasibility study to learn a control function for automated longitudinal control of an electric vehicle in an urban traffic scenario. The evolved control strategy produces a smooth trajectory with energy savings of up to 14%. The results highlight the great potential of reinforcement learning for automated control function development and prove the effectiveness of the proposed framework.

show abstract

Section: Introductionmentioning

confidence: 99%

Cloud-Based Reinforcement Learning in Automotive Control Function Development

Koch,

Roeser,

Badalian

et al. 2023

Vehicles

View full text Add to dashboard Cite

show abstract

“…5,6 In this context, the use of simulation in the automotive industry has become a fundamental step during the development of new projects 7 and it has not been different in the chassis design, 8,9 especially regarding the suspension [10][11][12] and its correlated systems behavior. 13,14 Parallel to this, the parameters that govern vehicle kinematics and compliance are among the most important keys to a vehicle chassis design. 15,16 In the same way, driver-in-the-loop (DIL) simulation, which includes a driver in the test loop, is being very used on dynamic vehicle simulators.…”

Section: Introductionmentioning

confidence: 99%

Vehicle dynamics response due to the adjustment of K&C parameters with the aid of a dynamic simulator with 9 DOF and a correlated virtual model

Dias

Landre

2022

Proceedings of the Institution of Mechanical Engineers, Part D:

View full text Add to dashboard Cite

Given the increasing use of virtual simulation in automotive vehicle designs, the correlation of virtual models is a fundamental part of the process. The present work seeks to develop a methodology for the development of virtual vehicle dynamics models correlated with kinematics and compliance (K&C) parameters for use in a real-time vehicle dynamics simulator. Aiming this and with the aid of an SPMM 5000e to obtain data from the physical vehicle and from the ADAMS/Car and VI-CarRealTime (VI-CRT) software to the development and simulation and from MATLAB to the correlation, a multibody virtual model correlated in K&C to a hatchback car, with McPherson strut and twist-beam front and rear suspensions, respectively, is developed. At the end of the work, it was observed that the correlation between the virtual model and the test vehicle was found successfully and that, therefore, the virtual simulation allows to find reliable results regarding the vehicle’s suspension and steering system behavior. Subsequently, with the tests carried out with a driver in a VI-Grade DiM 150, a real-time driver-in-the-loop simulator with 9 degrees of freedom (DOF), the model and thus the methodology is validated through the steering ratio analysis: The lateral acceleration analysis in the vehicle’s center of gravity in different maneuvers allowed to observe that the model with higher steering ratio requires greater steering wheel angle values about the model with the lowest steering ratio so that the same maneuver is performed.

show abstract

Control of a nonlinear active suspension system based on deep reinforcement learning and expert demonstrations

Tan,

Wen,

Pan

et al. 2023

Proceedings of the Institution of Mechanical Engineers, Part D:

View full text Add to dashboard Cite

A well-controlled active suspension system has the potential to provide better ride comfort. Benefiting from its powerful feature extraction and nonlinear generalization capabilities, the deep reinforcement learning (DRL), such as deep deterministic policy gradient (DDPG), has shown great potential to make decisions adaptively and intelligently in the control of active suspension system. However, the DDPG is troubled by the problem of low training efficiency due to the high proportion of illegal strategies. This paper proposed a novel DDPG controller for a nonlinear uncertain active suspension system by combining DRL with expert demonstrations. Specifically, the improved training method integrated with both a pre-training mechanism based on PID expert samples and an adaptive experience replay mechanism, is put forward for the DDPG to achieve both the goals of imitating the expert and improving the training efficiency. Moreover, considering the ride comfort and the state constraints as targets, a mixed reward function is designed to guide RL agents for learning effective actions. It is shown that the proposed training methods effectively accelerate the convergence of the DDPG. Furthermore, the comparison experiments demonstrate that the proposed controller provides great vibration attenuation, and has better adaptiveness to various working conditions and parametric uncertainty.

show abstract

A model free controller based on reinforcement learning for active steering system with uncertainties

Cited by 3 publications

References 30 publications

Cloud-Based Reinforcement Learning in Automotive Control Function Development

Cloud-Based Reinforcement Learning in Automotive Control Function Development

Vehicle dynamics response due to the adjustment of K&C parameters with the aid of a dynamic simulator with 9 DOF and a correlated virtual model

Control of a nonlinear active suspension system based on deep reinforcement learning and expert demonstrations

Contact Info

Product

Resources

About