Learning Time Reduction Using Warm-Start Methods for a Reinforcement Learning-Based Supervisory Control in Hybrid Electric Vehicle Applications

Xu, Bin; Hou, Jun; Shi, Junzhe; Li, Huayi; Rathod, Dhruvang; Wang, Zhe; Filipi, Zoran

doi:10.1109/tte.2020.3019009

Cited by 23 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Value estimation [106] parallel PHEV τ engine , n gear discrete combined SARSA [107] FC PHEV P FC , weight of penalty on P bat discrete continuous Q-learning (table-based) [49][50][51]70,76,77,81,82] parallel HEV P x (x = EM or engine) discrete continuous [46,71,85,86,108,109] power-split HEV P bat [66] power-split HEV τ engine , ω engine [52,[56][57][58][59]67,69,78,92] series HEV P engine [84,94,95] battery-UC EV i bat…”

Section: Rl Algorithm(s) Study System Controlled Control Action(s) Ac...mentioning

confidence: 99%

A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation

2023

View full text Add to dashboard Cite

One major cost of improving the automotive fuel economy while simultaneously reducing tailpipe emissions is increased powertrain complexity. This complexity has consequently increased the resources (both time and money) needed to develop such powertrains. Powertrain performance is heavily influenced by the quality of the controller/calibration. Since traditional control development processes are becoming resource-intensive, better alternate methods are worth pursuing. Recently, reinforcement learning (RL), a machine learning technique, has proven capable of creating optimal controllers for complex systems. The model-free nature of RL has the potential to streamline the control development process, possibly reducing the time and money required. This article reviews the impact of choices in two areas on the performance of RL-based powertrain controllers to provide a better awareness of their benefits and consequences. First, we examine how RL algorithm action continuities and control–actuator continuities are matched, via native operation or conversion. Secondly, we discuss the formulation of the reward function. RL is able to optimize control policies defined by a wide spectrum of reward functions, including some functions that are difficult to implement with other techniques. RL action and control–actuator continuity matching affects the ability of the RL-based controller to understand and operate the powertrain while the reward function defines optimal behavior. Finally, opportunities for future RL-based powertrain control development are identified and discussed.

show abstract

Section: Rl Algorithm(s) Study System Controlled Control Action(s) Ac...mentioning

confidence: 99%

A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation

2023

View full text Add to dashboard Cite

show abstract

“…In the calculation of neurons in the output layer, a linear activation function is adopted to predict the resistance in the given condition, 𝑅 = ∑ 𝑊 𝑖 𝐻 𝑖 𝑁 𝑖=1 (7) where 𝑅 is the predicted resistance, 𝑁 is the numbers of nodes in the hidden layer, 𝐻 𝑖 is a radial basis function in the hidden layer, and 𝑊 is the optimal weight vector. In the training processes, the values of the weight vector 𝑊 are determined by fitting the linear model [30] concerning outputs of the hidden layer and the mean squared error (MSE) between predicted voltage and actual measured voltage.…”

Section: Remaining Charging Time Estimation In the CV Stagementioning

confidence: 99%

“…There are many additional benefits of EVs, including high energy efficiency and instant torque supply availability [4]. With the rising demand for EVs, many studies have focused on EV operational optimization, such as energy management optimization [5][6] [7] and battery life optimization [8][9] [10]. However, achieving accurate RCT estimates is a prerequisite for optimal operations [11] [12].…”

Section: Introductionmentioning

confidence: 99%

Electric vehicle battery remaining charging time estimation considering charging accuracy and charging profile prediction

Shi¹,

Tian²,

Han³

et al. 2022

Journal of Energy Storage

Self Cite

View full text Add to dashboard Cite

“…EMS can be divided into rule-based, optimization-based, adaptive control and reinforcement learning-based (RL) strategies [7], [8]. In terms of the rule-based strategy, Li et al proposed a torque-leveling threshold-changing strategy, which can ensure the engine operates at an efficient operating point [9].…”

Section: Introductionmentioning

confidence: 99%

A Robust Design Method for Optimal Engine Operating Zone Design of Plug-in Hybrid Electric Bus

et al. 2022

View full text Add to dashboard Cite

The optimal engine operating zone of energy management plays an important role in the fuel economy improvement of Plug-in hybrid electric buses. However, the existing investigations usually design the engine operating zone by experience. This paper proposes a robust design method for the robust and optimal design of the engine operating zone. Firstly, a nonlinear model predictive control (NMPC)-based energy management together with a single-point preview SOC (state of charge) plan method is designed. Then, a Taguchi robust design model is designed to find the optimal engine operating zone by taking the energy management as underlying solving module. Particularly, the noises of driving cycles and stochastic vehicle mass are considered to improve the robust performance of the engine operating zone. Finally, The Monte Carlo Simulation is deployed to verify the robustness and optimal performances of the designed engine operating zone. Simulation results demonstrate that the proposed method is beneficial to the fuel economy improvement, where the fuel economy can be averagely improved by 9.10% compared with the experienced designed engine operating zone, and can be averagely improved by 16.34% compared with the rule-based energy management strategy.

show abstract

Learning Time Reduction Using Warm-Start Methods for a Reinforcement Learning-Based Supervisory Control in Hybrid Electric Vehicle Applications

Cited by 23 publications

References 30 publications

A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation

A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation

Electric vehicle battery remaining charging time estimation considering charging accuracy and charging profile prediction

A Robust Design Method for Optimal Engine Operating Zone Design of Plug-in Hybrid Electric Bus

Contact Info

Product

Resources

About