Energy Management for a Power-Split Plug-In Hybrid Electric Vehicle Based on Reinforcement Learning

Abstract

This paper proposes an energy management strategy for a power-split plug-in hybrid electric vehicle (PHEV) based on reinforcement learning (RL). Firstly, a control-oriented power-split PHEV model is built, and then the RL method is employed based on the Markov Decision Process (MDP) to find the optimal solution according to the built model. During the strategy search, several different standard driving schedules are chosen, and the transfer probability of the power demand is derived based on the Markov chain. Accordingly, the optimal control strategy is found by the Q-learning (QL) algorithm, which can decide suitable energy allocation between the gasoline engine and the battery pack. Simulation results indicate that the RL-based control strategy could not only lessen fuel consumption under different driving cycles, but also limit the maximum discharge power of battery, compared with the charging depletion/charging sustaining (CD/CS) method and the equivalent consumption minimization strategy (ECMS).

Parameter Matching Optimization of a Powertrain System of Hybrid Electric Vehicles Based on Multi-Objective Optimization

“…where E bat is the energy consumed of the battery per km, in kW/km; L dis is the driving range, in km; U bat is the voltage of battery pack, in V; D soc is the battery discharge depth, in %; η m is the assembly efficiency of the motor and controller; C bat is the rated capacity of a battery cell, in Ah. If the discharge depth of the battery is 70%, 100 LiB cells with 3.2 V are selected in series, that is, the average operating voltage of the battery pack is 320 V, and the number of battery packs in parallel takes the larger value of Equations (5) and (6). Considering the battery efficiency and attenuation, the number of power battery packs can be slightly larger, and the specific number in parallel also needs to be one of the variables in the later research of MOO parameter matching.…”

“…Unlike BEVs, HEVs have more complex structures. According to the structure and energy flow of the powertrain, HEVs are mainly divided into series HEVs, parallel HEVs, and series-parallel HEVs; while according to whether the battery can be externally charged, HEVs are divided into plug-in HEVs [6] and non-plug-in HEVs. The parameters of the powertrain system have a high coupling degree.…”

Special Issue “Plug-In Hybrid Electric Vehicle (PHEV)”

“…The first paper, authored by Zheng Chen [4], proposes an energy management strategy for a power-split plug-in hybrid electric vehicle (PHEV) based on reinforcement learning (RL). Firstly, a control-oriented power-split PHEV model was built, and then the RL method was employed based on the Markov decision process (MDP) to find the optimal solution according to the built model.…”

Torque Coordination Control of Hybrid Electric Vehicles Based on Hybrid Dynamical System Theory

“…Hybrid electric vehicles (HEVs) combine the advantages of both traditional fuel vehicles and battery electric vehicles (BEVs), which include two or more energy sources and can work in coordination [1][2][3]. Under the current technical level and application conditions, HEVs are undoubtedly the most industrialized and market-oriented vehicle in the electric vehicle (EV) category, as a transitional product from traditional fuel vehicles to BEVs.…”

“…Due to the complex structure of HEVs, the optimal energy distribution and control between the two or more energy sources are extremely challenging [1][2][3]. How to design an efficient energy management strategy (EMS) is one of the research hotspots in the field of vehicle control for HEVs.…”