A hierarchical framework for improving ride comfort of autonomous vehicles via deep reinforcement learning with external knowledge

Abstract

Ride comfort plays an important role in determining the public acceptance of autonomous vehicles (AVs). Many factors, such as road profile, driving speed, and suspension system, influence the ride comfort of AVs. This study proposes a hierarchical framework for improving ride comfort by integrating speed planning and suspension control in a vehicle-to-everything environment. Based on safe, comfortable, and efficient speed planning via dynamic programming, a deep reinforcement learning-based suspension control is proposed to adapt to the changing pavement conditions. Specifically, a deep deterministic policy gradient with external knowledge (EK-DDPG) algorithm is designed for the efficient selfadaptation of suspension control strategies. The external knowledge of action selection and value estimation from other AVs are combined into the loss functions of the DDPG algorithm. In numerical experiments, real-world pavements detected in 11 districts of Shanghai, China, are applied to verify the proposed method. Experimental results demonstrate that the EK-DDPG-based suspension control improves ride comfort on untrained rough pavements by 27.95% and 3.32%, compared to a model predictive control (MPC) baseline and a DDPG baseline, respectively. Meanwhile, the EK-DDPG-based suspension control improves computational efficiency by 22.97%, compared to the MPC baseline, and performs at the same level as the DDPD baseline. This study provides a generalized and computationally efficient approach for improving the ride comfort of AVs.

Safe, Efficient, and Comfortable Autonomous Driving Based on Cooperative Vehicle Infrastructure System

“…For a vehicle, vertical ride comfort is directly related to the vertical vibration of the seats, which results from the interactions between the seats, vehicle body, suspensions, tires, wheels, and road profiles. The interactions are formulated as mathematical models [5,7]. Since the most commonly used model, the quarter-car model, is too simple to reflect the entire vibration information, a full-car model with a seat modeling is used (see Figure 2) [7].…”

“…The interactions are formulated as mathematical models [5,7]. Since the most commonly used model, the quarter-car model, is too simple to reflect the entire vibration information, a full-car model with a seat modeling is used (see Figure 2) [7]. The dynamic equation of the full-car model is summarized as…”

“…The annoyance rate is formulated with random fuzzy evaluation models, membership functions, and probability distributions [19]: In the state-space formulation, the output is the acceleration in the time domain with irregular fluctuations. Conversely, the patterns of frequency-domain acceleration are more stable [7]. Hence, the time-domain data are translated into the frequency domain using the power spectral density.…”

“…Thus, for some untrained complex situations, it is difficult for supervised learning-based approaches to find optimal solutions. For speed control on rough pavements, model-based speed planning, such as dynamic programming, is commonly used [5,7]. However, model-based speed planning is also based on strong assumptions of the environment, so it struggles to address changing environments.…”

“…The road information was detected by advanced onboard sensors, including LiDAR, accelerometers, and GPS, under the operation of manual vehicles. The resolution of detected road profiles detected by LiDAR was 0.25 m. Based on unexpected vibration detected by accelerometers, the potential damage was located by GPS and captured using wavelet analysis [7]. We sampled 110 rough pavements in this dataset to form a rough pavement dataset for model training and testing.…”