Optimal Combination of Imitation and Reinforcement Learning for Self-driving Cars

Abstract

The two steps in human intelligence development, namely, mimicking and tentative application of expertise, are reflected by imitation learning (IL) and reinforcement learning (RL) in artificial intelligence (AI). However, the RL process does not always improve the skills learned from expert demonstrations and enhance the algorithm performance. To solve the problem, this paper puts forward a novel algorithm called optimal combination of imitation and reinforcement learning (OCIRL). First, the concept of deep q-learning from demonstrations (DQfD) was introduced to the actor-critic (A2C) model, creating the A2CfD model. Then, a threshold was estimated from a trained IL model with the same inputs and reward function with the DOfD, and applied to the A2CfD model. The threshold represents the minimum reward that conserves the learned expertise. The resulting A2CfDoC model was trained and tested on self-driving cars in both discrete and continuous environments. The results show that the model outperformed several existing algorithms in terms of speed and accuracy.

Transit Assignment Modeling Approaches based on Interval Uncertainty of Urban Public Transit Net Impedance

“…A future study may develop a transit assignment approach by modeling the feature of morning commute [25,26] and taking connected and autonomous vehicle environment into account [27][28][29].…”

Sound Visualization and Convolutional Neural Network in Fault Diagnosis of Electric Motorbike

“…In 2020, a study of malicious webpage detection algorithm based on image semantics that derived by the backpropagation neural network (BPNN) to obtain the target image, and combine the image with other functions of the malicious website, so that the malicious website can be detected [14]. In 2019, Fenjiro and Benbrahim proposed an optimal combination of imitation and reinforcement learning for selfdriving cars, carry out self-driving training through deep learning [15]. In 2019, a micro-expression recognition algorithm for students in classroom based on CNN to detect human faces and detect facial micro-expressions is studied [16].…”