Multi-task Autonomous Driving Based on Improved Convolutional Neural Network and ST Loss in MTS and MOD Modes

Abstract

Multi-task autonomous driving is a research hotspot in autonomous driving. However, existing research has only achieved single-task or dual-task autonomous driving. Therefore, we propose two novel multi-task approaches: a multi-task shared model mode (MTS) and a multi-object dual-model mode (MOD). In addition, existing neural network architectures are underperforming in multi-task autonomous driving, so we propose a novel neural network architecture - MT-ResNet26. Moreover, to alleviate the problem of noise and class imbalance from data, we propose a new loss function - Stable Loss (ST Loss). Finally, our smart car can achieve multi-task road tracking, left-right turn sign recognition, automatic obstacle avoidance, stop, real-time acceleration and deceleration. In addition, we compare the existing multi-task autonomous driving model YS-VGG17_MSE, which shows our MT-ResNet26_ST is superior in loss value and actual performance. Meanwhile, we use our proposed approaches to train two classical neural networks—ResNet18_MSE* and DenseNet121_MSE*, so that they also achieve multi-task autonomous driving with our proposed approaches, showing the applicability of MTS and MOD. Furthermore, we compare MT-ResNet26_MSE with MT-ResNet26_ST, and the results show that the model using our novel ST Loss outperforms the model using the original loss function MSE. To sum up, it is shown that the performance of multi-task autonomous driving can be achieved and improved using our proposed neural network architecture and loss function. Furthermore, we propose optimized multi-task modes. OMTS and OMOD optimize and accelerate the models using semi-precision techniques based on the TensorRT. The results show that the optimized multi-task autonomous driving accuracy has been further improved.