Research on multitask model of object detection and road segmentation in unstructured road scenes

Abstract

With the rapid development of artificial intelligence and computer vision technology, autonomous driving technology has become a hot area of concern. The driving scenarios of autonomous vehicles can be divided into structured scenarios and unstructured scenarios.
Compared with structured scenes, unstructured road scenes lack the constraints of lane lines and traffic rules, and the safety awareness of traffic participants is weaker. Therefore, there are new and higher requirements for the environment perception tasks of autonomous vehicles in unstructured road scenes. The current research rarely integrates the target detection and road segmentation to achieve the simultaneous processing of target detection and road segmentation of autonomous vehicle in unstructured road scenes. Aiming at the above issues, a multitask model for object detection and road segmentation in unstructured road scenes is proposed. Through the sharing and fusion of the object detection model and road segmentation model, multitask model can complete the tasks of multi-object detection and road segmentation in unstructured road scenes while inputting a picture. Firstly, MobileNetV2 is used to replace the backbone network of YOLOv5, and multi-scale feature fusion is used to realize the information exchange layer between different features.
Subsequently, a road segmentation model was designed based on the DeepLabV3+ algorithm.
Its main feature is that it uses MobileNetV2 as the backbone network and combines the binary classification focus Loss function for network optimization. Then, we fused the object detection algorithm and road segmentation algorithm based on the shared MobileNetV2 network to obtain a multitask model and trained it on both the public dataset and the self-built dataset NJFU. The training results demonstrate that the multitask model significantly enhances the algorithm's execution speed by approximately 10 FPS while maintaining the accuracy of object detection and road segmentation. Finally, we conducted validation of the multitask model on an actual vehicle.

Deep learning–based eye tracking system to detect distracted driving

“…However, drivers often cannot focus on multiple pieces of information simultaneously, thereby increasing safety risks [21]. Target detection algorithms have emerged as a mainstream method for sensing the driving environment [22][23][24][25][26]. In 2010, Mnih and Hinton [27] applied deep learning techniques to two datasets of remote sensing images, extracting road features for training with promising results.…”

TTIS-YOLO: a traffic target instance segmentation paradigm for complex road scenarios

“…Autonomous driving technology is a technology that integrates machine vision, sensor fusion, and artificial intelligence to achieve autonomous driving of vehicles [1]. It can improve the driving experience, reduce traffic congestion, ensure road safety, and improve traffic efficiency.…”