Occlusion-Free Road Segmentation Leveraging Semantics for Autonomous Vehicles

Abstract

The deep convolutional neural network has led the trend of vision-based road detection, however, obtaining a full road area despite the occlusion from monocular vision remains challenging due to the dynamic scenes in autonomous driving. Inferring the occluded road area requires a comprehensive understanding of the geometry and the semantics of the visible scene. To this end, we create a small but effective dataset based on the KITTI dataset named KITTI-OFRS (KITTI-occlusion-free road segmentation) dataset and propose a lightweight and efficient, fully convolutional neural network called OFRSNet (occlusion-free road segmentation network) that learns to predict occluded portions of the road in the semantic domain by looking around foreground objects and visible road layout. In particular, the global context module is used to build up the down-sampling and joint context up-sampling block in our network, which promotes the performance of the network. Moreover, a spatially-weighted cross-entropy loss is designed to significantly increases the accuracy of this task. Extensive experiments on different datasets verify the effectiveness of the proposed approach, and comparisons with current excellent methods show that the proposed method outperforms the baseline models by obtaining a better trade-off between accuracy and runtime, which makes our approach is able to be applied to autonomous vehicles in real-time.

Sensors and Sensing for Intelligent Vehicles

“…Concerning the different topics and subtopics, we have identified up to seven main categories, and some sub-categories that are presented in the following list (the number of papers per each category/sub-category is enclosed in parentheses): Object detection and scene understanding (11) Vehicle detection and tracking (4): [ 13 , 14 , 15 , 16 ]. Scene segmentation and interpretation (7) Road segmentation (2): [ 17 , 18 ]. Shadow detection (1): [ 19 ].…”

“…Scene segmentation and interpretation (7) Road segmentation (2): [ 17 , 18 ]. Shadow detection (1): [ 19 ].…”

“…Wang et al address in [ 18 ] the problem of performing semantic segmentation for occluded areas (the overall idea is depicted in Figure 10 ). This is a complex problem that requires a comprehensive understanding of the geometry and the semantics of the visible environment.…”

“… Comparison of road segmentation and proposed occlusion-free road segmentation. ( a ) RGB image; ( b ) standard road segmentation; ( c ) semantic segmentation; ( d ) occlusion-free road segmentation (images obtained from [ 18 ]). …”

Deep Learning Based Pavement Inspection Using Self-Reconfigurable Robot

“…Computer vision with Machine Learning (ML) and Deep Learning (DL) based defect, and cleanness inspection is an emerging technique [ 12 , 13 , 14 , 15 , 16 , 17 , 18 ]. It has been widely used for the detection of material defects, drivable region detection in autonomous vehicle, waste management industries [ 19 , 20 , 21 ]. In contrast with manual inspection scheme, computer vision with ML-based inspection methods are faster, high-precision, and more suitable for routine infrastructure and cleanness inspection task.…”