Multi-Scale Feature Fusion: Learning Better Semantic Segmentation For Road Pothole Detection

Fan, Jiahe; Bocus, Mohammud Z.; Hosking, Brett; Ri-gen, WU; Liu, Yanan; Vityazev, Sergey; Fan, Rui

doi:10.1109/icas49788.2021.9551165

Cited by 34 publications

(14 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1, the SoTA semantic segmentation networks are grouped into two major categories: (1) single-modal and (2) data-fusion. Single-modal networks generally segment RGB images with encoder-decoder architectures [100]. Data-fusion networks typically learn visual features from two different types of vision sensor data (color images and depth maps were used in FuseNet [104], color images and surface normal maps were used in SNE-RoadSeg series [105,106], and color images and transformed disparity images were used in AA-RTFNet [11]) and fuse the learned visual features to provide a better semantic understanding of the environment.…”

Section: Semantic Segmentation-based Methodsmentioning

confidence: 99%

“…Compared to supervised learning, semisupervised learning can greatly improve the overall F-score. Additionally, [100] incorporates an attention-based multi-scale feature fusion module (MSFFM) into DeepLabv3+ [107] for road pothole detection. Similarly, [99] proposes an attentionbased coupled framework for road pothole detection.…”

Section: Semantic Segmentation-based Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Computer Vision for Road Imaging and Pothole Detection: A State-of-the-Art Review of Systems and Algorithms

Ma,

Fan,

Wang

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Computer vision algorithms have been prevalently utilized for 3-D road imaging and pothole detection for over two decades. Nonetheless, there is a lack of systematic survey articles on state-of-the-art (SoTA) computer vision techniques, especially deep learning models, developed to tackle these problems. This article first introduces the sensing systems employed for 2-D and 3-D road data acquisition, including camera(s), laser scanners, and Microsoft Kinect. Afterward, it thoroughly and comprehensively reviews the SoTA computer vision algorithms, including (1) classical 2-D image processing, (2) 3-D point cloud modeling and segmentation, and (3) machine/deep learning, developed for road pothole detection. This article also discusses the existing challenges and future development trends of computer vision-based road pothole detection approaches: classical 2-D image processing-based and 3-D point cloud modeling and segmentation-based approaches have already become history; and Convolutional neural networks (CNNs) have demonstrated compelling road pothole detection results and are promising to break the bottleneck with the future advances in self/un-supervised learning for multi-modal semantic segmentation. We believe that this survey can serve as practical guidance for developing the next-generation road condition assessment systems.

show abstract

Section: Semantic Segmentation-based Methodsmentioning

confidence: 99%

Section: Semantic Segmentation-based Methodsmentioning

confidence: 99%

Computer Vision for Road Imaging and Pothole Detection: A State-of-the-Art Review of Systems and Algorithms

Ma,

Fan,

Wang

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Fan et al [ 39 ] proposed a spatial-attention-based multiscale-fusion module to combine both bottom- and top-layer feature maps. The correlation information between pixels from distinct feature maps is used to solve the semantic gap between scales.…”

Section: Multiscale-deep-learning Taxonomymentioning

confidence: 99%

A Review on Multiscale-Deep-Learning Applications

Elizar

Zulkifley

Muharar

et al. 2022

Sensors

View full text Add to dashboard Cite

In general, most of the existing convolutional neural network (CNN)-based deep-learning models suffer from spatial-information loss and inadequate feature-representation issues. This is due to their inability to capture multiscale-context information and the exclusion of semantic information throughout the pooling operations. In the early layers of a CNN, the network encodes simple semantic representations, such as edges and corners, while, in the latter part of the CNN, the network encodes more complex semantic features, such as complex geometric shapes. Theoretically, it is better for a CNN to extract features from different levels of semantic representation because tasks such as classification and segmentation work better when both simple and complex feature maps are utilized. Hence, it is also crucial to embed multiscale capability throughout the network so that the various scales of the features can be optimally captured to represent the intended task. Multiscale representation enables the network to fuse low-level and high-level features from a restricted receptive field to enhance the deep-model performance. The main novelty of this review is the comprehensive novel taxonomy of multiscale-deep-learning methods, which includes details of several architectures and their strengths that have been implemented in the existing works. Predominantly, multiscale approaches in deep-learning networks can be classed into two categories: multiscale feature learning and multiscale feature fusion. Multiscale feature learning refers to the method of deriving feature maps by examining kernels over several sizes to collect a larger range of relevant features and predict the input images’ spatial mapping. Multiscale feature fusion uses features with different resolutions to find patterns over short and long distances, without a deep network. Additionally, several examples of the techniques are also discussed according to their applications in satellite imagery, medical imaging, agriculture, and industrial and manufacturing systems.

show abstract

“…Moreover, a largescale multi-modal pothole detection dataset (containing RGB images and transformed disparity images), referred to as Pothole-600 4 , was published for research purposes. Similar to [5], [112] also introduced a road pothole detection approach based on single-modal semantic image segmentation in 2021. Its network architecture, as shown in Fig.…”

Section: Semantic Segmentation-based Approachesmentioning

confidence: 99%

Computer-Aided Road Inspection: Systems and Algorithms

Fan¹,

Guo²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Road damage is an inconvenience and a safety hazard, severely affecting vehicle condition, driving comfort, and traffic safety. The traditional manual visual road inspection process is pricey, dangerous, exhausting, and cumbersome. Also, manual road inspection results are qualitative and subjective, as they depend entirely on the inspector's personal experience. Therefore, there is an ever-increasing need for automated road inspection systems. This chapter first compares the five most common road damage types. Then, 2-D/3-D road imaging systems are discussed. Finally, state-of-the-art machine vision and intelligence-based road damage detection algorithms are introduced.

show abstract

Multi-Scale Feature Fusion: Learning Better Semantic Segmentation For Road Pothole Detection

Cited by 34 publications

References 20 publications

Computer Vision for Road Imaging and Pothole Detection: A State-of-the-Art Review of Systems and Algorithms

Computer Vision for Road Imaging and Pothole Detection: A State-of-the-Art Review of Systems and Algorithms

A Review on Multiscale-Deep-Learning Applications

Computer-Aided Road Inspection: Systems and Algorithms

Contact Info

Product

Resources

About