Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes

Pohlen, Tobias; Hermans, Alexander; Mathias, Markus; Leibe, Bastian

doi:10.1109/cvpr.2017.353

Cited by 528 publications

(343 citation statements)

References 51 publications

Supporting

Mentioning

333

Contrasting

Unclassified

Order By: Relevance

“…Unlike typical CNNs, which consist of repeated multiple layers, such as convolutional layers, pooling layers, the activation function, and fully connected layers, deep convolutional encoderdecoder networks employ two separate deep neural networks: an encoder network and a decoder network. This network has been successfully applied to many visual segmentation tasks, such as scene understanding (Kendall, Badrinarayanan, & Cipolla, 2015), autonomous driving (Teichmann, Weber, Zoellner, Cipolla, & Urtasun, 2016), biomedical image segmentation (Ronneberger, Fischer, & Brox, 2015), and semantic segmentation (Badrinarayanan, Kendall, & Cipolla, 2017;Pohlen, Hermans, Mathias, & Leibe, 2017).…”

Section: Encoder-decoder Network For Semantic Segmentationmentioning

confidence: 99%

Encoder–decoder network for pixel‐level road crack detection in black‐box images

Bang

Park

Kim

et al. 2019

Computer aided Civil Eng

353

170

View full text Add to dashboard Cite

Timely monitoring of pavement cracks is essential for successful maintenance of road infrastructure. Accurate information concerning crack location and severity enables proactive management of the infrastructure. Black‐box cameras, which are becoming increasingly widespread at an affordable price, can be used as efficient road‐image collectors over a wide area. However, the cracks in these images are difficult to detect, because the images containing them often include objects other than roads. Thus, we propose a pixel‐level detection method for identifying road cracks in black‐box images using a deep convolutional encoder–decoder network. The encoder consists of convolutional layers of the residual network for extracting crack features, and the decoder consists of deconvolutional layers for localizing the cracks in an input image. The proposed network was trained on 427 out of 527 images extracted from black‐box videos and tested on the remaining 100 images. Compared with VGG‐16, ResNet‐50, ResNet‐101, ResNet‐200 with transfer learning, and ResNet‐152 without transfer learning, ResNet‐152 with transfer learning exhibited the best performance, achieving recall, precision, and intersection of union of 71.98%, 77.68%, and 59.65%, respectively. The experimental results prove that the proposed method is optimal for detecting cracks in black‐box images at the pixel level.

show abstract

Section: Encoder-decoder Network For Semantic Segmentationmentioning

confidence: 99%

Encoder–decoder network for pixel‐level road crack detection in black‐box images

Bang

Park

Kim

et al. 2019

Computer aided Civil Eng

353

170

View full text Add to dashboard Cite

show abstract

“…30 . The beneficial implementation of residual learning has been thoroughly documented in not only image classification and segmentation 31 but also in areas of speech recognition 32 . Fullyconvolutional networks 33 , or networks designed such that input of any spatial dimensionality can be analyzed with no loss in performance, offer enormous benefit to problems where 1) prior knowledge of input size is inherently variable and 2) the experimental data of interest is memory exhaustive.…”

Section: D-cnn Architecture Training and Validationmentioning

confidence: 99%

Ultra-fast fit-free analysis of complex fluorescence lifetime imaging via deep learning

Smith

Yao

Sinsuebphon

et al. 2019

Preprint

View full text Add to dashboard Cite

Fluorescence lifetime imaging (FLI) provides unique quantitative information in biomedical and molecular biology studies, but relies on complex data fitting techniques to derive the quantities of interest. Herein, we propose a novel fit-free approach in FLI image formation that is based on Deep Learning (DL) to quantify complex fluorescence decays simultaneously over a whole image and at ultra-fast speeds. Our deep neural network (DNN), named FLI-Net, is designed and model-based trained to provide all lifetime-based parameters that are typically employed in the field. We demonstrate the accuracy and generalizability of FLI-Net by performing quantitative microscopic and preclinical experimental lifetime-based studies across the visible and NIR spectra, as well as across the two main data acquisition technologies. Our results demonstrate that FLI-Net is well suited to quantify complex fluorescence lifetimes, accurately, in real time in cells and intact animals without any parameter settings. Hence, it paves the way to reproducible and quantitative lifetime studies at unprecedented speeds, for improved dissemination and impact of FLI in many important biomedical applications, especially in clinical settings.

show abstract

“…To illustrate this dilemma, Fig. 1 gives the accuracy (mIoU) and inference speed (frames per second (fps)) obtained by several state-of-the-art methods, including FCN-8s [9], CRF-RNN [17], DeepLab [10], DeepLabv2 [12], DeepLabv3+ [13], ResNet-38 [18], PSPNet [11], DUC [19], RefineNet [20], LRR [21], DPN [22], FRRN [23], TwoColumn [24], SegNet [25], SQNet [26], ENet [27], arXiv:2003.08736v2 [cs.CV] 3 Apr 2020 ERFNet [28], ICNet [29], SwiftNetRN [30], LEDNet [31], BiSeNet1 [32], BiSeNet2 [32], DFANet [33] and our proposed method, on the Cityscapes test dataset. Clearly, how to achieve a good tradeoff between accuracy and speed is still a challenging problem.…”

Section: Introductionmentioning

confidence: 99%

Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes

Dong

Yan

Shen

et al. 2021

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Deep Convolutional Neural Networks (DCNNs) have recently shown outstanding performance in semantic image segmentation. However, state-of-the-art DCNN-based semantic segmentation methods usually suffer from high computational complexity due to the use of complex network architectures. This greatly limits their applications in the real-world scenarios that require real-time processing. In this paper, we propose a real-time high-performance DCNN-based method for robust semantic segmentation of urban street scenes, which achieves a good trade-off between accuracy and speed. Specifically, a Lightweight Baseline Network with Atrous convolution and Attention (LBN-AA) is firstly used as our baseline network to efficiently obtain dense feature maps. Then, the Distinctive Atrous Spatial Pyramid Pooling (DASPP), which exploits the different sizes of pooling operations to encode the rich and distinctive semantic information, is developed to detect objects at multiple scales. Meanwhile, a Spatial detail-Preserving Network (SPN) with shallow convolutional layers is designed to generate highresolution feature maps preserving the detailed spatial information. Finally, a simple but practical Feature Fusion Network (FFN) is used to effectively combine both shallow and deep features from the semantic branch (DASPP) and the spatial branch (SPN), respectively. Extensive experimental results show that the proposed method respectively achieves the accuracy of 73.6% and 68.0% mean Intersection over Union (mIoU) with the inference speed of 51.0 fps and 39.3 fps on the challenging Cityscapes and CamVid test datasets (by only using a single NVIDIA TITAN X card). This demonstrates that the proposed method offers excellent performance at the real-time speed for semantic segmentation of urban street scenes.

show abstract

Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes

Cited by 528 publications

References 51 publications

Encoder–decoder network for pixel‐level road crack detection in black‐box images

Encoder–decoder network for pixel‐level road crack detection in black‐box images

Ultra-fast fit-free analysis of complex fluorescence lifetime imaging via deep learning

Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes

Contact Info

Product

Resources

About