Fishyscapes: A Benchmark for Safe Semantic Segmentation in Autonomous Driving

Blum, Hermann; Sarlin, Paul-Edouard; Nieto, Juan; Siegwart, Roland; Cadena, César

doi:10.1109/iccvw.2019.00294

Cited by 83 publications

(69 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Fishyscapes dataset (Blum et al 2019) is intended to assess the anomaly detection performance of semantic segmentation algorithms for autonomous driving. The task is to train a supervised model on the Cityscapes dataset and, during inference, to localize anomalous objects that were inserted artificially into the test images.…”

Section: Datasetsmentioning

confidence: 99%

Beyond Dents and Scratches: Logical Constraints in Unsupervised Anomaly Detection and Localization

et al. 2022

View full text Add to dashboard Cite

The unsupervised detection and localization of anomalies in natural images is an intriguing and challenging problem. Anomalies manifest themselves in very different ways and an ideal benchmark dataset for this task should contain representative examples for all of them. We find that existing datasets are biased towards local structural anomalies such as scratches, dents, or contaminations. In particular, they lack anomalies in the form of violations of logical constraints, e.g., permissible objects occurring in invalid locations. We contribute a new dataset based on industrial inspection scenarios that evenly covers both types of anomalies. We provide pixel-precise ground truth data for each anomalous region and define a generalized evaluation metric that addresses localization ambiguities that can arise for logical anomalies. Furthermore, we propose a novel algorithm that improves over the state of the art in the joint detection of structural and logical anomalies. It consists of a local and a global network branch. The first one inspects confined regions independent of their spatial locations in the input image and is primarily responsible for the detection of entirely new local structures. The second one learns a globally consistent representation of the training data through a bottleneck that enables the detection of violations of long-range dependencies, a key characteristic of many logical anomalies. We perform extensive evaluations on our new dataset to corroborate our claims.

show abstract

Section: Datasetsmentioning

confidence: 99%

Beyond Dents and Scratches: Logical Constraints in Unsupervised Anomaly Detection and Localization

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Each defect is coarsely annotated with a bounding box. Blum et al (2019) recently introduced Fishyscapes, a dataset intended to benchmark semantic segmentation algorithms with respect to their ability to detect out-of- The bottom row gives a close-up view. For anomalous images, the close-up highlights the anomalous regions distribution inputs.…”

Section: Segmentation Of Anomalous Regionsmentioning

confidence: 99%

The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection

et al. 2021

View full text Add to dashboard Cite

The detection of anomalous structures in natural image data is of utmost importance for numerous tasks in the field of computer vision. The development of methods for unsupervised anomaly detection requires data on which to train and evaluate new approaches and ideas. We introduce the MVTec anomaly detection dataset containing 5354 high-resolution color images of different object and texture categories. It contains normal, i.e., defect-free images intended for training and images with anomalies intended for testing. The anomalies manifest themselves in the form of over 70 different types of defects such as scratches, dents, contaminations, and various structural changes. In addition, we provide pixel-precise ground truth annotations for all anomalies. We conduct a thorough evaluation of current state-of-the-art unsupervised anomaly detection methods based on deep architectures such as convolutional autoencoders, generative adversarial networks, and feature descriptors using pretrained convolutional neural networks, as well as classical computer vision methods. We highlight the advantages and disadvantages of multiple performance metrics as well as threshold estimation techniques. This benchmark indicates that methods that leverage descriptors of pretrained networks outperform all other approaches and deep-learning-based generative models show considerable room for improvement.

show abstract

“…The experiments showed that the proposed method can better explain the mechanism CNN uses for prediction tasks. Hermann et al [21] built Fishyscapes based on the data from Cityscapes, which is a public benchmark for uncertainty estimation in the realworld task of semantic segmentation for urban driving.…”

Section: Related Workmentioning

confidence: 99%

Pyramid Bayesian Method for Model Uncertainty Evaluation of Semantic Segmentation in Autonomous Driving

2022

View full text Add to dashboard Cite

With the fast-developing deep learning models in the field of autonomous driving, the research on the uncertainty estimation of deep learning models has also prevailed. Herein, a pyramid Bayesian deep learning method is proposed for the model uncertainty evaluation of semantic segmentation. Semantic segmentation is one of the most important perception problems in understanding visual scene, which is critical for autonomous driving. This study to optimize Bayesian SegNet for uncertainty evaluation. This paper first simplifies the network structure of Bayesian SegNet by reducing the number of MC-Dropout layer and then introduces the pyramid pooling module to improve the performance of Bayesian SegNet. mIoU and mPAvPU are used as evaluation matrics to test the proposed method on the public Cityscapes dataset. The experimental results show that the proposed method improves the sampling effect of the Bayesian SegNet, shortens the sampling time, and improves the network performance.

show abstract

Fishyscapes: A Benchmark for Safe Semantic Segmentation in Autonomous Driving

Cited by 83 publications

References 22 publications

Beyond Dents and Scratches: Logical Constraints in Unsupervised Anomaly Detection and Localization

Beyond Dents and Scratches: Logical Constraints in Unsupervised Anomaly Detection and Localization

The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection

Pyramid Bayesian Method for Model Uncertainty Evaluation of Semantic Segmentation in Autonomous Driving

Contact Info

Product

Resources

About