Inferring Spatial Uncertainty in Object Detection

Wang, Zining; Feng, Di; Zhou, Yiyang; Rosenbaum, Lars; Timm, Fabian; Dietmayer, Klaus; Tomizuka, Masayoshi; Zhan, Wei

doi:10.1109/iros45743.2020.9340798

Cited by 22 publications

(19 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This has been pointed out and quantified with respect to labeling uncertainty in Ref. [166]. Thus, the quality of the reference data (or labels) should be taken into account such that test results are not misleading [21].…”

Section: Uncertainty In Reference Datamentioning

confidence: 99%

A Review of Testing Object-Based Environment Perception for Safe Automated Driving

2022

View full text Add to dashboard Cite

Safety assurance of automated driving systems must consider uncertain environment perception. This paper reviews literature addressing how perception testing is realized as part of safety assurance. The paper focuses on testing for verification and validation purposes at the interface between perception and planning, and structures the analysis along the three axes (1) test criteria and metrics, (2) test scenarios, and (3) reference data. Furthermore, the analyzed literature includes related safety standards, safety-independent perception algorithm benchmarking, and sensor modeling. It is found that the realization of safety-oriented perception testing remains an open issue since challenges concerning the three testing axes and their interdependencies currently do not appear to be sufficiently solved.

show abstract

Section: Uncertainty In Reference Datamentioning

confidence: 99%

A Review of Testing Object-Based Environment Perception for Safe Automated Driving

2022

View full text Add to dashboard Cite

show abstract

“…The counts of TP and FP were used in [113] to evaluate their proposed probabilistic object detector. TP and FP scores were also used to construct the Receiver Operating Characteristic (ROC) curves in [6], [116]. Based on TP, FP, and FN, the evaluation metrics Precision, Recall, and the F1-Score can be derived, given by:…”

Section: Evaluation Metricsmentioning

confidence: 99%

“…In fact, if two probabilistic object detectors predict bounding boxes with the same mean values but wildly different covariance matrices, they will have the same mAP performance. Nevertheless, the majority of recent work on probabilistic object detection [8], [11], [13]- [16], [114], [116]- [118] still use mAP as the only metric to provide a quantitative assessment of their proposed methods, emphasizing a secondary effect of accuracy improvement when integrating probabilistic detection methods, instead of focusing on the correctness of the output distribution. This leads to the need of better and more consistent metrics, which we highlight in Sec.…”

Section: Evaluation Metricsmentioning

confidence: 99%

“…5) Jaccard IOU: Recently, Wang et al [116] propose Jaccard IOU (JIOU) as a probabilistic generalization of IOU. Unlike IOU which simply compares the (deterministic) geometric overlap between two bounding boxes, JIOU measures the similarity of their spatial distributions.…”

Section: ) Minimum Uncertainty Errormentioning

confidence: 99%

“…Unlike IOU which simply compares the (deterministic) geometric overlap between two bounding boxes, JIOU measures the similarity of their spatial distributions. Such distributions can be predicted by probabilistic object detection networks, or by inferring the uncertainties inherent in ground truth labels (which serve as the "references of probability distribution") [116]. In fact, JIOU simplifies to IOU when bounding boxes are assumed to follow simple uniform distributions.…”

Section: ) Minimum Uncertainty Errormentioning

confidence: 99%

See 2 more Smart Citations

A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving

Feng,

Harakeh,

Waslander

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Capturing uncertainty in object detection is indispensable for safe autonomous driving. In recent years, deep learning has become the de-facto approach for object detection, and many probabilistic object detectors have been proposed. However, there is no summary on uncertainty estimation in deep object detection, and existing methods are not only built with different network architectures and uncertainty estimation methods, but also evaluated on different datasets with a wide range of evaluation metrics. As a result, a comparison among methods remains challenging, as does the selection of a model that best suits a particular application. This paper aims to alleviate this problem by providing a review and comparative study on existing probabilistic object detection methods for autonomous driving applications. First, we provide an overview of generic uncertainty estimation in deep learning, and then systematically survey existing methods and evaluation metrics for probabilistic object detection. Next, we present a strict comparative study for probabilistic object detection based on an image detector and three public autonomous driving datasets. Finally, we present a discussion of the remaining challenges and future works. Code has been made available at https://github.com/asharakeh/pod_ compare.git.

show abstract

Probabilistic Object Detection with an Ensemble of Experts

Miller

2020

Computer Vision – ECCV 2020 Workshops

View full text Add to dashboard Cite

Inferring Spatial Uncertainty in Object Detection

Cited by 22 publications

References 18 publications

A Review of Testing Object-Based Environment Perception for Safe Automated Driving

A Review of Testing Object-Based Environment Perception for Safe Automated Driving

A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving

Probabilistic Object Detection with an Ensemble of Experts

Contact Info

Product

Resources

About