Masksembles for Uncertainty Estimation

Abstract

3D object detection is an essential task for computer vision applications in autonomous vehicles and robotics. However, models often struggle to quantify detection reliability, leading to poor performance on unfamiliar scenes. We introduce a framework for quantifying uncertainty in 3D object detection by leveraging an evidential learning loss on Bird's Eye View representations in the 3D detector. These uncertainty estimates require minimal computational overhead and are generalizable across different architectures. We demonstrate both the efficacy and importance of these uncertainty estimates on identifying out-of-distribution scenes, poorly localized objects, and missing (false negative) detections; our framework consistently improves over baselines by 10-20% on average. Finally, we integrate this suite of tasks into a system where a 3D object detector auto-labels driving scenes and our uncertainty estimates verify label correctness before the labels are used to train a second model. Here, our uncertainty-driven verification results in a 1% improvement in mAP and a 1-2% improvement in NDS.

Leveraging Self-Supervision for Cross-Domain Crowd Counting

“…We propose a fully unsupervised approach to fine-tuning a network that has been trained on annotated synthetic data, so that it can operate effectively on real data despite a potentially large domain shift. At the heart of our method is a network that estimates people-density at every location while incorporating a variant of the deep ensemble approach [13] to provide uncertainties about these. The key to success is to first pre-train this network so that these uncertainties are meaningful and then to exploit them to recursively fine-tune the network.…”

“…However, they require training many different copies of the network, which can be very slow and memory consuming. Instead, we rely on Masksembles, a recent approach [13] that operates on the same basic principle as MC-Dropout. However, instead of achieving randomness by dropping different subsets of weights for each observed sample, it relies on a set of precomputed binary masks that specify the network parameters to be dropped.…”

“…Uncertainty Estimation. We use Masksembles [13] for uncertainty estimation because of its effectiveness and simplicity. However we could also have used MC-Dropout [15] Model Table 6.…”

“…As can be seen in Tab. 6, using Masksembles [13] as in OURS clearly outperform MC-Dropout [15] and is comparable with Deep Ensembles [32]. However, training Ensembles takes three times longer, which motivates our use of Masksembles.…”

“…In this paper, we remedy this shortcoming by training with both synthetic images, along with their associated labels, and unlabeled real images. We force our network to learn perspective-aware features on the real images and build into it the ability to use these features to predict its own uncertainty using a fast variant of the ensemble method [13] to effectively use pseudo labels for fine-tuning. We train it as follows:…”