Epistemic Uncertainty Aware Semantic Localization and Mapping for Inference and Belief Space Planning

Abstract

We investigate the problem of autonomous object classification and semantic SLAM, which in general exhibits a tight coupling between classification, metric SLAM and planning under uncertainty. We contribute a unified framework for inference and belief space planning (BSP) that addresses prominent sources of uncertainty in this context: classification aliasing (classifier cannot distinguish between candidate classes from certain viewpoints), classifier epistemic uncertainty (classifier receives data "far" from its training set), and localization uncertainty (camera and object poses are uncertain). Specifically, we develop two methods for maintaining a joint distribution over robot and object poses, and over posterior class probability vector that consider epistemic uncertainty in a Bayesian fashion. The first approach is Multi-Hybrid (MH), where multiple hybrid beliefs over poses and classes are maintained to approximate the joint belief over poses and posterior class probability. The second approach is Joint Lambda Pose (JLP), where the joint belief is maintained directly using a novel JLP factor. Furthermore, we extend both methods to BSP, planning while reasoning about future posterior epistemic uncertainty indirectly, or directly via a novel information-theoretic reward function. Both inference methods utilize a novel viewpointdependent classifier uncertainty model that leverages the coupling between poses and classification scores, and predicts the epistemic uncertainty from certain viewpoints. In addition, this model is used to generate predicted measurements during planning. To the best of our knowledge, this is the first work that reasons about classifier epistemic uncertainty within semantic SLAM and BSP. We evaluate extensively our inference and BSP approaches in simulation and using real data from the Active Vision Dataset. Results clearly indicate superior classification performance of our methods compared to an approach that is not epistemic uncertainty aware.

Hybrid Belief Pruning with Guarantees for Viewpoint-Dependent Semantic SLAM

“…where Σ = I M σ 2 s and σ s = 0.015. The same distribution is used in equation [17] as well. It is possible to interpret this distribution as the output of a classifier since the samples are vectors of positive elements that sum to 1.…”

“…Tchuiev and Indelman [16] developed a method for sequential reasoning about the posterior uncertainty of a semantic model. This method was later applied to autonomous planning scenarios [17]. Feldman and Indelman [18] replaced categorical class variables with latent continuous object representations.…”

“…(17) Since the normalized belief is required only for the probabilities of classes, the state will be marginalized later. Thus, we could apply the same expectation as in (17), and the normalization factor bg k will be eliminated. Until now, the derivation has been general and does not assume independent priors and therefore applies also to the dependent case.…”

“…Using (19), the computing complexity of the normalization factor is significantly reduced. Let us compare the running time of the naive approach in (17) and the efficient approach in (19). In both cases, integration of the state and calculation of ψ k (n, c, X k ) is required.…”

“…Consider that the expectation over X k is approximated by N s samples drawn from b g k [X k ]. Using the naive approach, one must sum the unnormalized probabilities for all hypotheses, as shown in (17). Therefore, the naive approach runs in O(M N • N s ).…”