Confidence Estimation for Superpixel-Based Stereo Matching

Abstract

In this paper we propose an approach for estimating the confidence of stereo matches for superpixel-based disparity estimation. To our knowledge, this is the first such method reported in the literature. Starting from a simple superpixel stereo algorithm, we present a representative set of features that can be extracted from the disparity map and the superpixel fitting process. A random forest classifier is then trained on these features to predict whether the disparity assigned to each pixel of a test disparity map is correct or not. We perform experiments on the KITTI stereo benchmark and show that our confidence estimator is very accurate in predicting which disparities are correct and which are not. We also present a post-processing algorithm for improving the accuracy of the disparity maps that exploits the confidence estimates to reject wrong disparity values and achieves significant error reduction.

Ambiguity Concept in Stereo Matching Pipeline

“…This set of metrics has been completed by Poggi (Poggi et al, 2017) especially with machine learning-based metrics. In ensemble learning-based approaches, confidence metrics are estimated through random forests (Haeusler et al, 2013;Spyropoulos et al, 2014;Min-Gyu Park and Yoon, 2015;Gouveia et al, 2015). The features used for random forests correspond to a selection of confidence metrics mostly defined in (Hu and Mordohai, 2012).…”

CBMV: A Coalesced Bidirectional Matching Volume for Disparity Estimation

“…This changed recently with publications such as the one by Haeusler et al [10] who train a random forest to predict the correctness of the output disparities of the SGM algorithm [12] using features computed on the images, disparity maps and matching cost volume. Gouveia et al [9] extend the confidence estimator of [40] to be applicable to a superpixel-based stereo algorithm. The classifier is able to remove errors from the disparity maps, which are filled in using conventional techniques.…”

“…The same authors [32] improved a number of previous methods by training a CNN to refine confidence maps. The classifier's predictions in all cases [9,10,31,32] are effective in sparsifying the disparity maps by removing potential errors, but do not help in the generation of more accurate disparity estimates.…”

Dual Guided Aggregation Network for Stereo Image Matching

“…Taking advantage of disparity information from the left-to-right and right-to-left mutual disparity curves is an effective strategy to reduce the number of mismatched pixels. Applying a learningbased method to predict the correctness of output disparities [15,16] is another way to remove mismatched pixels. However, the traditional left-right consistency check and the learning-based disparity correctness prediction address the removal of mismatched pixels rather than the improvement of disparity estimation accuracy.…”