CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching

Abstract

Due to the domain differences and unbalanced disparity distribution across multiple datasets, current stereo matching approaches are commonly limited to a specific dataset and generalize poorly to others. Such domain shift issue is usually addressed by substantial adaptation on costly target-domain ground-truth data, which cannot be easily obtained in practical settings. In this paper, we propose to dig into uncertainty estimation for robust stereo matching. Specifically, to balance the disparity distribution, we employ a pixel-level uncertainty estimation to adaptively adjust the next stage disparity searching space, in this way driving the network progressively prune out the space of unlikely correspondences. Then, to solve the limited ground truth data, an uncertainty-based pseudo-label is proposed to adapt the pre-trained model to the new domain, where pixel-level and area-level uncertainty estimation are proposed to filter out the high-uncertainty pixels of predicted disparity maps and generate sparse while reliable pseudo-labels to align the domain gap. Experimentally, our method shows strong cross-domain, adapt, and joint generalization and obtains 1st place on the stereo task of Robust Vision Challenge 2020. Additionally, our uncertainty-based pseudo-labels can be extended to train monocular depth estimation networks in an unsupervised way and even achieves comparable performance with the supervised methods. The code will be available at https://github.com/gallenszl/UCFNet.

Attention Concatenation Volume for Accurate and Efficient Stereo Matching

“…ETH3D. As shown in Table 5, our ACVNet outperforms the state-of-the-art methods, HITNet [18] and CFNet [16].…”

“…Specifically, we denote the model after applying our method as GwcNet-ACV, PSMNet-ACV and CFNet-ACV for comparison respectively. As shown in is reduced by 39.5% for GwcNet [7], 42.2% for PSMNet [2] and 14.4% for CFNet [16]. We experimentally compare our ACV with cascaded approaches.…”

“…Following the 4D combined cost volume, cascade cost volumes [6,16,19] further reduce the memory and computational complexity of cost volume construction by building a cost volume pyramid in a coarse-to-fine manner to progressively narrow down the target disparity range and refine the depth map. However, such coarse-to-fine strategy inevitably involves accumulated errors, i.e.…”

“…mp denotes multi-level adaptive patch (Figure 3) which has different rates and adaptive weights for three-level feature maps. Model D1 (%) EPE (px) PSMNet [2] 3.89 1.09 PSMNet-ACV 2.17 0.63 GwcNet [7] 2.71 0.76 GwcNet-ACV 1.55 0.46 CFNet [16] 4.51 0.97 CFNet-ACV 4.02 0.83 Table 3. Universality study of ACV on Scene Flow [12].…”

“…To demonstrate the universality of our ACV, we integrate our ACV into three state-of-the-art models, i.e. Gwc-Net [7], PSMNet [2] and CFNet [16], and compare the performance of the original models with those after using our ACV. Specifically, we denote the model after applying our method as GwcNet-ACV, PSMNet-ACV and CFNet-ACV for comparison respectively.…”