AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

Abstract

Recently, records on stereo matching benchmarks are constantly broken by end-to-end disparity networks. However, the domain adaptation ability of these deep models is quite limited. Addressing such problem, we present a novel domain-adaptive approach called AdaStereo that aims to align multi-level representations for deep stereo matching networks. Compared to previous methods, our AdaStereo realizes a more standard, complete and effective domain adaptation pipeline. Firstly, we propose a non-adversarial progressive color transfer algorithm for input image-level alignment. Secondly, we design an efficient parameter-free cost normalization layer for internal feature-level alignment. Lastly, a highly related auxiliary task, self-supervised occlusion-aware reconstruction is presented to narrow the gaps in output space. We perform intensive ablation studies and break-down comparisons to validate the effectiveness of each proposed module. With no extra inference overhead and only a slight increase in training complexity, our AdaStereo models achieve state-of-the-art cross-domain performance on multiple benchmarks, including KITTI, Middlebury, ETH3D and DrivingStereo, even outperforming some state-of-the-art disparity networks finetuned with target-domain ground-truths. Moreover, based on two additional evaluation metrics, the superiority of our domain-adaptive stereo matching pipeline is further uncovered from more perspectives. Finally, we demonstrate that our method is robust to various domain adaptation settings, and can be easily integrated into quick adaptation application scenarios and real-world deployments.

CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching

“…Specifically, compared to our pre-trained model UCFNet pretrain, UCFNet adapt achieves 37.78%, 40.38%, 20%, 37.5% error reduction on KITTI2012, KITTI2015, Middlebury, and ETH3D, respectively. Moreover, compared to the current best-published domain adaptation method AdaStereo [52], [53], our method can still outperform it on three of four datasets, which further proves the effectiveness of the proposed method. Note that our method doesn't employ the non-adversarial progressive color transfer and cost normalization proposed in AdaStereo, thus, the performance of our method has the potential for further improvement.…”

“…Threshold δ of the ground truth uncertainty mask is 1. Asymmetric chromatic augmentation and asymmetric occlusion [64] [59] synthetic+TDD(no gt) 8.6 7.8 --ZOLE [38] synthetic+TDD(no gt) -6.8 --MADNet [57] synthetic+TDD(no gt) 9.3 8.5 --AdaStereo [52], [53] synthetic+TDD(no gt) epochs with a learning rate of 0.0001. The core idea of our threestage finetune strategy is to prevent the small datasets from being overwhelmed by large datasets.…”