AutoDispNet: Improving Disparity Estimation With AutoML

Saikia, Tonmoy; Marrakchi, Yassine; Zela, Arbër; Hutter, Frank; Brox, Thomas

doi:10.1109/iccv.2019.00190

Cited by 73 publications

(75 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired from the semantic segmentation task , recent methods (Mayer et al 2016;Kendall et al 2017;Saikia et al 2019;Cheng et al 2020) adopt fully-convolutional networks to regress disparity maps, which further improves the accuracy and effi-ciency of stereo matching. Concretely, current end-to-end stereo matching networks can be roughly categorized into two types: correlation-based 2-D stereo networks and costvolume based 3-D stereo networks.…”

Section: Stereo Matchingmentioning

confidence: 99%

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

et al. 2022

View full text Add to dashboard Cite

Recently, records on stereo matching benchmarks are constantly broken by end-to-end disparity networks. However, the domain adaptation ability of these deep models is quite limited. Addressing such problem, we present a novel domain-adaptive approach called AdaStereo that aims to align multi-level representations for deep stereo matching networks. Compared to previous methods, our AdaStereo realizes a more standard, complete and effective domain adaptation pipeline. Firstly, we propose a non-adversarial progressive color transfer algorithm for input image-level alignment. Secondly, we design an efficient parameter-free cost normalization layer for internal feature-level alignment. Lastly, a highly related auxiliary task, self-supervised occlusion-aware reconstruction is presented to narrow the gaps in output space. We perform intensive ablation studies and break-down comparisons to validate the effectiveness of each proposed module. With no extra inference overhead and only a slight increase in training complexity, our AdaStereo models achieve state-of-the-art cross-domain performance on multiple benchmarks, including KITTI, Middlebury, ETH3D and DrivingStereo, even outperforming some state-of-the-art disparity networks finetuned with target-domain ground-truths. Moreover, based on two additional evaluation metrics, the superiority of our domain-adaptive stereo matching pipeline is further uncovered from more perspectives. Finally, we demonstrate that our method is robust to various domain adaptation settings, and can be easily integrated into quick adaptation application scenarios and real-world deployments.

show abstract

Section: Stereo Matchingmentioning

confidence: 99%

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Inspired from the semantic segmentation task [49,8], recent methods [55,39,66,13] adopt fully-convolutional networks to regress disparity maps, which further improves the accuracy and efficiency of stereo matching. Concretely, current end-to-end stereo matching networks can be roughly categorized into two types: correlation-based 2-D stereo networks and cost-volume based 3-D stereo networks.…”

Section: Stereo Matchingmentioning

confidence: 99%

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

Song¹,

Yang²,

Zhu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, records on stereo matching benchmarks are constantly broken by end-to-end disparity networks. However, the domain adaptation ability of these deep models is quite limited. Addressing such problem, we present a novel domain-adaptive approach called AdaStereo that aims to align multi-level representations for deep stereo matching networks. Compared to previous methods, our AdaStereo realizes a more standard, complete and effective domain adaptation pipeline. Firstly, we propose a non-adversarial progressive color transfer algorithm for input image-level alignment. Secondly, we design an efficient parameter-free cost normalization layer for internal feature-level alignment. Lastly, a highly related auxiliary task, self-supervised occlusionaware reconstruction is presented to narrow the gaps in output space. We perform intensive ablation studies and breakdown comparisons to validate the effectiveness of each proposed module. With no extra inference overhead and only a slight increase in training complexity, our AdaStereo models achieve state-of-the-art cross-domain performance on multiple benchmarks, including KITTI, Middlebury, ETH3D and DrivingStereo, even outperforming some state-of-the-art disparity networks finetuned with target-domain ground-truths. Moreover, based on two additional evaluation metrics, the superiority of our domain-adaptive stereo matching pipeline is further uncovered from more perspectives. Finally, we demonstrate that our method is robust to various domain adaptation settings, and can be easily integrated into quick adaptation application scenarios and real-world deployments.

show abstract

“…The early studies mainly focus on optimizing the existing network architectures by enormous hands-on trial-and-error tweaking efforts. Besides, recent studies also leverage multi-task learning [33]- [35] to combine other prior vision information and NAS-based methods [11], [27] to tweak the network structure as well as the operator hyper-parameters (i.e., kernel size and channel number for the convolution layer). According to the basic operator (related to the computational efficiency) and the network pipeline, we mainly discuss two branches of network structures for disparity estimation, the ED-Conv2D series and the CVM-Conv3D series.…”

Section: Related Workmentioning

confidence: 99%

“…The architectures of DNN are very essential to achieve accurate estimation, and can be categorized into two classes, the encoder-decoder network with 2D convolution (ED-Conv2D) and the cost volume matching with 3D convolution (CVM-Conv3D). Besides, recent studies [11], [12] begin to reveal the potential of automated machine learning (AutoML) for neural architecture search (NAS) on stereo matching. In practice, to measure whether a DNN model is applicable in real-world applications, we not only need to evaluate its accuracy on unseen stereo images (whether it can estimate the disparity correctly), but also need to evaluate its time efficiency (whether it can generate the results in real-time).…”

mentioning

confidence: 99%

“…To ease the human efforts of designing an efficient network structure for stereo matching, some recent studies [11], [27] also take advantages of automated machine learning (AutoML) [12], especially the neural architecture search (NAS) technique, to search the optimal set of network operators as well as their connections. However, those state-of-the-art methods are still far from real-time inference even on a server GPU since they are still based on either the complicated network stacking or low-efficient 3D convolution operations.…”

mentioning

confidence: 99%

See 1 more Smart Citation

FADNet++: Real-Time and Accurate Disparity Estimation with Configurable Networks

Wang¹,

Shi²,

Zheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep neural networks (DNNs) have achieved great success in the area of computer vision. The disparity estimation problem tends to be addressed by DNNs which achieve much better prediction accuracy than traditional hand-crafted feature based methods. However, the existing DNNs hardly serve both efficient computation and rich expression capability, which makes them difficult for deployment in real-time and highquality applications, especially on mobile devices. To this end, we propose an efficient, accurate, and configurable deep network for disparity estimation named FADNet++. Leveraging several liberal network design and training techniques, FADNet++ can boost its accuracy with a fast model inference speed for realtime applications. Besides, it enables users to easily configure different sizes of models for balancing accuracy and inference efficiency. We conduct extensive experiments to demonstrate the effectiveness of FADNet++ on both synthetic and realistic datasets among six GPU devices varying from server to mobile platforms. Experimental results show that FADNet++ and its variants achieve state-of-the-art prediction accuracy, and run at a significant order of magnitude faster speed than existing 3D models. With the constraint of running at above 15 frames per second (FPS) on a mobile GPU, FADNet++ achieves a new stateof-the-art result for the SceneFlow dataset.

show abstract

AutoDispNet: Improving Disparity Estimation With AutoML

Cited by 73 publications

References 46 publications

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

FADNet++: Real-Time and Accurate Disparity Estimation with Configurable Networks

Contact Info

Product

Resources

About