Group-Wise Correlation Stereo Network

Abstract

Stereo matching estimates the disparity between a rectified image pair, which is of great importance to depth sensing, autonomous driving, and other related tasks. Previous works built cost volumes with cross-correlation or concatenation of left and right features across all disparity levels, and then a 2D or 3D convolutional neural network is utilized to regress the disparity maps. In this paper, we propose to construct the cost volume by group-wise correlation. The left features and the right features are divided into groups along the channel dimension, and correlation maps are computed among each group to obtain multiple matching cost proposals, which are then packed into a cost volume. Group-wise correlation provides efficient representations for measuring feature similarities and will not lose too much information like full correlation. It also preserves better performance when reducing parameters compared with previous methods. The 3D stacked hourglass network proposed in previous works is improved to boost the performance and decrease the inference computational cost. Experiment results show that our method outperforms previous methods on Scene Flow, KITTI 2012, and KITTI 2015 datasets. The code is available at https: //github.com/xy-guo/GwcNet

Real-Time Self-Adaptive Deep Stereo

“…On Tab. 1 we report our result compared to other (published) fast inference architectures on the leaderboard (runtime measured on NVIDIA 1080Ti) as well as with a slower and more accurate one, GWCNet [11]. At the time of writing, our method ranks 90 th .…”

“…Although not achieving state-ofthe-art accuracy, this seminal work turned out quite disruptive compared to the traditional stereo paradigm outlined in [35], highlighting the potential for a totally new approach. Thereby, [21] ignited the spread of end-to-end stereo architectures [17,24,19,4,16,11] that quickly outmatched any other technique on the KITTI benchmarks by leveraging on a peculiar training protocol. In particular, the deep network is initially trained on a large amount of synthetic data with groundtruth labels [21] and then fine-tuned on the target domain (e.g., KITTI) based on stereo pairs with groundtruth.…”

Review of Stereo Matching Algorithms Based on Deep Learning

“…All pixels Non-occluded pixels Runtime (s) Environment D1-bg (%) D1-fg (%) D1-all (%) D1-bg (%) D1-fg (%) D1-all (%) PSMNet [28] 1 [31] and sparse technique [5], to address this problem, there is still a long way to go before a practical solution is developed. And of course, this kind of end-to-end stereo matching network needs corresponding ground truth depth data for training, which is a challenging problem.…”

“…Tons of algorithms based on this have been proposed. ese methods could roughly be categorized into two groups: 2D encode-decoder structures [23][24][25][26][27] and regularization modules composed of 3D convolutions [28][29][30][31]. DispNetC [24] computes a correlation volume from the left and right image features (encoding) and utilizes a CNN to directly regress (decoding) disparity maps.…”

Fast Stereo Disparity Maps Refinement By Fusion of Data-Based And Model-Based Estimations

“…Following this seminal work, [13] proposed an end-to-end architecture inspired by the classical stereo matching pipeline. Many works followed this trend [2,23,20,18,24,9,5] and completely outmatched previous methods in terms of accuracy. However, this gain in accuracy is at the cost of high computation requirement.…”