Learned Collaborative Stereo Refinement

Abstract

In this work, we propose a learning-based method to denoise and refine disparity maps. The proposed variational network arises naturally from unrolling the iterates of a proximal gradient method applied to a variational energy defined in a joint disparity, color, and confidence image space. Our method allows to learn a robust collaborative regularizer leveraging the joint statistics of the color image, the confidence map and the disparity map. Due to the variational structure of our method, the individual steps can be easily visualized, thus enabling interpretability of the method. We can therefore provide interesting insights into how our method refines and denoises disparity maps. To this end, we can visualize and interpret the learned filters and activation functions and prove the increased reliability of the predicted pixel-wise confidence maps. Furthermore, the optimization based structure of our refinement module allows us to compute eigen disparity maps, which reveal structural properties of our refinement module. The efficiency of our method is demonstrated on the publicly available stereo benchmarks Middlebury 2014 and Kitti 2015.

An Image Stereo Matching Algorithm with Multi-Spectral Attention Mechanism

“…As the error threshold increased to 3 and 5, the reduction in mismatch rate became more significant, indicating an overall decrease in mismatch rate. (6) The qualitative comparison results on the KITTI2012 test dataset, using disparity prediction as the evaluation metric, are presented in Figure 11.…”

“…As the error threshold increased to 3 and 5, the reduction in mismatch rate became more significant, indicating an overall decrease in mismatch rate. (6) The qualitative comparison results on the KITTI2012 test dataset, using disparity prediction as the evaluation metric, are presented in Figure 11. From the figure, it can be observed that, compared to the PSMNet algorithm, the MANet algorithm predicted more continuous disparities in areas such as the sky and trees.…”

“…With the improvement of computing power and the increase in data volume, convolutional neural networks have brought more possibilities for solving the problem of stereo matching and have gradually become a research focus [5]. Deep learning techniques optimize algorithm performance through a large amount of image data [6], achieving optimal results by autonomously learning and optimizing representations, thereby improving the accuracy and robustness of stereo matching. The stereo matching algorithm based on deep learning extracts multiple features through multi-layer convolution for cost computation and uses regularization methods for cost aggregation to optimize the cost volume, thereby reducing the mismatch rate and improving the matching speed.…”