End-to-end information fusion method for transformer-based stereo matching

Abstract

In stereo matching, the application of transformers can overcome the limitations of disparity range and capture long-range matching information. However, the lack of cross-epipolar context information often leads to numerous mismatches, especially in low-texture regions. An end-to-end information fusion stereo matching method is proposed to address this issue. In the proposed method, a feature extraction method that combines dense connections and a residual block is proposed. Global and local semantic information can be effectively fused by incorporating dense connections among multiscale feature maps. Additionally, the inclusion of a residual block helps extract more representative feature maps. The idea of criss-cross attention is introduced in the transformer implicit matching process. Criss-cross attention enables the capture of cross-epipolar context information by combining horizontal and vertical attention mechanisms. This method improves the matching accuracy from the perspective of multi-path information fusion. According to the matching resluts, the disparity regression layer and the context adjustment layer are used to generate the initial and final disparity maps, respectively. The proposed method is evaluated on the Scene Flow, KITTI 2012, and Middlebury 2014 datasets. Experimental results indicate that the proposed method effectively enhances matching accuracy. Moreover, the proposed method exhibits strong generalization ability, allowing for direct application to synthetic, real outdoor, and real indoor scene images.

A cascaded GRU-based stereoscopic matching network for precise plank measurement

“…Inspired by the rapid advancements in deep learning, researchers have come to recognize that stereo matching methods based on deep learning have the potential to surpass the limitations of traditional approaches through training on specialized data for specific scenarios [13][14][15][16]. It exhibits remarkable adaptability and robustness in addressing diverse challenges, including variations in lighting conditions, textureless region, and interference of noise.…”