Semi-Dense Stereo Matching Using Dual CNNs

Although many algorithms perform very well on certain datasets, existing stereo matching algorithms still fail to obtain ideal disparity images with high precision in practical robotic applications with weak or untextured objects. This greatly limits the application of binocular vision for robotic arm guidance. Traditional stereo matching algorithms suffer from disparity loss, dilation and other problems, and deep learning algorithms have weakly generalization ability, making high‐accuracy results impossible with non‐training images. We propose an algorithm that uses segments and edges as matching units. We find the mapping relationship between two‐dimensional images and three‐dimensional scenes using segments. The algorithm obtains highly accurate results in industrial robotic applications with mainly planar facets. We combine it with a deep learning algorithm to obtain very good high‐accuracy results in both general scenes and applications of industrial robots. The algorithm effectively improves the non‐linear optimization ability of traditional algorithms and generalization ability of deep learning, and provides an effective method for the binocular vision guidance of industrial robotic scenes. We used the algorithm to guide the robot arm for threading with a success rate of 70%.

show abstract

“…Mao et al . upgraded the CNN network [MWZG19] and achieved semi‐dense stereo matching using dual CNNs. Guo et al .…”

Section: Related Workmentioning

confidence: 99%

A Stereo Matching Algorithm for High‐Precision Guidance in a Weakly Textured Industrial Robot Environment Dominated by Planar Facets

Wei

Ling-jiang

2022

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…The first end-to-end deep stereo matching approach, DispNet [21], produces a shared dataset, which remains essential for pre-training deep learning algorithms until now. Mao et al upgraded the CNN network [22] and achieved semi-dense stereo matching using dual CNNs. Guo et al [23] proposed construction of the cost volume by group-wise correlation to enhance performance.…”

Section: Related Workmentioning

confidence: 99%

“…Mao et al. upgraded the CNN network [22] and achieved semi‐dense stereo matching using dual CNNs. Guo et al.…”

Section: Related Workmentioning

confidence: 99%

A binocular reconstruction based on perspective projection constraints and its application on robot eye‐hand coordination

Wei

Ling-jiang

2022

IET Computer Vision

View full text Add to dashboard Cite

Stereo matching algorithms have been developed for many years but basically focus only on the implementation of existing datasets and are rarely applied to real scenarios, such as industrial robot scenarios. Traditional stereo matching algorithms have a high error rate, and deep learning algorithms are difficult to obtain good results in real scenarios because of their weak generalisation ability and difficult access to training data. In order to use stereo matching algorithms for industrial robot guidance, it is better to design a new traditional algorithm with low time complexity for the characteristics of industrial robot scenarios dominated by planar facets. This paper proposes a new matching method based on subrows of pixels, instead of individual pixels, in order to improve robustness of matching and reduce running time. First, the pixel strings from the same row of the left and right images are divided into several colour-identical or colour-gradient segments. Then, the colour and length of the two left and right pixel segment are used as clues to determine a matching relation and obtain the matching type. Then, all match types can be determined according to non-crossing mapping. Each match type can reason backward to the corresponding spatial state of the stimulus source so that the disparity of pixels in pixel segments representing the spatial state can be calculated. This new matching method makes full use of the stimulus homology constraints and projective geometric constraints of row-aligned images. The method can obtain good results in industrial robot scenarios and be applied for industrial robot guidance.

show abstract

“…Mao et al [18] have presented a robust deep learning solution for semidense stereo matching. In this literature, two CNN models are utilized for computing stereo matching cost and performing confidence-based filtering.…”

Section: Related Workmentioning

confidence: 99%

A Road Environment Prediction System for Intelligent Vehicle

Sun

Pei

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

The road environment prediction is an essential task for intelligent vehicle. In this study, we provide a flexible system that focuses on freespace detection and road environment prediction to host vehicle. The hardware of this system includes two parts: a binocular camera and a low-power mobile platform, which is flexible and portable for a variety of intelligent vehicle. We put forward a multiscale stereo matching algorithm to reduce the computing cost of the hardware unit. Based on disparity space and points cloud, we propose a weighted probability grid map to detect freespace region and a state model to describe the road environment. The experiments show that the proposed system is accurate and robust, which indicates that this technique is fully competent for road environment prediction for intelligent vehicle.

show abstract

Semi-Dense Stereo Matching Using Dual CNNs

Cited by 6 publications

References 43 publications

A Stereo Matching Algorithm for High‐Precision Guidance in a Weakly Textured Industrial Robot Environment Dominated by Planar Facets

A Stereo Matching Algorithm for High‐Precision Guidance in a Weakly Textured Industrial Robot Environment Dominated by Planar Facets

A binocular reconstruction based on perspective projection constraints and its application on robot eye‐hand coordination

A Road Environment Prediction System for Intelligent Vehicle

Contact Info

Product

Resources

About