A Stereo Matching Algorithm for High‐Precision Guidance in a Weakly Textured Industrial Robot Environment Dominated by Planar Facets

Abstract

Although many algorithms perform very well on certain datasets, existing stereo matching algorithms still fail to obtain ideal disparity images with high precision in practical robotic applications with weak or untextured objects. This greatly limits the application of binocular vision for robotic arm guidance. Traditional stereo matching algorithms suffer from disparity loss, dilation and other problems, and deep learning algorithms have weakly generalization ability, making high‐accuracy results impossible with non‐training images. We propose an algorithm that uses segments and edges as matching units. We find the mapping relationship between two‐dimensional images and three‐dimensional scenes using segments. The algorithm obtains highly accurate results in industrial robotic applications with mainly planar facets. We combine it with a deep learning algorithm to obtain very good high‐accuracy results in both general scenes and applications of industrial robots. The algorithm effectively improves the non‐linear optimization ability of traditional algorithms and generalization ability of deep learning, and provides an effective method for the binocular vision guidance of industrial robotic scenes. We used the algorithm to guide the robot arm for threading with a success rate of 70%.

Robot grasping detection technology based on 3D structured light

“…The calibration data shown in Table 1 can be obtained. According to the hand eye calibration data in Table 1, the hand eye calibration matrix shown in the Formula ( 7) can be calculated by the hand eye calibration method [3], that is, the conversion relationship between the 3D camera and the flange end of the robot. Through inverse calculation of hand eye calibration matrix, the error of hand eye calibration can be obtained, with an average error of 0.12mm.…”

“…Robot grasping detection technology [1] is to obtain all or local features of the product through measuring sensors, obtain its deviation from the teaching grasping position, and then feed back to the robot to achieve the normal grasping of objects. In the engineering application, the robot end measurement sensor mostly adopts monocular vision [2], and 3D vision [2][3][4]. The monocular vision mode [2] has a small depth of field, which can obtain the plane features of the product, but it is unable to obtain the depth information of the product or the depth information accuracy is low, which is suitable for the case of plane loading and small height change.…”

“…The monocular vision mode [2] has a small depth of field, which can obtain the plane features of the product, but it is unable to obtain the depth information of the product or the depth information accuracy is low, which is suitable for the case of plane loading and small height change. 3D vision can also be divided into binocular vision [3] and structured light [4][5][6][7][8]. Binocular vision [3] is applicable to products with rich surface features for feature matching, but cannot be applied to products with few or no surface textures.…”

“…3D vision can also be divided into binocular vision [3] and structured light [4][5][6][7][8]. Binocular vision [3] is applicable to products with rich surface features for feature matching, but cannot be applied to products with few or no surface textures. 3D structured light [4] is to project the light fringe pattern with coding information onto the product surface, and obtain the pattern modulated on the surface by the camera, and decode the pattern to calculte the 3D point cloud morphology of the product.…”

A target-free stereo matching method of 3D vision-based for measuring composite fiber bundles

“…A significant advantage of this method is its ability to measure any fiber bundle within the field of view of a stereo camera without restrictions on the number or position of fiber bundles, all while enabling real-time comprehensive detection. The core of 3D visual reconstruction technology lies in stereo matching algorithms, and any errors or invalid matching points between stereo images can impact the accuracy of 3D reconstruction [8,9,10]. The accuracy of traditional stereo matching algorithms heavily relies on the feature completeness of image points.…”