Research of Hand–Eye System with 3D Vision towards Flexible Assembly Application

Liang, Peidong; Lin, Wen‐Wei; Luo, Guantai; Zhang, Chentao

doi:10.3390/electronics11030354

Cited by 11 publications

(4 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In such scenarios, it is also often not feasible to perform complicated calibration procedures, as they increase the time and resource requirements in environments where the resources are constrained. Use cases where the system greatly benefits from the detachment of the scanner from the robot are applications such as robot-assisted surgical procedures [ 32 , 33 , 34 ], automatic robotic assembly [ 35 ], and object grasping [ 36 ].…”

Section: Methods and Experimental Setupmentioning

confidence: 99%

Submillimeter-Accurate Markerless Hand–Eye Calibration Based on a Robot’s Flange Features

Đalić,

Jovanović,

Marić

2024

Sensors

View full text Add to dashboard Cite

An accurate and reliable estimation of the transformation matrix between an optical sensor and a robot is a key aspect of the hand–eye system calibration process in vision-guided robotic applications. This paper presents a novel approach to markerless hand–eye calibration that achieves streamlined, flexible, and highly accurate results, even without error compensation. The calibration procedure is mainly based on using the robot’s tool center point (TCP) as the reference point. The TCP coordinate estimation is based on the robot’s flange point cloud, considering its geometrical features. A mathematical model streamlining the conventional marker-based hand–eye calibration is derived. Furthermore, a novel algorithm for the automatic estimation of the flange’s geometric features from its point cloud, based on a 3D circle fitting, the least square method, and a nearest neighbor (NN) approach, is proposed. The accuracy of the proposed algorithm is validated using a calibration setting ring as the ground truth. Furthermore, to establish the minimal required number and configuration of calibration points, the impact of the number and the selection of the unique robot’s flange positions on the calibration accuracy is investigated and validated by real-world experiments. Our experimental findings strongly indicate that our hand–eye system, employing the proposed algorithm, enables the estimation of the transformation between the robot and the 3D scanner with submillimeter accuracy, even when using the minimum of four non-coplanar points for calibration. Our approach improves the calibration accuracy by approximately four times compared to the state of the art, while eliminating the need for error compensation. Moreover, our calibration approach reduces the required number of the robot’s flange positions by approximately 40%, and even more if the calibration procedure utilizes just four properly selected flange positions. The presented findings introduce a more efficient hand–eye calibration procedure, offering a superior simplicity of implementation and increased precision in various robotic applications.

show abstract

Section: Methods and Experimental Setupmentioning

confidence: 99%

Submillimeter-Accurate Markerless Hand–Eye Calibration Based on a Robot’s Flange Features

Đalić,

Jovanović,

Marić

2024

Sensors

View full text Add to dashboard Cite

show abstract

“…In depth estimation and optimization models based on U-Net have achieved significant breakthroughs. Lin et al [30] constructed an attention feature fusion module based on the original U-Net, which is called AFFM and consists of channel and spatial attention modules. The improved U-Net model increased the segmentation accuracy of the image.…”

Section: U-netmentioning

confidence: 99%

SAU-Net: Monocular Depth Estimation Combining Multi-Scale Features and Attention Mechanisms

Zhao,

Song,

Wang

2023

IEEE Access

View full text Add to dashboard Cite

Monocular depth estimation technology is widely utilized in autonomous driving for sensing and obstacle avoidance. Recent advancements in deep-learning techniques have resulted in significant progress in monocular depth estimation. However, monocular depth estimation is mainly optimized for the luminosity error of pixels, mostly disregarding the related problems of result ambiguity and boundary artifacts in the image. To address these issues, we developed an improved network model called SAU-Net. The superposition of excessive convolutional layers in conventional convolution networks impairs the network's timeliness and results in the loss of primary information. Therefore, we propose a convolutionfree stratified transformer as an image feature extractor at the network's coding end, which limits selfattention to innumerable windows and leverages sliding windows for characterization to reduce the network delay. This study also addresses the issue of critical information loss. We connect each feature map directly to another from a different scale. In addition, an attention module is introduced to focus on the effective features, which increases the amount of target information in the depth map. We employ the gradient loss function during the training stage to improve the segmentation accuracy of the network and the smoothness of the output image. Training and testing were conducted using the KITTI dataset. To ensure the robustness of the algorithm in practical applications, we also validated the algorithm using a campus dataset that we collected. The experimental results indicated that the accuracy of the algorithm was 89.1%, 96.4%, and 98.5% under three proportional thresholds. The estimated depth map was clear in details and edges, with fewer artifacts.INDEX TERMS autonomous driving, monocular depth estimation, SAU-net, stratified transformer.

show abstract

“…Without the application of industrial robots, it is impossible to accomplish automation and modernization of a manufacturing process in any industrial branch. Due to their strong adaptability and flexibility, industrial robots are commonly assigned for dull, dangerous, and unpleasant jobs to replace human beings, including automatic spray painting [1], welding [2], grinding [3], logistics sorting [4], assembling [5] and so on.…”

Section: Introductionmentioning

confidence: 99%

“…However, 3D segmentation is still one of the most challenging tasks in computer vision. The results of 3D segmentation will directly affect object recognition [6], pose estimation [7] and positioning [5] in the application of industrial robots. The goal of the segmentation process is to group points that belong to the same objects into clusters or sets, where each cluster has similar properties by some criteria.…”

Section: Introductionmentioning

confidence: 99%

An Improved Supervoxel Clustering Algorithm of 3D Point Clouds for the Localization of Industrial Robots

Xie

Liang²,

Tao

et al. 2022

Electronics

Self Cite

View full text Add to dashboard Cite

Supervoxels have a widespread application of instance segmentation on account of the merit of providing a highly approximate representation with fewer data. However, low accuracy, mainly caused by point cloud adhesion in the localization of industrial robots, is a crucial issue. An improved bottom-up clustering method based on supervoxels was proposed for better accuracy. Firstly, point cloud data were preprocessed to eliminate the noise points and background. Then, improved supervoxel over-segmentation with moving least squares (MLS) surface fitting was employed to segment the point clouds of workpieces into supervoxel clusters. Every supervoxel cluster can be refined by MLS surface fitting, which reduces the occurrence that over-segmentation divides the point clouds of two objects into a patch. Additionally, an adaptive merging algorithm based on fusion features and convexity judgment was proposed to accomplish the clustering of the individual workpiece. An experimental platform was set up to verify the proposed method. The experimental results showed that the recognition accuracy and the recognition rate in three different kinds of workpieces were all over 0.980 and 0.935, respectively. Combined with the sample consensus initial alignment (SAC-IA) coarse registration and iterative closest point (ICP) fine registration, the coarse-to-fine strategy was adopted to obtain the location of the segmented workpieces in the experiments. The experimental results demonstrate that the proposed clustering algorithm can accomplish the localization of industrial robots with higher accuracy and lower registration time.

show abstract

Research of Hand–Eye System with 3D Vision towards Flexible Assembly Application

Cited by 11 publications

References 35 publications

Submillimeter-Accurate Markerless Hand–Eye Calibration Based on a Robot’s Flange Features

Submillimeter-Accurate Markerless Hand–Eye Calibration Based on a Robot’s Flange Features

SAU-Net: Monocular Depth Estimation Combining Multi-Scale Features and Attention Mechanisms

An Improved Supervoxel Clustering Algorithm of 3D Point Clouds for the Localization of Industrial Robots

Contact Info

Product

Resources

About