Accurate Rapid Grasping of Small Industrial Parts from Charging Tray in Clutter Scenes

Abstract

The rapid detection and fine pose estimation of textureless objects in red-green-blue and depth (RGB-D) images are challenging tasks, especially for small dark industrial parts on the production line in clutter scenes. In this paper, a novel practical method based on an RGB-D sensor, which includes 3D object segmentation and 6D pose estimation, is proposed. At the 3D object segmentation stage, 3D virtual and detected bounding boxes are combined to segment 3D scene point clouds. The 3D virtual bounding boxes are determined from prior information on the parts and charging tray, and the 3D detected bounding boxes are obtained from the 2D detected bounding boxes in part detection based on a Single Shot MultiBox Detector (SSD) network in an RGB image. At the 6D pose estimation stage, the coarse pose is estimated by fitting the central axis of the part from the observed 3D point clouds accompanied by a lot of noise, and then refined with part model point clouds by using the iterative closest point (ICP) algorithm. The proposed method has been successfully applied to robotic grasping on the industrial production line with a customer-leverldepth camera. The results verified that grasping speed reaches the subsecond level and that grasping accuracy reaches the millimeter level. The stability and robustness of the automation system meet the production requirement.

Object pose estimation based on stereo vision with improvedK‐DtreeICPalgorithm

“…Three translational and 3 rotational angles in total are used to ensure that the robot can grasp the target object for rigid objects. [19][20][21][22][23] A pose estimation system includes data acquisition, preprocessing, feature extraction, feature matching, refined processing and so forth. 1,[24][25][26][27] Therefore, vision-based object pose estimation has become a difficult problem to be overcome in robot grasping tasks.…”

“…Real‐time accurate position and attitude information of the target object need to be obtained before robot grasping. Three translational and 3 rotational angles in total are used to ensure that the robot can grasp the target object for rigid objects 19‐23 . A pose estimation system includes data acquisition, preprocessing, feature extraction, feature matching, refined processing and so forth 1,24‐27 .…”

Manipulated-Object Detection and Pose Estimation Based on Multimodal Feature Points and Neighboring Patches

“…In recent years, methods based on convolutional neural networks (CNNs) have shown significant advantages in tasks of object detection and pose estimation, which benefit from their ability to automatically learn features from raw images. (4)(5)(6) However, for new user-defined objects, hundreds of new samples together with their ground-truth poses are needed to retrain CNNs. This is usually very inconvenient in practical application scenarios, where various and changing objects are commonly involved.…”