A vision-based robotic grasping system using deep learning for 3D object recognition and pose estimation

Abstract

Pose estimation of object is one of the key problems for the automatic-grasping task of robotics. In this paper, we present a new vision-based robotic grasping system, which can not only recognize different objects but also estimate their poses by using a deep learning model, finally grasp them and move to a predefined destination. The deep learning model demonstrates strong power in learning hierarchical features which greatly facilitates the recognition mission. We apply the Max-pooling Convolutional Neural Network (MPCNN), one of the most popular deep learning models, in this system, and assign different poses of objects as different classes in MPCNN. Besides, a new object detection method is also presented to overcome the disadvantage of the deep learning model. We have built a database comprised of 5 objects with different poses and illuminations for experimental performance evaluation. The experimental results demonstrate that our system can achieve high accuracy on object recognition as well as pose estimation. And the vision-based robotic system can grasp objects successfully regardless of different poses and illuminations.

Sensors and systems for fruit detection and localization: A review

“…However, geometric parameters of the fruit such as location of calyx, outline or bounding box of fruit, and the symmetry of the shape could provide a way to estimate fruit orientation, and stem location and orientation. Investigation of deep learning models used in pose estimation (Yu et al, 2013;Liang et al, 2014) for application in outdoor environment may be another area of research with potential contribution to estimating fruit and stem orientation.…”

Label Fusion: A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenes

“…[1] and [2] are both state of the art pipelines for estimating object poses from RGBD images in clutter -both approaches use RGB pixelwise segmentation neural networks (trained on their datasets described in the previous section) to crop point clouds which are then fed into ICP-based algorithms to estimate object poses by registering against prior known meshes. Another approach is to directly learn pose estimation [11]. The upcoming SIXD Challenge 2017 [12] will provide a comparison of state of the art methods for 6DOF pose estimation on a common dataset.…”

A deep-learning based feature hybrid framework for spatiotemporal saliency detection inside videos

“…where ) tanh( represents the hyperbolic tangent function; i P and i Q are respectively the width and height of the convolutional kernel; m is the index of feature maps in the optimizedthrough back-propagation (BP) algorithm. As for the max-pooling layers, their purpose is to achieve spatial invariance by reducing the resolution of the feature maps [90]. Meanwhile, max-pooling reduces the computational complexity of upper layers by selecting superior invariant features.…”