Bin-Picking for Planar Objects Based on a Deep Learning Network: A Case Study of USB Packs

The challenge of getting machines to understand and interact with natural objects is encountered in important areas such as medicine, agriculture, and, in our case, slaughterhouse automation. Recent breakthroughs have enabled the application of Deep Neural Networks (DNN) directly to point clouds, an efficient and natural representation of 3D objects. The potential of these methods has mostly been demonstrated for classification and segmentation tasks involving rigid man-made objects. We present a method, based on the successful PointNet architecture, for learning to regress correct tool placement from human demonstrations, using virtual reality. Our method is applied to a challenging slaughterhouse cutting task, which requires an understanding of the local geometry including the shape, size, and orientation. We propose an intermediate five-Degree of Freedom (DoF) cutting plane representation, a point and a normal vector, which eases the demonstration and learning process. A live experiment is conducted in order to unveil issues and begin to understand the required accuracy. Eleven cuts are rated by an expert, with 8 / 11 being rated as acceptable. The error on the test set is subsequently reduced through the addition of more training data and improvements to the DNN. The result is a reduction in the average translation from 1.5 cm to 0.8 cm and the orientation error from 4.59° to 4.48°. The method’s generalization capacity is assessed on a similar task from the slaughterhouse and on the very different public LINEMOD dataset for object pose estimation across view points. In both cases, the method shows promising results. Code, datasets, and other materials are available in Supplementary Materials.

Section: Discussionsupporting

confidence: 85%

Cutting Pose Prediction from Point Clouds

Philipsen

Moeslund

2020

“…The investigated systems were developed for either pickand-place applications [5,12,19,20] or production inspections (i.e., [4] and this work). For planar manipulations [5,12], image-based object recognition would be enough; on the other hand, 3D scanning was utilized for recognizing the objects that were either randomly placed (i.e., [20] and this work) or had arbitrary shapes [19]. The challenges of the robot manipulations increased as the objects Sensors 2020, 20, x FOR PEER REVIEW 9 of 14 mark would be measured during the object following and inline inspection to show the accuracy of the optical inspection.…”

Section: Experiments and Results Of The Proposed Inline Inspectionmentioning

confidence: 99%

“…A comparison between the presented IIIR and some other existing methods that were developed in the past few years is given in Table 3. The investigated systems were developed for either pick-and-place applications [5,12,19,20] or production inspections (i.e., [4] and this work). For planar manipulations [5,12], image-based object recognition would be enough; on the other hand, 3D scanning was utilized for recognizing the objects that were either randomly placed (i.e., [20] and this work) or had arbitrary shapes [19].…”

Section: Experiments and Results Of The Proposed Inline Inspectionmentioning

confidence: 99%

Inline Inspection with an Industrial Robot (IIIR) for Mass-Customization Production Line

Lin

Chang

et al. 2020

Robots are essential for the rapid development of Industry 4.0. In order to truly achieve autonomous robot control in customizable production lines, robots need to be accurate enough and capable of recognizing the geometry and orientation of an arbitrarily shaped object. This paper presents a method of inline inspection with an industrial robot (IIIR) for mass-customization production lines. A 3D scanner was used to capture the geometry and orientation of the object to be inspected. As the object entered the working range of the robot, the end effector moved along with the object and the camera installed at the end effector performed the requested optical inspections. The detailed information about the developed methodology was introduced in this paper. The experiments showed there was a relative movement between the moving object and the following camera and the speed was around 0.34 mm per second (worst case was around 0.94 mm per second). For a camera of 60 frames per second, the relative moving speed between the object and the camera was around 6 micron (around 16 micron for the worst case), which was stable enough for most industrial production inspections.

“…The packaging process is divided into two tasks: object-picking and packing. The former means picking one of the items randomly arranged within a container, and it has been tried to be automated in various directions with deep learning including the recent Amazon robotic challenge outcomes [ 1 , 2 , 3 ]. The latter implies placing the picked up item in the most suitable position in another container, and mostly it has been implemented in a theoretical way or in a hybrid form with human collaboration [ 4 , 5 ].…”

Section: Introductionmentioning

confidence: 99%

Smart Pack: Online Autonomous Object-Packing System Using RGB-D Sensor Data

Hong

Kim

Lee

2020

This paper proposes a novel online object-packing system which can measure the dimensions of every incoming object and calculate its desired position in a given container. Existing object-packing systems have the limitations of requiring the exact information of objects in advance or assuming them as boxes. Thus, this paper is mainly focused on the following two points: (1) Real-time calculation of the dimensions and orientation of an object; (2) Online optimization of the object’s position in a container. The dimensions and orientation of the object are obtained using an RGB-D sensor when the object is picked by a manipulator and moved over a certain position. The optimal position of the object is calculated by recognizing the container’s available space using another RGB-D sensor and minimizing the cost function that is formulated by the available space information and the optimization criteria inspired by the way people place things. The experimental results show that the proposed system successfully places the incoming various shaped objects in their proper positions.