Deep Learning-Based Pose Estimation of Apples for Inspection in Logistic Centers Using Single-Perspective Imaging

Abstract

Fruit packaging is a time-consuming task due to its low automation level. The gentle handling required by some kinds of fruits and their natural variations complicates the implementation of automated quality controls and tray positioning for final packaging. In this article, we propose a method for the automatic localization and pose estimation of apples captured by a Red-Green-Blue (RGB) camera using convolutional neural networks. Our pose estimation algorithm uses a cascaded structure composed of two independent convolutional neural networks: one for the localization of apples within the images and a second for the estimation of the three-dimensional rotation of the localized and cropped image area containing an apple. We used a single shot multi-box detector to find the bounding boxes of the apples in the images. Lie algebra is used for the regression of the rotation, which represents an innovation in this kind of application. We compare the performances of four different network architectures and show that this kind of representation is more suitable than using state-of-the-art quaternions. By using this method, we achieved a promising accuracy for the rotation regression of 98.36%, considering an error range lower than 15 degrees, forming a base for the automation of fruit packing systems.

Predictive Control of Cooperative Robots Sharing Common Workspace

“…During the experiment, the robots fill two trays each, corresponding to picking up a total of 24 objects and placing them in the slots of the trays. The estimation of the pose of the objects is done by applying the deep learning algorithm presented in [51]. For the pick-andplace task execution, scheduling is performed first to assign a tray and a sequence of objects to a robot.…”

An Algorithm for Calculating Apple Picking Direction Based on 3D Vision

“…Giefer et al developed a convolutional neural network algorithm for automatic localization and posture determination using images of apples captured by RGB cameras [7]. The algorithm needs to collect high-definition images of fruits at close range under lighting conditions to detect fruit textures, so it is not suitable for long-distance recording images in outdoor unstructured environments.…”

Fast and Accurate Pose Estimation of Additive Manufactured Objects from Few X-Ray Projections

“…In X-ray inspection, it is common to demand 100 μm of accuracy and down to 1 μm for metrology tasks (Tan, Kiekens, Kruth, Voet, & Dewulf, 2011). This is far from the accuracy of the state-of-the-art networks for pose estimation from images, which reaches a few degrees and millimeters or centimeters (Brynte & Kahl, 2020;Giefer, Castellanos, Babr, & Freitag, 2019;Kendall, Grimes, & Cipolla, 2016). To the author's knowledge, so far very few works have been proposed for object pose estimation from X-ray radiographs.…”