Fast Object Learning and Dual-arm Coordination for Cluttered Stowing, Picking, and Packing

Abstract

Robotic picking from cluttered bins is a demanding task, for which Amazon Robotics holds challenges. The 2017 Amazon Robotics Challenge (ARC) required stowing items into a storage system, picking specific items, and packing them into boxes. In this paper, we describe the entry of team NimbRo Picking. Our deep object perception pipeline can be quickly and efficiently adapted to new items using a custom turntable capture system and transfer learning. It produces high-quality item segments, on which grasp poses are found. A planning component coordinates manipulation actions between two robot arms, minimizing execution time. The system has been demonstrated successfully at ARC, where our team reached second places in both the picking task and the final stow-and-pick task. We also evaluate individual components. * All authors are with the

Supervised Autonomous Locomotion and Manipulation for Disaster Response with a Centaur-Like Robot

“…We apply our object segmentation approach to RGB images from the Kinect V2 [20]. This approach is able to produce pixel-(or point-)wise segmentation directly.…”

“…The pretrained RefineNet network from the semantic segmentation is used to extract features. To generate the ground truth poses for training the network, the data acquisition pipeline described in [20] was extended to record turntable poses automatically and fuse captures with different object poses or different objects with minimal user intervention.…”

Autonomous Dual-Arm Manipulation of Familiar Objects

“…Those are composed of a small number of captured background images which are augmented randomly with inserted objects. This approach follows Schwarz et al [9] closely, with the exception that the inserted object segments are rendered from CAD meshes using the open-source Blender renderer. The core of the model consists of four ResNet blocks.…”

“…At inference time, also following Schwarz et al [9], we postprocess the semantic segmentation to find individual object contours. The dominant object is found using the pixel count and is extracted from the input image for further processing.…”

Joint Learning of Instance and Semantic Segmentation for Robotic Pick-and-Place with Heavy Occlusions in Clutter

“…For robotic manipulation, pixel-wise object segmentation is a crucial component. Previous work utilizes semantic segmentation models for pick-and-place of various objects [8]- [13]. Since semantic segmentation can not segment different instances in the same class, the work assumes that same class objects are not closely located and can be segmented by clustering.…”