Autonomous Dual-Arm Manipulation of Familiar Objects

Abstract

Autonomous dual-arm manipulation is an essential skill to deploy robots in unstructured scenarios. However, this is a challenging undertaking, particularly in terms of perception and planning. Unstructured scenarios are full of objects with different shapes and appearances that have to be grasped in a very specific manner so they can be functionally used. In this paper we present an integrated approach to perform dual-arm pick tasks autonomously. Our method consists of semantic segmentation, object pose estimation, deformable model registration, grasp planning and arm trajectory optimization. The entire pipeline can be executed onboard and is suitable for on-line grasping scenarios. For this, our approach makes use of accumulated knowledge expressed as convolutional neural network models and low-dimensional latent shape spaces. For manipulating objects, we propose a stochastic trajectory optimization that includes a kinematic chain closure constraint. Evaluation in simulation and on the real robot corroborates the feasibility and applicability of the proposed methods on a task of picking up unknown watering cans and drills using both arms.

Learning Postural Synergies for Categorical Grasping Through Shape Space Registration

“…This approach combines the Coherent Point Drift (CPD) with subspaces methods to create a shape (latent) space that encodes typical geometrical variations inside a category. This method has been applied to transfer control poses for approaching the objects to grasp [8,9], and to accumulate experiences on the motion for grasping different objects into a canonical model [10]. In this paper, we mainly concentrate on the configuration of the hand, i.e., on how the hand configuration changes according to the shape of the objects.…”

Category-Level 3D Non-Rigid Registration from Single-View RGB Images

“…Registering 3D objects in a non-rigid way is essential for many real world applications, including robot manipulation [1,2], human body analysis [3,4] and grasp planning [5,6]. This registration plays a key role for transferring domain knowledge, such as the transfer of control poses for approaching and grasping objects.…”

UQ eSpace

“…Similarly, [14] presents a pipeline for the visual detection and grasp planning of objects on dual arm systems by enhancing the popular STOMP method, stochastic trajectory optimization for motion planning, to generate feasible grasping plans.…”

“…Despite the reasonable results seen in [12], and the ability of [13] and [14] to generate collision free motions for their dual-arm systems, the algorithms are restricted by limitations such as having to know the target's pose beforehand or relying on accurate behaviour of the robots.…”

“…Despite the advances on manipulation research, most studies that take into account the inherent uncertainty in real environments are still based on one arm systems. While literature is available for dual arm manipulation, most of existing studies [12], [13], [14], are still based on approaches that require deterministic behaviour or sensing from the robot.…”