Real-Time Online Re-Planning for Grasping Under Clutter and Uncertainty

Abstract

This work is published as a conference paper [1] at IEEE Humanoids 2018.We consider the problem of grasping in clutter. While there have been motion planners developed to address this problem in recent years, these planners are mostly tailored for openloop execution. Open-loop execution in this domain, however, is likely to fail, since it is not possible to model the dynamics of the multi-body multi-contact physical system with enough accuracy, neither is it reasonable to expect robots to know the exact physical properties of objects, such as frictional, inertial, and geometrical. Therefore, we propose an online re-planning approach for grasping through clutter. The main challenge is the long planning times this domain requires, which makes fast re-planning and fluent execution difficult to realize. In order to address this, we propose an easily parallelizable stochastic trajectory optimization based algorithm that generates a sequence of optimal controls. We show that by running this optimizer only for a small number of iterations, it is possible to perform real time re-planning cycles to achieve reactive manipulation under clutter and uncertainty.

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

“…They overcame these limitations in their next work [115] by combining image-based learning systems with look-ahead planning. Other studies assumed object geometry [116] [117], and Cartesian coordinates were relied on to represent the state. Meanwhile, Song et al [118] utilized object geometries to increase the efficiency of manipulation motions.…”

Online Replanning With Human-in-the-Loop for Non-Prehensile Manipulation in Clutter — A Trajectory Optimization Based Approach

“…The autonomous solutions to the reaching through clutter problem can be categorized into three groups: There are sampling-based planning approaches [5], [6], [9], trajectory optimization based approaches [3], [14], and learning-based approaches [4], [7], [15], [16]. While these approaches show varying degrees of success, the difficult instances of this problem are still challenging for autonomous systems, due to the problem being high-dimensional and under-actuated, and also due to real-world physics uncertainty.…”

Parareal with a learned coarse model for robotic manipulation

“…Physics predictions play a major role in robotic manipulation planning and control-to generate uncertainty averse robotic pushing plans [6], to manipulate objects in clutter through online re-planning [2], to rearrange objects in clutter through dynamic actions [17], and also to use human guidance to generate pushing motions [30]. However, planning is slow since physics predictions are computationally expensive.…”

“…Such an optimization-based MPC approach to pushing manipulation is frequently used to handle uncertainty and improve success in the real-world [2,6,18,23]. Here, our focus is to evaluate the performance of Parareal with learned coarse model for planning and control.…”