Non-Prehensile Manipulation in Clutter with Human-In-The-Loop

Abstract

We propose a human-operator guided planning approach to pushing-based manipulation in clutter. Most recent approaches to manipulation in clutter employs randomized planning. The problem, however, remains a challenging one where the planning times are still in the order of tens of seconds or minutes, and the success rates are low for difficult instances of the problem. We build on these control-based randomized planning approaches, but we investigate using them in conjunction with human-operator input. In our framework, the human operator supplies a high-level plan, in the form of an ordered sequence of objects and their approximate goal positions. We present experiments in simulation and on a real robotic setup, where we compare the success rate and planning times of our human-in-the-loop approach with fully autonomous sampling-based planners. We show that with a minimal amount of human input, the low-level planner can solve the problem faster and with higher success rates.

Toward Future Automatic Warehouses: An Autonomous Depalletizing System Based on Mobile Manipulation and 3D Perception

“…In [17], a new planning framework to exploit the funneling effect of pushing to deal with uncertain and clutter environments is proposed. Similarly, in [18] a framework that takes into account the human presence in cluttered environments is analyzed. Acharya et al [19] presented a motion planning analysis for the optimization of the stability and control of an asymmetric object during non-prehensile manipulation.…”

Online Replanning With Human-in-the-Loop for Non-Prehensile Manipulation in Clutter — A Trajectory Optimization Based Approach

“…In this respect, a key novelty of our method is the integration of human-interaction into this online-replanning architecture, enabling the system to use human help at any point in time during execution, not only as an initial input to the planning problem. Our previous work uses human guidance to solve reaching through clutter problems [12], but this approach requires human input to be provided before planning. It also uses a sampling-based planning approach (as opposed to the trajectory optimization approach we use in this paper).…”

“…A final novel feature of our proposed system is the efficient use of human time. Our previous work [12] requires human help irrespective of whether an autonomous planner can solve the problem efficiently or not. We propose that the human should be recruited for help only if and when an autonomous planner is expected to fail or when human help is expected to speed up planning significantly.…”

“…Instead, we focus on nonprehensile manipulation. Our previous work [12], proposes a planner for non-prehensile manipulation using human-guidance, but the current work has significant differences in terms of the planning method (trajectory optimization versus samplingbased planning), the execution (closed-loop online re-planning versus open-loop execution), and most importantly how human input is requested (human input requested only when needed versus human input requested for all problems).…”

Occlusion-Aware Search for Object Retrieval in Clutter

“…The robot has to reason over a history of partial observations to efficiently explore where the target object might be. Furthermore, it is notoriously hard to predict the outcome of an action in multicontact physics environments [4], [5], [6], [7]. Modelling error on the physics parameters such as friction, inertia, and objects shapes impede open-loop execution of long action sequences.…”