Planning with a Receding Horizon for Manipulation in Clutter Using a Learned Value Function

Abstract

Manipulation in clutter requires solving complex sequential decision making problems in an environment rich with physical interactions. The transfer of motion planning solutions from simulation to the real world, in open-loop, suffers from the inherent uncertainty in modelling real world physics. We propose interleaving planning and execution in real-time, in a closed-loop setting, using a Receding Horizon Planner (RHP) for pushing manipulation in clutter. In this context, we address the problem of finding a suitable value function based heuristic for efficient planning, and for estimating the cost-to-go from the horizon to the goal. We estimate such a value function first by using plans generated by an existing sampling-based planner. Then, we further optimize the value function through reinforcement learning. We evaluate our approach and compare it to state-of-the-art planning techniques for manipulation in clutter. We conduct experiments in simulation with artificially injected uncertainty on the physics parameters, as well as in real world tasks of manipulation in clutter. We show that this approach enables the robot to react to the uncertain dynamics of the real world effectively. arXiv:1803.08100v2 [cs.RO]

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

“…8), which can be generalized to new environments without training in the same environment. In terms of interleaving planning and execution in real time and closed-loop settings, Bejjani et al [114] implemented a receding horizon planner (RHP) for pushing manipulation in clutter, as shown in Fig. 9.…”

Online Replanning With Human-in-the-Loop for Non-Prehensile Manipulation in Clutter — A Trajectory Optimization Based Approach

“…The autonomous solutions to the reaching through clutter problem can be categorized into three groups: There are sampling-based planning approaches [5], [6], [9], trajectory optimization based approaches [3], [14], and learning-based approaches [4], [7], [15], [16]. While these approaches show varying degrees of success, the difficult instances of this problem are still challenging for autonomous systems, due to the problem being high-dimensional and under-actuated, and also due to real-world physics uncertainty.…”

Occlusion-Aware Search for Object Retrieval in Clutter

“…By continuously updating the simulator state, where planning takes place, from real-world observations, RHP circumvents the problem of compounding modelling errors over long sequences of actions. Under the assumption of a fully observable environment, we have shown in our previous work how RHP can be used with a heuristic to guide physics-based roll-outs and to estimate the cost-to-go from the horizon to the goal [16]. This approach balances the advantages of model-based sequential reasoning with a model-free scalable heuristic [17], [18].…”