Interleaving Monte Carlo Tree Search and Self-Supervised Learning for Object Retrieval in Clutter

Abstract

In this study, working with the task of object retrieval in clutter, we have developed a robot learning framework in which Monte Carlo Tree Search (MCTS) is first applied to enable a Deep Neural Network (DNN) to learn the intricate interactions between a robot arm and a complex scene containing many objects, allowing the DNN to partially clone the behavior of MCTS. In turn, the trained DNN is integrated into MCTS to help guide its search effort. We call this approach Monte Carlo tree search and learning for Object REtrieval (MORE), which delivers significant computational efficiency gains and added solution optimality. MORE is a self-supervised robotics framework/pipeline capable of working in the real world that successfully embodies the System 2 → System 1 learning philosophy proposed by Kahneman, where learned knowledge, used properly, can help greatly speed up a time-consuming decision process over time. Videos and supplementary material can be found at https://github.com/arc-l/more.

Coarse-to-fine Q-attention with Tree Expansion

“…Outside of symbolic planning, Huang et al [26] learned to retrieving a target object from clutter performing a Monte Carlo search over highlevel non-prehensile actions. Its search time was subsequently improved by learning to predict the discounted reward of each branch in MCTS without the need to roll out [27]. Bai et al [28] also tackle non-prehensile manipulation by proposing an MCTS algorithm guided by a policy network which is trained by imitation and reinforcement.…”