Visuomotor Mechanical Search: Learning to Retrieve Target Objects in Clutter

Abstract

When searching for objects in cluttered environments, it is often necessary to perform complex interactions in order to move occluding objects out of the way and fully reveal the object of interest and make it graspable. Due to the complexity of the physics involved and the lack of accurate models of the clutter, planning and controlling precise predefined interactions with accurate outcome is extremely hard, when not impossible. In problems where accurate (forward) models are lacking, Deep Reinforcement Learning (RL) has shown to be a viable solution to map observations (e.g. images) to good interactions in the form of close-loop visuomotor policies. However, Deep RL is sample inefficient and fails when applied directly to the problem of unoccluding objects based on images. In this work we present a novel Deep RL procedure that combines i) teacheraided exploration, ii) a critic with privileged information, and iii) mid-level representations, resulting in sample efficient and effective learning for the problem of uncovering a target object occluded by a heap of unknown objects. Our experiments show that our approach trains faster and converges to more efficient uncovering solutions than baselines and ablations, and that our uncovering policies lead to an average improvement in the graspability of the target object, facilitating downstream retrieval applications.

Mechanical Search on Shelves using a Novel "Bluction" Tool

“…Danielczuk et al [6] formulate the mechanical search problem and introduce a twostage perception and search policy pipeline that uses heuristic high-level policies to guide pushing and grasping within the bin. Kurenkov et al [12] extend this work by introducing a learned, non-linear pushing action to uncover the target. Zeng et al [26], Novkovic et al [18], and Yang et al [24] explore the problem in a tabletop setting and jointly learn coordinated pushing and grasping strategies.…”

“…Finding a target object in lateral-access settings such as shelves-common in homes, warehouses, and retail storesis complicated by objects toward the front blocking visibility of and access to objects further back. In contrast with the overhead-access bin settings explored in prior works [6,12,24,26], where objects can be heaped in arbitrary poses, objects on shelves consistently rest in stable poses. If a target object is occluded on a shelf, other objects must be carefully moved without toppling to reveal the target.…”

Total Singulation With Modular Reinforcement Learning

“…Recent works, use deep reinforcement learning and pushing actions in order to render fully visible an occluded target [18], [19]. In contrast to singulation, achieving full visibility of the target does not always create free space between the target and surrounding obstacles, which is necessary for prehensile grasping.…”

INVIGORATE: Interactive Visual Grounding and Grasping in Clutter

“…In recent years, both data-driven deep learning and model-based planning helped bring about significant progress in object grasping in general [9,21,22,29] and goal-directed object grasping in particular [8,13,24,42,44]. Several recent methods aim at goal-directed object grasping in clutter scenes [6,18,44]. Specifically, Zhang et al [44] propose to learn object blocking relationships for grasping.…”