Mechanical Search on Shelves using Lateral Access X-RAY

Abstract

Efficiently finding an occluded object with lateral access arises in many contexts such as warehouses, retail, healthcare, shipping, and homes. We introduce LAX-RAY (Lateral Access maXimal Reduction of occupancY support Area), a system to automate the mechanical search for occluded objects on shelves. For such lateral access environments, LAX-RAY couples a perception pipeline predicting a target object occupancy support distribution with a mechanical search policy that sequentially selects occluding objects to push to the side to reveal the target as efficiently as possible. Within the context of extruded polygonal objects and a stationary target with a known aspect ratio, we explore three lateral access search policies: Uniform, Distribution Area Reduction (DAR) and Distribution Entropy Reduction over n Steps (DER-n) utilizing the support distribution and prior information. We evaluate these policies using the First-Order Shelf Simulator (FOSS) in which we simulate 800 random shelf environments of varying difficulty, and in a physical shelf environment with a Fetch robot and an embedded PrimeSense RGBD Camera. Average simulation results of 87.3% success rate demonstrate better performance of DER-2. Physical results show a success rate of at least 80% for DAR and DER-n, suggesting that LAX-RAY can efficiently reveal the target object in reality. Both results show significantly better performance of DAR and DER-n compared to the uniform policy with uniform probability distribution assumption in non-trivial cases, suggesting the importance of distribution prediction. Code, videos, and supplementary material can be found at https://sites.google.com/ berkeley.edu/lax-ray.

Graph-based Cluttered Scene Generation and Interactive Exploration using Deep Reinforcement Learning

“…Other approaches deal with occluded objects that may be partially or completely invisible [13], [14], [15], [16], [17]. Often in such heavily cluttered scenes, such as a bin [18], [13], [7], [14] or a shelf [19], [20], the agent has to move other objects around to search for the target. While these approaches learn to search for a given target object, a more generalized version of this problem is to identify and locate all objects present in the scene.…”