Mechanical Search on Shelves using Lateral Access X-RAY

Huang, Huang; Dominguez-Kuhne, Marcus; Satish, Vishal; Danielczuk, Michael; Sanders, Kate; Ichnowski, Jeffrey; Lee, Andrew; Angelova, Anelia; Vanhoucke, Vincent; Goldberg, Ken

doi:10.1109/iros51168.2021.9636629

Cited by 22 publications

(18 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically for the task of object search and retrieval, [8] uses the POMDP framework to formulate the problem, namely the Mechanical Search problem. It has recently seen applications in different settings [18,15]. However, discussion of the general occlusion-aware manipulation tasks is still missing.…”

Section: Problem Classification and Related Workmentioning

confidence: 99%

“…Model-Based Object Retrieval in a Shelf Many works in the literature focus on the problem of object search and retrieval with known object models from a shelf [10,23,2,32,19,15]. In this setting, The perception module is assumed to fully recognize the object once a sufficient part is observed, thus ignoring partial occlusion.…”

Section: Problem Classification and Related Workmentioning

confidence: 99%

“…More recently, the focus has shifted to cases where object models are unavailable. Nevertheless, most of these methods do not provide theoretical guarantees, either due to the dependence on machine learning [2,8,18,15], or assumptions that are valid only in limited scenarios [14]. Learning-based methods bring great promise to achieve efficiency but are highly desirable if they can be applied within a safe and complete framework.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Safe, Occlusion-Aware Manipulation for Online Object Reconstruction in Confined Spaces

Miao¹,

Wang²,

Bekris³

2022

Preprint

View full text Add to dashboard Cite

Recent work in robotic manipulation focuses on object retrieval in cluttered space under occlusion. Nevertheless, the majority of efforts lack an analysis of conditions for the completeness of the approaches or the methods apply only when objects can be removed from the workspace. This work formulates the general, occlusion-aware manipulation task, and focuses on safe object reconstruction in a confined space with in-place relocation. A framework that ensures safety with completeness guarantees is proposed. Furthermore, an algorithm, which is an instantiation of this framework for monotone instances, is developed and evaluated empirically by comparing against a random and a greedy baseline on randomly generated experiments in simulation. Even for cluttered scenes with realistic objects, the proposed algorithm significantly outperforms the baselines and maintains a high success rate across experimental conditions.

show abstract

Section: Problem Classification and Related Workmentioning

confidence: 99%

Section: Problem Classification and Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Safe, Occlusion-Aware Manipulation for Online Object Reconstruction in Confined Spaces

Miao¹,

Wang²,

Bekris³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Some experiments, especially physical experiments, can be expensive, time-consuming, or risky (Lim et al, 2021;Li et al, 2021). Meanwhile, well-designed experiments can accelerate simulation tuning (Lim et al, 2021); benefit learning for robotics (Zhao et al, 2021), for instance improving robot Sim2Real transfer (Lim et al, 2021); and improve perception models (Huang et al, 2020). Bayesian Optimal Experi-mental Design (BOED) (Lindley, 1956) is one framework for designing experiments to maximize the information that observations yield about unknown parameters of interest.…”

Section: Introductionmentioning

confidence: 99%

Policy-Based Bayesian Experimental Design for Non-Differentiable Implicit Models

Lim¹,

Novoseller²,

Ichnowski³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

For applications in healthcare, physics, energy, robotics, and many other fields, designing maximally informative experiments is valuable, particularly when experiments are expensive, timeconsuming, or pose safety hazards. While existing approaches can sequentially design experiments based on prior observation history, many of these methods do not extend to implicit models, where simulation is possible but computing the likelihood is intractable. Furthermore, they often require either significant online computation during deployment or a differentiable simulation system. We introduce Reinforcement Learning for Deep Adaptive Design (RL-DAD), a method for simulation-based optimal experimental design for non-differentiable implicit models. RL-DAD extends prior work in policy-based Bayesian Optimal Experimental Design (BOED) by reformulating it as a Markov Decision Process with a reward function based on likelihood-free information lower bounds, which is used to learn a policy via deep reinforcement learning. The learned design policy maps prior histories to experiment designs offline and can be quickly deployed during online execution. We evaluate RL-DAD and find that it performs competitively with baselines on three benchmarks.

show abstract

“…However this is time-consuming, errorprone, and expensive [36], especially when dense or 3-D annotations are required [5,11,12,23,43]. An alternative approach is to use simulated data, where data annotation can be densely and autonomously generated at scale at relatively low cost [5,12,17,18,23,28,43]. However, sim-toreal transfer is still an active area of research with many current limitations and open questions [50].…”

Section: Introductionmentioning

confidence: 99%

All You Need is LUV: Unsupervised Collection of Labeled Images using Invisible UV Fluorescent Indicators

Thananjeyan¹,

Kerr²,

Huang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Large-scale semantic image annotation is a significant challenge for learning-based perception systems in robotics. Current approaches often rely on human labelers, which can be expensive, or simulation data, which can visually or physically differ from real data. This paper proposes Labels from UltraViolet (LUV), a novel framework that enables rapid, labeled data collection in real manipulation environments without human labeling. LUV uses transparent, ultravioletfluorescent paint with programmable ultraviolet LEDs to collect paired images of a scene in standard lighting and UV lighting to autonomously extract segmentation masks and keypoints via color segmentation. We apply LUV to a suite of diverse robot perception tasks to evaluate its labeling quality, flexibility, and data collection rate. Results suggest that LUV is 180-2500 times faster than a human labeler across the tasks. We show that LUV provides labels consistent with human annotations on unpainted test images. The networks trained on these labels are used to smooth and fold crumpled towels with 83% success rate and achieve 1.7mm position error with respect to human labels on a surgical needle pose estimation task. The low cost of LUV makes it ideal as a lightweight replacement for human labeling systems, with the one-time setup costs at $300 equivalent to the cost of collecting around 200 semantic segmentation labels on Amazon Mechanical Turk. Code, datasets, visualizations, and supplementary material can be found at https://sites. google.com/berkeley.edu/luv.

show abstract

Mechanical Search on Shelves using Lateral Access X-RAY

Cited by 22 publications

References 35 publications

Safe, Occlusion-Aware Manipulation for Online Object Reconstruction in Confined Spaces

Safe, Occlusion-Aware Manipulation for Online Object Reconstruction in Confined Spaces

Policy-Based Bayesian Experimental Design for Non-Differentiable Implicit Models

All You Need is LUV: Unsupervised Collection of Labeled Images using Invisible UV Fluorescent Indicators

Contact Info

Product

Resources

About