AVPLUG: Approach Vector PLanning for Unicontact Grasping amid Clutter

Avigal, Yahav; Satish, Vishal; Tam, Zachary; Huang, Huang; Zhang, Harry; Danielczuk, Michael; Ichnowski, Jeffrey; Goldberg, Ken

doi:10.1109/case49439.2021.9551652

Cited by 7 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recent work has started looking at the structure of the pile and constrains the potential target item locations by leveraging the geometry of both the pile and the target object [1]. Other work has also looked at lateral search, where objects are retrieved from the side rather than from a pile [4,17]. One of the main challenges of this vision-based approach to mechanical search is that as piles become larger and more complex, the uncertainty grows and the systems become more inefficient.…”

Section: Related Workmentioning

confidence: 99%

FuseBot: Mechanical Search of Rigid and Deformable Objects via Multi-Modal Perception

Boroushaki

Dodds

Naeem

et al. 2023

Preprint

View full text Add to dashboard Cite

Mechanical search is a robotic problem where a robot needs to retrieve a target item that is partially or fully-occluded from its camera. State-of-the-art approaches for mechanical search either require an expensive search process to find the target item, or they require the item to be tagged with a radio frequency identification tag (eg, RFID), making their approach beneficial only to tagged items in the environment. We present FuseBot, the first robotic system for RF-Visual mechanical search that enables efficient retrieval of both RF-tagged and untagged items in a pile. Rather than requiring all target items in a pile to be RF-tagged, FuseBot leverages the mere existence of an RF-tagged item in the pile to benefit both tagged and untagged items. Our design introduces two key innovations. The first is RF-Visual Mapping, a technique that identifies and locates RF-tagged items in a pile and uses this information to construct an RF-Visual occupancy distribution map. The second is RF-Visual Extraction, a policy formulated as an optimization problem that minimizes the number of actions required to extract the target object by accounting for the probabilistic occupancy distribution, the expected grasp quality, and the expected information gain from future actions. We built a real-time end-to-end prototype of our system on a UR5e robotic arm with in-hand vision and RF perception modules. We conducted over 180 real-world experimental trials to evaluate FuseBot and compare its performance to a state-of-the-art vision-based system named X-Ray. Our experimental results demonstrate that FuseBot outperforms X-Ray’s efficiency by more than 40% in terms of the number of actions required for successful mechanical search. Furthermore, in comparison to X-Ray’s success rate of 84%, FuseBot achieves a success rate of 95% in retrieving untagged items, demonstrating for the first time that the benefits of RF perception extend beyond tagged objects in the mechanical search problem.

show abstract

Section: Related Workmentioning

confidence: 99%

FuseBot: Mechanical Search of Rigid and Deformable Objects via Multi-Modal Perception

Boroushaki

Dodds

Naeem

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Since the flow vector with the maximum magnitude is often at one of these extreme points, we must choose an alternative grasp point. While contact selection for suction-based grasping is a well-studied problem [2,16,17], we find that a simple heuristic performs acceptably; we choose the point with the highest flow magnitude subject to the following constraints:…”

Section: E Flowbot3d: a Robot Articulation Systemmentioning

confidence: 99%

FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects

Eisner¹,

Zhang²,

Held³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

We explore a novel method to perceive and manipulate 3D articulated objects that generalizes to enable a robot to articulate unseen classes of objects. We propose a vision-based system that learns to predict the potential motions of the parts of a variety of articulated objects to guide downstream motion planning of the system to articulate the objects. To predict the object motions, we train a neural network to output a dense vector field representing the point-wise motion direction of the points in the point cloud under articulation. We then deploy an analytical motion planner based on this vector field to achieve a policy that yields maximum articulation. We train the vision system entirely in simulation, and we demonstrate the capability of our system to generalize to unseen object instances and novel categories in both simulation and the real world, deploying our policy on a Sawyer robot with no finetuning. Results show that our system achieves state-of-the-art performance in both simulated and real-world experiments. Code, data, and supplementary materials are available at this website.

show abstract

“…More recent work has started looking at the structure of the pile and constrains the potential target item locations by leveraging the geometry of both the pile and the target object [10]. Other work has also looked at lateral search, where objects are retrieved from the side rather than from a pile [16,1]. One of the main challenges of this vision-based approach to mechanical search is that as piles become larger and more complex, the uncertainty grows and the systems become more inefficient.…”

Section: Related Workmentioning

confidence: 99%

“…leveraging RF perception. A positive answer to this question would extend the benefits of RF perception to new application scenarios, such as those where the target item cannot be tagged with inexpensive RFIDs (e.g., metal tools and liquid bottles) 1 and instances when the robot is presented with piles of items that are not fully tagged.…”

Section: Introductionmentioning

confidence: 99%

FuseBot: RF-Visual Mechanical Search

Boroushaki¹,

Dodds²,

Naeem³

et al. 2022

Robotics: Science and Systems XVIII

View full text Add to dashboard Cite

Mechanical search is a robotic problem where a robot needs to retrieve a target item that is partially or fullyoccluded from its camera. State-of-the-art approaches for mechanical search either require an expensive search process to find the target item, or they require the item to be tagged with a radio frequency identification tag (e.g., RFID), making their approach beneficial only to tagged items in the environment.We present FuseBot, the first robotic system for RF-Visual mechanical search that enables efficient retrieval of both RFtagged and untagged items in a pile. Rather than requiring all target items in a pile to be RF-tagged, FuseBot leverages the mere existence of an RF-tagged item in the pile to benefit both tagged and untagged items. Our design introduces two key innovations. The first is RF-Visual Mapping, a technique that identifies and locates RF-tagged items in a pile and uses this information to construct an RF-Visual occupancy distribution map. The second is RF-Visual Extraction, a policy formulated as an optimization problem that minimizes the number of actions required to extract the target object by accounting for the probabilistic occupancy distribution, the expected grasp quality, and the expected information gain from future actions.We built a real-time end-to-end prototype of our system on a UR5e robotic arm with in-hand vision and RF perception modules. We conducted over 180 real-world experimental trials to evaluate FuseBot and compare its performance to a state-ofthe-art vision-based system named X-Ray [10]. Our experimental results demonstrate that FuseBot outperforms X-Ray's efficiency by more than 40% in terms of the number of actions required for successful mechanical search. Furthermore, in comparison to X-Ray's success rate of 84%, FuseBot achieves a success rate of 95% in retrieving untagged items, demonstrating for the first time that the benefits of RF perception extend beyond tagged objects in the mechanical search problem.

show abstract

AVPLUG: Approach Vector PLanning for Unicontact Grasping amid Clutter

Cited by 7 publications

References 28 publications

FuseBot: Mechanical Search of Rigid and Deformable Objects via Multi-Modal Perception

FuseBot: Mechanical Search of Rigid and Deformable Objects via Multi-Modal Perception

FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects

FuseBot: RF-Visual Mechanical Search

Contact Info

Product

Resources

About