Unseen Object Instance Segmentation for Robotic Environments

Xie, Christopher; Xiang, Yu; Mousavian, Arsalan; Fox, Dieter

doi:10.1109/tro.2021.3060341

Cited by 104 publications

(86 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One main drawback of the above methods is that they usually sacrifice generality to unknown objects since they cannot be recognized by most object detectors or scene segmentation algorithms. One possible way to solve this problem is leveraging recent unseen object instance segmentation methods [252,253], which makes it possible to remove unknown objects in clutter for getting targets [231]. Nonetheless, it still cannot recognize the semantics of unknown objects.…”

Section: Object-specific Grasp Synthesismentioning

confidence: 99%

Robotic Grasping from Classical to Modern: A Survey

Zhang¹,

Tang²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

Robotic Grasping has always been an active topic in robotics since grasping is one of the fundamental but most challenging skills of robots. It demands the coordination of robotic perception, planning, and control for robustness and intelligence. However, current solutions are still far behind humans, especially when confronting unstructured scenarios.In this paper, we survey the advances of robotic grasping, starting from the classical formulations and solutions to the modern ones. By reviewing the history of robotic grasping, we want to provide a complete view of this community, and perhaps inspire the combination and fusion of different ideas, which we think would be helpful to touch and explore the essence of robotic grasping problems. In detail, we firstly give an overview of the analytic methods for robotic grasping. After that, we provide a discussion on the recent state-ofthe-art data-driven grasping approaches rising in recent years. With the development of computer vision, semantic grasping is being widely investigated and can be the basis of intelligent manipulation and skill learning for autonomous robotic systems in the future.Therefore, in our survey, we also briefly review the recent progress in this topic. Finally, we discuss the open problems and the future research directions that may be important for the human-level robustness, autonomy, and intelligence of robots.

show abstract

Section: Object-specific Grasp Synthesismentioning

confidence: 99%

Robotic Grasping from Classical to Modern: A Survey

Zhang¹,

Tang²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Object segmentation for robotic picking tasks. The authors of [39] provide a method for unseen object instance segmentation for robotic environments by using an encoder/decoder CNN architecture with clustering based on Hough voting. The authors of [8] predict classagnostic segmentation of objects from depth images using a variant of Mask R-CNN [13].…”

Section: Related Workmentioning

confidence: 99%

“…This additional information is used to identify and pick specific objects or a class of objects, and to understand the relation between objects in a cluttered scene. Other researchers focused on class-agnostic object instance segmentation [16,39], arguing that pixel-accurate instance segmentation of unknown objects is highly important for robotic picking, as it helps to understand each object's shape and location accurately and can be used to avoid potential collisions during grasping (as shown in [27]). Although these methods provide highly accurate results for instance segmentation, they do not directly provide grasp candidates.…”

Section: Introductionmentioning

confidence: 99%

Depth-aware Object Segmentation and Grasp Detection for Robotic Picking Tasks

Ainetter¹,

Böhm²,

Rohit³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we present a novel deep neural network architecture for joint classagnostic object segmentation and grasp detection for robotic picking tasks using a parallelplate gripper. We introduce depth-aware Coordinate Convolution (CoordConv), a method to increase accuracy for point proposal based object instance segmentation in complex scenes without adding any additional network parameters or computation complexity. Depth-aware CoordConv uses depth data to extract prior information about the location of an object to achieve highly accurate object instance segmentation. These resulting segmentation masks, combined with predicted grasp candidates, lead to a complete scene description for grasping using a parallel-plate gripper. We evaluate the accuracy of grasp detection and instance segmentation on challenging robotic picking datasets, namely Siléane and OCID_grasp, and show the benefit of joint grasp detection and segmentation on a real-world robotic picking task.

show abstract

“…to predicted object centers. The training loss is designed as the 𝐿 2 loss between the predicted and the ground-truth offsets [47]. The network is trained independently, since joint end-to-end training with the following networks has been observed to cause instability during training.…”

Section: Instance Segmentation In Dense Cluttermentioning

confidence: 99%

CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulation

Wen¹,

Lian²,

Bekris³

et al. 2021

Preprint

View full text Add to dashboard Cite

Task-relevant grasping is critical for industrial assembly, where downstream manipulation tasks constrain the set of valid grasps. Learning how to perform this task, however, is challenging, since task-relevant grasp labels are hard to define and annotate. There is also yet no consensus on proper representations for modeling or off-the-shelf tools for performing task-relevant grasps. This work proposes a framework to learn task-relevant grasping for industrial objects without the need of time-consuming real-world data collection or manual annotation. To achieve this, the entire framework is trained solely in simulation, including supervised training with synthetic label generation and self-supervised, hand-object interaction. In the context of this framework, this paper proposes a novel, object-centric canonical representation at the category level, which allows establishing dense correspondence across object instances and transferring task-relevant grasps to novel instances. Extensive experiments on task-relevant grasping of densely-cluttered industrial objects are conducted in both simulation and real-world setups, demonstrating the effectiveness of the proposed framework. Code and data will be released upon acceptance at https://sites.google.com/view/catgrasp.

show abstract

Unseen Object Instance Segmentation for Robotic Environments

Cited by 104 publications

References 60 publications

Robotic Grasping from Classical to Modern: A Survey

Robotic Grasping from Classical to Modern: A Survey

Depth-aware Object Segmentation and Grasp Detection for Robotic Picking Tasks

CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulation

Contact Info

Product

Resources

About