Self-supervised Interactive Object Segmentation Through a Singulation-and-Grasping Approach

Yu, Houjian; Choi, Changhyun

doi:10.1007/978-3-031-19842-7_36

Cited by 7 publications

(3 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…UOIS [1]- [3] aims to detect all arbitrary object instances in an image. It serves various robotic applications, including object pushing [4], [5], grasping [6]- [8], and re-arrangement [22]- [25]. Various segmentation methods, such as graphbased segmentation [26], surface patches with SVM [27], and ambiguity graph [28] have been proposed, and recent advancements in deep learning have led to the introduction of category-agnostic instance segmentation networks trained on large-scale synthetic data [1]- [3], [8]- [10], [13], [29].…”

Section: A Unseen Object Instance Segmentationmentioning

confidence: 99%

“…Unseen Object Instance Segmentation (UOIS) [1]- [3], the task of segmenting untrained objects in cluttered scenes, is essential for robotic manipulation such as object pushing [4], [5] and grasping [6]- [8]. Recently, category-agnostic instance segmentation networks [1]- [3], [8]- [12], which learn the concept of objectness from large-scale synthetic datasets, have demonstrated state-of-the-art performance.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling

Back¹,

Lee²,

Kim³

et al. 2022

2022 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Efficient and accurate segmentation of unseen objects is crucial for robotic manipulation. However, it remains challenging due to over-or under-segmentation. Although existing refinement methods can enhance the segmentation quality, they fix only minor boundary errors or are not sufficiently fast. In this work, we propose INSTAnce Boundary Explicit Error Estimation and Refinement (INSTA-BEEER), a novel refinement model that allows for adding and deleting instances and sharpening boundaries. Leveraging an error-estimationthen-refinement scheme, the model first estimates the pixelwise boundary explicit errors: true positive, true negative, false positive, and false negative pixels of the instance boundary in the initial segmentation. It then refines the initial segmentation using these error estimates as guidance. Experiments show that the proposed model significantly enhances segmentation, achieving state-of-the-art performance. Furthermore, with a fast runtime (less than 0.1 s), the model consistently improves performance across various initial segmentation methods, making it highly suitable for practical robotic applications.

show abstract

Section: A Unseen Object Instance Segmentationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling

Back¹,

Lee²,

Kim³

et al. 2022

2022 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…However, masks recovered from naive image subtraction are imperfect, resulting in worse performance at higher IOU thresholds, which is insufficient for high-performing robotic applications. Optical flow methods can also infer contiguous groups of pixels that move together as the robot is pushing them around [15], [7], [16]. This may be an impractical approach if continuous high-bandwidth video is not available from the robot camera, or if the robot is expected to be grasping objects instead of pushing them in a production environment.…”

Section: B Self-supervised Segmentationmentioning

confidence: 99%

Architecture Colorization via Self-supervised Learning and Instance Segmentation

Liu

Chen

2022

Pattern Recognition and Computer Vision

View full text Add to dashboard Cite

Instance segmentation is a fundamental skill for many robotic applications. We propose a self-supervised method that uses grasp interactions to collect segmentation supervision for an instance segmentation model. When a robot grasps an item, the mask of that grasped item can be inferred from the images of the scene before and after the grasp. Leveraging this insight, we learn a grasp segmentation model to segment the grasped object from before and after grasp images. Such a model can segment grasped objects from thousands of grasp interactions without costly human annotation. Using the segmented grasped objects, we can "cut" objects from their original scenes and "paste" them into new scenes to generate instance supervision. We show that our grasp segmentation model provides a 5x error reduction when segmenting grasped objects compared with traditional image subtraction approaches. Combined with our "cut-and-paste" generation method, instance segmentation models trained with our method achieve better performance than a model trained with 10x the amount of labeled data. On a real robotic grasping system, our instance segmentation model reduces the rate of grasp errors by over 3x compared to an image subtraction baseline.

show abstract

Manipulator Reinforcement Learning with Mask Processing Based on Residual Network

Wang,

et al. 2023

2023 35th Chinese Control and Decision Conference (CCDC)

View full text Add to dashboard Cite

Self-supervised Interactive Object Segmentation Through a Singulation-and-Grasping Approach

Cited by 7 publications

References 36 publications

Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling

Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling

Architecture Colorization via Self-supervised Learning and Instance Segmentation

Manipulator Reinforcement Learning with Mask Processing Based on Residual Network

Contact Info

Product

Resources

About