A Sim-to-Real Object Recognition and Localization Framework for Industrial Robotic Bin Picking

Li, Xianzhi; Cao, Rui; Feng, Yidan; Chen, Kai; Yang, Biqi; Fu, Chi‐Wing; Li, Yichuan; Dou, Qi; Liu, Yunhui; Heng, Pheng‐Ann

doi:10.1109/lra.2022.3149026

Cited by 39 publications

(11 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, computer graphics-based synthetic images allow control over the class specification, objects' location, and synthetic image style. Studies have used computer graphics to produce synthetic training images for object detection tasks [18], including industrial object detection [2], [19]. Mangat et al [2] generated synthetic images for an object-picking task without considering realistic appearance, whereas, Li et al [19] used synthetic images for industrial bin packing from small cluttered parts.…”

Section: B Synthetic Training Data In Industrial Inspectionmentioning

confidence: 99%

Generating Synthetic Training Images to Detect Split Defects in Stamped Components

Singh,

Bashford-Rogers,

Hazra

et al. 2024

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

show abstract

Section: B Synthetic Training Data In Industrial Inspectionmentioning

confidence: 99%

Generating Synthetic Training Images to Detect Split Defects in Stamped Components

Singh,

Bashford-Rogers,

Hazra

et al. 2024

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

show abstract

“…The second is on adapting to unseen objects with least data requirement. Some methods use sim-to-real technique [21], [22], [4], [23], [6], [5], [24] while others [9], [25] harness the strength of knowledge distillation, trained in a weaklysupervised manner. Meanwhile, various datasets are released for facilitating studies on instance segmentation for autostore.…”

Section: A Instance Segmentation For Auto-storementioning

confidence: 99%

On Improving Boundary Quality of Instance Segmentation in Cluttered and Chaotic Scenarios

Yang

Gao

et al. 2023

2023 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

In large-scale storehouses, precise instance masks are crucial for robotic bin picking but are challenging to obtain. Existing instance segmentation methods typically rely on a tedious process of scene collection, mask annotation, and network fine-tuning for every single Stock Keeping Unit (SKU). This paper presents SKU-Patch, a new patch-guided instance segmentation solution, leveraging only a few image patches for each incoming new SKU to predict accurate and robust masks, without tedious manual effort and model re-training. Technical-wise, we design a novel transformer-based network with (i) a patch-image correlation encoder to capture multi-level image features calibrated by patch information and (ii) a patchaware transformer decoder with parallel task heads to generate instance masks. Extensive experiments on four storehouse benchmarks manifest that SKU-Patch is able to achieve the best performance over the state-of-the-art methods. Also, SKU-Patch yields an average of nearly 100% grasping success rate on more than 50 unseen SKUs in a robot-aided auto-store logistic pipeline, showing its effectiveness and practicality.

show abstract

“…Though challenging, the perception requirement is not unique to tool use. General robot manipulation also requires similar perceptual capabilities ( Li et al., 2022 ).…”

Section: Introductionmentioning

confidence: 99%

Robot tool use: A survey

Qin

Brawer²,

Scassellati³

2023

Front. Robot. AI

View full text Add to dashboard Cite

Using human tools can significantly benefit robots in many application domains. Such ability would allow robots to solve problems that they were unable to without tools. However, robot tool use is a challenging task. Tool use was initially considered to be the ability that distinguishes human beings from other animals. We identify three skills required for robot tool use: perception, manipulation, and high-level cognition skills. While both general manipulation tasks and tool use tasks require the same level of perception accuracy, there are unique manipulation and cognition challenges in robot tool use. In this survey, we first define robot tool use. The definition highlighted the skills required for robot tool use. The skills coincide with an affordance model which defined a three-way relation between actions, objects, and effects. We also compile a taxonomy of robot tool use with insights from animal tool use literature. Our definition and taxonomy lay a theoretical foundation for future robot tool use studies and also serve as practical guidelines for robot tool use applications. We first categorize tool use based on the context of the task. The contexts are highly similar for the same task (e.g., cutting) in non-causal tool use, while the contexts for causal tool use are diverse. We further categorize causal tool use based on the task complexity suggested in animal tool use studies into single-manipulation tool use and multiple-manipulation tool use. Single-manipulation tool use are sub-categorized based on tool features and prior experiences of tool use. This type of tool may be considered as building blocks of causal tool use. Multiple-manipulation tool use combines these building blocks in different ways. The different combinations categorize multiple-manipulation tool use. Moreover, we identify different skills required in each sub-type in the taxonomy. We then review previous studies on robot tool use based on the taxonomy and describe how the relations are learned in these studies. We conclude with a discussion of the current applications of robot tool use and open questions to address future robot tool use.

show abstract

A Sim-to-Real Object Recognition and Localization Framework for Industrial Robotic Bin Picking

Cited by 39 publications

References 49 publications

Generating Synthetic Training Images to Detect Split Defects in Stamped Components

Generating Synthetic Training Images to Detect Split Defects in Stamped Components

On Improving Boundary Quality of Instance Segmentation in Cluttered and Chaotic Scenarios

Robot tool use: A survey

Contact Info

Product

Resources

About