Automatic Image Generation Pipeline for Instance Segmentation of Deformable Linear Objects

Dirr, Jonas; Gebauer, Daniel; Yao, Jiajun; Daub, Rüdiger

doi:10.3390/s23063013

Cited by 11 publications

(11 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the preceding work in [24], synthetic training data is generated by simulation and rendering with blender. There, the same electric cable dataset is used for testing as in Sect.…”

Section: Discussion and Limitationsmentioning

confidence: 99%

“…One test dataset of about 80 images with diverse scenes and backgrounds is available for each part type. The test images of the rigid parts are captured with the Google Pixel 4a smartphone, while cables [22] and hoses are imaged with the industrial camera rc_visard 65 color from Roboception. The corresponding labels are prepared manually using Hasty 4 and labelme 5 .…”

Section: Experiments 1: Instance Segmentationmentioning

confidence: 99%

See 1 more Smart Citation

Cut-paste image generation for instance segmentation for robotic picking of industrial parts

Dirr,

Bauer,

Gebauer

et al. 2023

Int J Adv Manuf Technol

Self Cite

View full text Add to dashboard Cite

Vision-based robotic picking enables automation of commissioning and sortation of disordered parts. To locate parts for grasping, state-of-the-art approaches rely on convolutional neural networks for instance segmentation in 2D images. However, this requires sufficiently large training datasets, which are expensive to capture and annotate. Therefore, training with synthetic data is promising as the data can be generated automatically. We present an approach for the cut-paste method to create synthetic images for industrial use cases. With this approach, an end-user first prepares the image generation with just a smartphone and about 20 minutes of manual effort. Then, a versatile dataset with instance segmentation labels is generated automatically. In addition, a procedure for grasp pose computation is applied to enable robotic picking based on instance segmentation. For evaluation, training data is generated for a wide range of rigid parts and deformable linear objects. Testing with real-world data and practical experiments demonstrates the effectiveness of the proposed cut-paste method for industrial applications.

show abstract

“…In the preceding work in [24], synthetic training data is generated by simulation and rendering with blender. There, the same electric cable dataset is used for testing as in Sect.…”

Section: Discussion and Limitationsmentioning

confidence: 99%

Section: Experiments 1: Instance Segmentationmentioning

confidence: 99%

Cut-paste image generation for instance segmentation for robotic picking of industrial parts

Dirr,

Bauer,

Gebauer

et al. 2023

Int J Adv Manuf Technol

Self Cite

View full text Add to dashboard Cite

show abstract

“…The open-source 3D creation software Blender is a popular tool amongst many researchers to generate synthetic training images for computer vision tasks, e.g., [ 19 , 45 , 46 , 47 ]. Blender utilizes a path tracing rendering engine called Cycles for producing physically-based renders and can be automated using its Python API.…”

Section: Methodsmentioning

confidence: 99%

Combining Synthetic Images and Deep Active Learning: Data-Efficient Training of an Industrial Object Detection Model

Eversberg,

Lambrecht

2024

J. Imaging

View full text Add to dashboard Cite

Generating synthetic data is a promising solution to the challenge of limited training data for industrial deep learning applications. However, training on synthetic data and testing on real-world data creates a sim-to-real domain gap. Research has shown that the combination of synthetic and real images leads to better results than those that are generated using only one source of data. In this work, the generation of synthetic training images via physics-based rendering is combined with deep active learning for an industrial object detection task to iteratively improve model performance over time. Our experimental results show that synthetic images improve model performance, especially at the beginning of the model’s life cycle with limited training data. Furthermore, our implemented hybrid query strategy selects diverse and informative new training images in each active learning cycle, which outperforms random sampling. In conclusion, this work presents a workflow to train and iteratively improve object detection models with a small number of real-world images, leading to data-efficient and cost-effective computer vision models.

show abstract

“…The aim of this work is to recognise surgical instruments on an instrument tray by applying the crop-and-past method to an existing real data set in order to artificially expand it. The method defined by Lehr, R. et al 5 , the purely rendering-based data generation of is used in the work of Dirr, J. et al 3 , where deformable pipe sections are to be recognised using purely synthetic data. In addition, different degrees of abstraction in the generated models are analysed for their applicability.…”

Section: Synthetic Data Generationmentioning

confidence: 99%

“…One advantage of this method is the relatively low effort required to expand an existing data set. A major disadvantage is that all images used come from the same environment as the target images, which leads to a small domain gap 3 , which in turn results in a low generalisability of the networks. The second strategy entails the fabrication of the data set by rendering scenes using sophisticated computer-generated imagery.…”

Section: Introductionmentioning

confidence: 99%

Medical instrument detection with synthetically generated data

Wiese,

Hinz,

Reithmeier

2024

Medical Imaging 2024: Imaging Informatics for Healthcare, Research, and Applications

View full text Add to dashboard Cite

The persistent need for more qualified personnel in operating theatres exacerbates the remaining staff's workload. This increased burden can result in substantial complications during surgical procedures. To address this issue, this research project works on a comprehensive operating theatre system. The system offers real-time monitoring of all surgical instruments in the operating theatre, aiming to alleviate the problem. The foundation of this endeavour involves a neural network trained to classify and identify eight distinct instruments belonging to four distinct surgical instrument groups. A novel aspect of this study lies in the approach taken to select and generate the training and validation data sets. The data sets used in this study consist of synthetically generated image data rather than real image data. Additionally, three virtual scenes were designed to serve as the background for a generation algorithm. This algorithm randomly positions the instruments within these scenes, producing annotated rendered RGB images of the generated scenes. To assess the efficacy of this approach, a separate real data set was also created for testing the neural network. Surprisingly, it was discovered that neural networks trained solely on synthetic data performed well when applied to real data. This research paper shows that it is possible to train neural networks with purely synthetically generated data and use them to recognise surgical instruments in real images.

show abstract

Automatic Image Generation Pipeline for Instance Segmentation of Deformable Linear Objects

Cited by 11 publications

References 33 publications

Cut-paste image generation for instance segmentation for robotic picking of industrial parts

Cut-paste image generation for instance segmentation for robotic picking of industrial parts

Combining Synthetic Images and Deep Active Learning: Data-Efficient Training of an Industrial Object Detection Model

Medical instrument detection with synthetically generated data

Contact Info

Product

Resources

About