A Weakly Supervised Semi-Automatic Image Labeling Approach for Deformable Linear Objects

Caporali, Alessio; Pantano, Matteo; Janisch, Lucas; Regulin, Daniel; Palli, Gianluca; Lee, Dongheui

doi:10.1109/lra.2023.3234799

Cited by 11 publications

(6 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, as BlenderProc is a generic solution, it has some limitations when working with deformable objects. Therefore, in [33] the DLOs had to be modeled as spline curves, reducing slightly the level of realism as the randomness of their shape was not captured. A different Blender-based solution was presented in [34] to generate realistic plant images.…”

Section: Synthetic Image Segmentation Datasetsmentioning

confidence: 99%

“…This solution has been adopted by several authors. Two examples of this are Adam et al, who extended the approach by combining it with domain randomization in [32], and Caporali et al, who employed it to generate synthetic cable images in [33]. However, as BlenderProc is a generic solution, it has some limitations when working with deformable objects.…”

Section: Synthetic Image Segmentation Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Generation of realistic synthetic cable images to train Deep Learning segmentation models

Fresnillo,

Mohammed,

Vasudevan

et al. 2024

Preprint

View full text Add to dashboard Cite

One of the main contemporary challenges in robotics is the automation of tasks that involve handling Deformable Linear Objects (DLOs), such as cables or ropes. Due to their changing shape, constant perception is required, which makes computer vision, and in particular, image segmentation, fundamental. Even though image segmentation is a very studied problem, which has been addressed with high accuracy by many different Deep Learning models, they need to be properly trained to segment the objects of interest (DLOs in this case). To do this, it is necessary to have a large and diverse dataset. However, such a dataset hasn’t been created for cable images, and doing it manually would be complicated and extremely time-consuming. This paper addresses this issue with a novel methodology, which automatically generates synthetic cable datasets to train image segmentation models. This methodology utilizes Blender to create photo-realistic scenes and a Python pipeline to interact with them. To ensure the diversity of the dataset, before the generation of each image, the pipeline performs random variations on the elements of the scene. After this, the scene is animated, dropping the cables from a certain height and letting them deform naturally after landing, which results in realistic arrangements of the cables. The effectiveness of the methodology was demonstrated by training six popular segmentation models with synthetic datasets and using them to segment real cable images with great results (IoU over 70% and Dice coefficient over 80% for all the models).

show abstract

Section: Synthetic Image Segmentation Datasetsmentioning

confidence: 99%

Section: Synthetic Image Segmentation Datasetsmentioning

confidence: 99%

Generation of realistic synthetic cable images to train Deep Learning segmentation models

Fresnillo,

Mohammed,

Vasudevan

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…This solution has been adopted by several authors. Two examples of this are Adam et al, who extended the approach by combining it with domain randomization in [39], and Caporali et al, who employed it to generate synthetic cable images in [40]. However, as BlenderProc is a generic solution, it has some limitations when working with deformable objects.…”

Section: Synthetic Image Segmentation Datasetsmentioning

confidence: 99%

“…However, as BlenderProc is a generic solution, it has some limitations when working with deformable objects. Therefore, in [40] the DLOs had to be modeled as spline curves, reducing slightly the level of realism as the randomness of their shape was not captured. A different Blender-based solution was presented in [41] to generate realistic plant images.…”

Section: Synthetic Image Segmentation Datasetsmentioning

confidence: 99%

Generation of realistic synthetic cable images to train deep learning segmentation models

MalvidoFresnillo,

Mohammed,

Vasudevan

et al. 2024

Machine Vision and Applications

View full text Add to dashboard Cite

Semantic segmentation is one of the most important and studied problems in machine vision, which has been solved with high accuracy by many deep learning models. However, all these models present a significant drawback, they require large and diverse datasets to be trained. Gathering and annotating all these images manually would be extremely time-consuming, hence, numerous researchers have proposed approaches to facilitate or automate the process. Nevertheless, when the objects to be segmented are deformable, such as cables, the automation of this process becomes more challenging, as the dataset needs to represent their high diversity of shapes while keeping a high level of realism, and none of the existing solutions have been able to address it effectively. Therefore, this paper proposes a novel methodology to automatically generate highly realistic synthetic datasets of cables for training deep learning models in image segmentation tasks. This methodology utilizes Blender to create photo-realistic cable scenes and a Python pipeline to introduce random variations and natural deformations. To prove its performance, a dataset composed of 25000 synthetic cable images and their corresponding masks was generated and used to train six popular deep learning segmentation models. These models were then utilized to segment real cable images achieving outstanding results (over 70% IoU and 80% Dice coefficient for all the models). Both the methodology and the generated dataset are publicly available in the project’s repository.

show abstract

“…From a perception standpoint, dealing with DLOs is a tough task. Their ambiguity can make it difficult to distinguish different parts of them or to distinguish DLOs from other objects [3], especially given their relatively small size [4], [5]. Moreover, the detection of the full DLO state, including the twist along the DLO, poses also a challenge [6], [7].…”

Section: Introductionmentioning

confidence: 99%

Deformable Linear Objects Manipulation With Online Model Parameters Estimation

Caporali,

Kicki,

Galassi

et al. 2024

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Manipulating Deformable Linear Objects (DLOs) is a challenging task for a robotic system due to their unpredictable configuration, high-dimensional state space and complex nonlinear dynamics. This paper presents a framework addressing the manipulation of DLOs, specifically targeting the model-based shape control task with the simultaneous online gradient-based estimation of model parameters. In the proposed framework, a neural network is trained to mimic the DLO dynamics using the data generated with an analytical DLO model for a broad spectrum of its parameters. The neural network-based DLO model is conditioned on these parameters and employed in an online phase to perform the shape control task by estimating the optimal manipulative action through a gradient-based procedure. In parallel, gradient-based optimization is used to adapt the DLO model parameters to make the neural network-based model better capture the dynamics of the real-world DLO being manipulated and match the observed deformations. To assess its effectiveness, the framework is tested across a variety of DLOs, surfaces, and target shapes in a series of experiments. The results of these experiments demonstrate the validity and efficiency of the proposed methodology compared to existing methods. Project website at https://sites.google.com/view/dlo-manipulation.

show abstract

A Weakly Supervised Semi-Automatic Image Labeling Approach for Deformable Linear Objects

Cited by 11 publications

References 47 publications

Generation of realistic synthetic cable images to train Deep Learning segmentation models

Generation of realistic synthetic cable images to train Deep Learning segmentation models

Generation of realistic synthetic cable images to train deep learning segmentation models

Deformable Linear Objects Manipulation With Online Model Parameters Estimation

Contact Info

Product

Resources

About