Large Scale Labelled Video Data Augmentation for Semantic Segmentation in Driving Scenarios

Budvytis, Ignas; Sauer, Patrick; Roddick, Thomas; Breen, Kesar; Cipolla, Roberto

doi:10.1109/iccvw.2017.36

Cited by 36 publications

(20 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two typical approaches have been studied. The first approach is based on label propagation that warps labels of sparsely-annotated frames to generate pseudo labels for unannotated frames via patch matching [1,4], motion cues [58,69] or optical flow [47,68,48,17]. The other approach is based on self-training that generates pseudo labels through a distillation across multiple augmentations [5].…”

Section: Video Semantic Segmentationmentioning

confidence: 99%

Domain Adaptive Video Segmentation via Temporal Consistency Regularization

Guan¹,

Huang²,

Xiao³

et al. 2021

Preprint

View full text Add to dashboard Cite

Video semantic segmentation is an essential task for the analysis and understanding of videos. Recent efforts largely focus on supervised video segmentation by learning from fully annotated data, but the learnt models often experience clear performance drop while applied to videos of a different domain. This paper presents DA-VSN, a domain adaptive video segmentation network that addresses domain gaps in videos by temporal consistency regularization (TCR) for consecutive frames of target-domain videos. DA-VSN consists of two novel and complementary designs. The first is cross-domain TCR that guides the prediction of target frames to have similar temporal consistency as that of source frames (learnt from annotated source data) via adversarial learning. The second is intra-domain TCR that guides unconfident predictions of target frames to have similar temporal consistency as confident predictions of target frames. Extensive experiments demonstrate the superiority of our proposed domain adaptive video segmentation network which outperforms multiple baselines consistently by large margins.

show abstract

Section: Video Semantic Segmentationmentioning

confidence: 99%

Domain Adaptive Video Segmentation via Temporal Consistency Regularization

Guan¹,

Huang²,

Xiao³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Budvytis et al [43] investigated the effect of video data augmentation for semantic segmentation in driving environments. They increased the segmentation performance of different networks by performing label propagation from coarsely labeled frames to adjacent unlabeled ones.…”

Section: Data Augmentation For Improving Performancementioning

confidence: 99%

Weakly-Supervised Defect Segmentation on Periodic Textures Using CycleGAN

Kim

Ra³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The importance of an automated defect inspection system has been increasing in the manufacturing industries. Various products to be examined have periodic textures. Among image-based inspection systems, it is common that supervised defect segmentation requires a great number of defect images with their own region-level labels; however, it is difficult to prepare sufficient training data. Because most products are of normal quality, it is difficult to obtain images of product defects. Pixelwise annotation for semantic segmentation tasks is an exhausting and time-consuming process. To solve these problems, we propose a weakly-supervised defect segmentation framework for defect images with periodic textures and a data augmentation process using generative adversarial networks. With only imagelevel labeling, the proposed segmentation framework translates a defect image into its defect-free version, called a golden template, using CycleGAN and then segments the defects by comparing the two images. The proposed augmentation process creates whole new synthetic defect images from real defect images to obtain sufficient data. Furthermore, synthetic non-defect images are generated even from real defect images through the augmentation process. The experimental results demonstrate that the proposed framework with data augmentation outperforms an existing weakly-supervised method and shows remarkable results comparable to those of supervised segmentation methods. INDEX TERMS Automated defect inspection, visual inspection system, weakly-supervised learning, periodic textures, data augmentation, generative adversarial networks.

show abstract

“…A notable exception is the work in [16], where a systematic analysis about the quality of pseudo ground truth (PGT) was presented and the impact of PGT on training a CNN was discussed. Similarly, the effect of large scale PGT on deep learning based classification is investigated in [17]. In such cases, the propagated annotations were merely employed as augmented ground truth data and trained together with manually labeled ground truth.…”

Section: Introductionmentioning

confidence: 99%

Automatic Annotation of Airborne Images by Label Propagation Based on a Bayesian-CRF Model

et al. 2019

View full text Add to dashboard Cite

The tremendous advances in deep neural networks have demonstrated the superiority of deep learning techniques for applications such as object recognition or image classification. Nevertheless, deep learning-based methods usually require a large amount of training data, which mainly comes from manual annotation and is quite labor-intensive. In order to reduce the amount of manual work required for generating enough training data, we hereby propose to leverage existing labeled data to generate image annotations automatically. Specifically, the pixel labels are firstly transferred from one image modality to another image modality via geometric transformation to create initial image annotations, and then additional information (e.g., height measurements) is incorporated for Bayesian inference to update the labeling beliefs. Finally, the updated label assignments are optimized with a fully connected conditional random field (CRF), yielding refined labeling for all pixels in the image. The proposed approach is tested on two different scenarios, i.e., (1) label propagation from annotated aerial imagery to unmanned aerial vehicle (UAV) imagery and (2) label propagation from map database to aerial imagery. In each scenario, the refined image labels are used as pseudo-ground truth data for training a convolutional neural network (CNN). Results demonstrate that our model is able to produce accurate label assignments even around complex object boundaries; besides, the generated image labels can be effectively leveraged for training CNNs and achieve comparable classification accuracy as manual image annotations, more specifically, the per-class classification accuracy of the networks trained by the manual image annotations and the generated image labels have a difference within ± 5 % .

show abstract

Large Scale Labelled Video Data Augmentation for Semantic Segmentation in Driving Scenarios

Cited by 36 publications

References 31 publications

Domain Adaptive Video Segmentation via Temporal Consistency Regularization

Domain Adaptive Video Segmentation via Temporal Consistency Regularization

Weakly-Supervised Defect Segmentation on Periodic Textures Using CycleGAN

Automatic Annotation of Airborne Images by Label Propagation Based on a Bayesian-CRF Model

Contact Info

Product

Resources

About