VSPW: A Large-scale Dataset for Video Scene Parsing in the Wild

Miao, Jiaxu; Wei, Yunchao; Wu, Yu; Chen, Liang; Li, Guangrui; Yang, Yi

doi:10.1109/cvpr46437.2021.00412

Cited by 83 publications

(81 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following experiments are conducted on VSPW2021 test dataset and mIoU score is reported. The baseline is the original Spatial-Temporal PPM with ResNet-101 backbone [8] We apply Swin Transformer-Large as backbone for Spatial-Temporal PPM, which achieves better mIoU score of 43.63%, with a significant improvement of 6.17%. Then we demonstrate the effectiveness of our proposed Bilateral Network with Vision Transformer, which can bring an improvement of 2.10%.…”

Section: Ablation Studymentioning

confidence: 99%

“…The video scene parsing is primarily a task of video semantic segmentation, but it is more difficult and challenging due to the wide range of real-world scenarios and dense annotation for every pixel, that's to say we must segment the specific categories for backgrounds(e.g., road, wall, sky). VSPW [8] is a suitable benchmark to tackle the challenging video scene parsing task.…”

Section: Introductionmentioning

confidence: 99%

“…Deep learning based methods have achieved promising results for image semantic segmentation over the past few years, such as DeepLab series [3,2,4], OCRNet [14], BiSeNet series [13,12], etc. Several methods are proposed to solve the video smantic segmentation task, TDNet [6], TMANet [11] and TCBNet [8] are latest representative methods, which can capture temporal information between frames by particular design of temporal module.…”

Section: Introductionmentioning

confidence: 99%

“…Inspired by BiSeNet [13] and Swin Transformer [7], we design a bilateral network with vision transformer, spatial path with convolutions to generate low level features which can preserve the spatial information, and context path with vision transformer is employed to obtain sufficient context information. Finally, in order to effectively harness temporal information from the past frames, we use the method proposed by TCBNet [8] to get a better and more stable segment results.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

TBN-ViT: Temporal Bilateral Network with Vision Transformer for Video Scene Parsing

Yan¹,

Cao²,

Wang³

2021

Preprint

View full text Add to dashboard Cite

Video scene parsing in the wild with diverse scenarios is a challenging and great significance task, especially with the rapid development of automatic driving technique. The dataset Video Scene Parsing in the Wild(VSPW) contains well-trimmed long-temporal, dense annotation and high resolution clips. Based on VSPW, we design a Temporal Bilateral Network with Vision Transformer. We first design a spatial path with convolutions to generate low level features which can preserve the spatial information. Meanwhile, a context path with vision transformer is employed to obtain sufficient context information. Furthermore, a temporal context module is designed to harness the inter-frames contextual information. Finally, the proposed method can achieve the mean intersection over union(mIoU) of 49.85% for the VSPW2021 Challenge test dataset.

show abstract

Section: Ablation Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

TBN-ViT: Temporal Bilateral Network with Vision Transformer for Video Scene Parsing

Yan¹,

Cao²,

Wang³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…To address these issues, we introduce a new benchmark that more closely aligns with the target task. Our benchmark uses the PASCAL-VOC dataset [19] as the training set and we re-purpose a densely labelled video semantic segmentation dataset, VSPW [20], as the test set. We refer to this cross-domain dataset as PASCAL-to-MiniVSPW.…”

Section: Introductionmentioning

confidence: 99%

Temporal Transductive Inference for Few-Shot Video Object Segmentation

Siam¹,

Derpanis²,

Wildes³

2022

Preprint

View full text Add to dashboard Cite

Few-shot video object segmentation (FS-VOS) aims at segmenting video frames using a few labelled examples of classes not seen during initial training. In this paper, we present a simple but effective temporal transductive inference (TTI) approach that leverages temporal consistency in the unlabelled video frames during few-shot inference. Key to our approach is the use of both global and local temporal constraints. The objective of the global constraint is to learn consistent linear classifiers for novel classes across the image sequence, whereas the local constraint enforces the proportion of foreground/background regions in each frame to be coherent across a local temporal window. These constraints act as spatiotemporal regularizers during the transductive inference to increase temporal coherence and reduce overfitting on the few-shot support set. Empirically, our model outperforms state-of-the-art meta-learning approaches in terms of mean intersection over union on YouTube-VIS by 2.8%. In addition, we introduce improved benchmarks that are exhaustively labelled (i.e. all object occurrences are labelled, unlike the currently available), and present a more realistic evaluation paradigm that targets data distribution shift between training and testing sets. Our empirical results and in-depth analysis confirm the added benefits of the proposed spatiotemporal regularizers to improve temporal coherence and overcome certain overfitting scenarios.

show abstract

NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion

Liang

et al. 2022

Lecture Notes in Computer Science

109

View full text Add to dashboard Cite

VSPW: A Large-scale Dataset for Video Scene Parsing in the Wild

Cited by 83 publications

References 47 publications

TBN-ViT: Temporal Bilateral Network with Vision Transformer for Video Scene Parsing

TBN-ViT: Temporal Bilateral Network with Vision Transformer for Video Scene Parsing

Temporal Transductive Inference for Few-Shot Video Object Segmentation

NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion

Contact Info

Product

Resources

About