YouTube-VOS: Sequence-to-Sequence Video Object Segmentation

Xu, Ning; Yang, Linjie; Yi, Fan; Yang, Jianchao; Yue, Dingcheng; Liang, Yuchen; Price, Brian; Cohen, Scott; Huang, Thomas S.

doi:10.48550/arxiv.1809.00461

Cited by 7 publications

(7 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We don't run the original PReMVOS method but the PReMVOS Fast-finetuned version (see Section 4.3), which can be evaluated on this larger dataset in a more reasonable amount of time. Our results are much better than [33], the only other method that has published results on this dataset. It is also better than all other methods that submitted results to the 2017 1st Large-scale Video Object Segmentation Challenge.…”

Section: Further Large-scale Evaluationmentioning

confidence: 60%

“…In Table 6 we present the results of PReMVOS on the new YouTube-VOS dataset [33] our results on the test set obtained 1st place in the 1st Large-scale Video Object Segmentation Challenge. We don't run the original PReMVOS method but the PReMVOS Fast-finetuned version (see Section 4.3), which can be evaluated on this larger dataset in a more reasonable amount of time.…”

Section: Further Large-scale Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

PReMVOS: Proposal-Generation, Refinement and Merging for Video Object Segmentation

Luiten

Voigtlaender

Leibe

2019

Lecture Notes in Computer Science

240

263

View full text Add to dashboard Cite

We address semi-supervised video object segmentation, the task of automatically generating accurate and consistent pixel masks for objects in a video sequence, given the first-frame ground truth annotations. Towards this goal, we present the PReMVOS algorithm (Proposalgeneration, Refinement and Merging for Video Object Segmentation). Our method separates this problem into two steps, first generating a set of accurate object segmentation mask proposals for each video frame and then selecting and merging these proposals into accurate and temporally consistent pixel-wise object tracks over a video sequence in a way which is designed to specifically tackle the difficult challenges involved with segmenting multiple objects across a video sequence. Our approach surpasses all previous state-of-the-art results on the DAVIS 2017 video object segmentation benchmark with a J &F mean score of 71.6 on the test-dev dataset, and achieves first place in both the DAVIS 2018 Video Object Segmentation Challenge and the YouTube-VOS 1st Large-scale Video Object Segmentation Challenge.

show abstract

Section: Further Large-scale Evaluationmentioning

confidence: 60%

Section: Further Large-scale Evaluationmentioning

confidence: 99%

PReMVOS: Proposal-Generation, Refinement and Merging for Video Object Segmentation

Luiten

Voigtlaender

Leibe

2019

Lecture Notes in Computer Science

240

263

View full text Add to dashboard Cite

show abstract

“…( 7)) for temporally coherent predictions. We use videos in the Youtube-VOS dataset [29] as ground-truth for the training. It is a large-scale dataset for video object segmentation containing 4000+ YouTube videos with 70+ common objects.…”

Section: Two-stage Trainingmentioning

confidence: 99%

“…In the context of the video object removal task, masks with the most realistic appearance and motion can be obtained from video object segmentation datasets. We use the foreground segmentation masks of the YouTube-VOS dataset [29].…”

Section: Video Object Maskmentioning

confidence: 99%

Deep Video Inpainting

Kim

Woo

Lee

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

188

102

View full text Add to dashboard Cite

Video inpainting aims to fill spatio-temporal holes with plausible content in a video. Despite tremendous progress of deep neural networks for image inpainting, it is challenging to extend these methods to the video domain due to the additional time dimension. In this work, we propose a novel deep network architecture for fast video inpainting. Built upon an image-based encoder-decoder model, our framework is designed to collect and refine information from neighbor frames and synthesize still-unknown regions. At the same time, the output is enforced to be temporally consistent by a recurrent feedback and a temporal memory module. Compared with the state-of-the-art image inpainting algorithm, our method produces videos that are much more semantically correct and temporally smooth. In contrast to the prior video completion method which relies on time-consuming optimization, our method runs in near realtime while generating competitive video results. Finally, we applied our framework to video retargeting task, and obtain visually pleasing results.

show abstract

“…The second set of approaches formulates segmentation as a foreground-background classification task, detecting regions that correspond to foreground objects and matching the resulting appearance models with other information such as salience maps, shape estimates, and pairwise constraints [32,22,46]. The third set of approaches incorporate the classification approach with a memory module for propagating region estimates in time [44,48]. The latter set of approaches begin to incorporate temporal dynamics information into the learning problem, and due to their success, motivate the method, observation, and dataset proposed in our work.…”

Section: Semantic Segmentation In Videomentioning

confidence: 99%

Value of Temporal Dynamics Information in Driving Scene Segmentation

Ding¹,

Terwilliger²,

Sherony³

et al. 2019

Preprint

View full text Add to dashboard Cite

Semantic scene segmentation has primarily been addressed by forming representations of single images both with supervised and unsupervised methods. The problem of semantic segmentation in dynamic scenes has begun to recently receive attention with video object segmentation approaches. What is not known is how much extra information the temporal dynamics of the visual scene carries that is complimentary to the information available in the individual frames of the video. There is evidence that the human visual system can effectively perceive the scene from temporal dynamics information of the scene's changing visual characteristics without relying on the visual characteristics of individual snapshots themselves. Our work takes steps to explore whether machine perception can exhibit similar properties by combining appearance-based representations and temporal dynamics representations in a joint-learning problem that reveals the contribution of each toward successful dynamic scene segmentation. Additionally, we provide the MIT Driving Scene Segmentation dataset, which is a large-scale full driving scene segmentation dataset, densely annotated for every pixel and every one of 5,000 video frames. This dataset is intended to help further the exploration of the value of temporal dynamics information for semantic segmentation in video.

show abstract

YouTube-VOS: Sequence-to-Sequence Video Object Segmentation

Cited by 7 publications

References 25 publications

PReMVOS: Proposal-Generation, Refinement and Merging for Video Object Segmentation

PReMVOS: Proposal-Generation, Refinement and Merging for Video Object Segmentation

Deep Video Inpainting

Value of Temporal Dynamics Information in Driving Scene Segmentation

Contact Info

Product

Resources

About