Semantic Co-segmentation in Videos

Tsai, Yi Hsuan; Zhong, Guangyu; Yang, Ming–Hsuan

doi:10.1007/978-3-319-46493-0_46

Cited by 80 publications

(55 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semantic co-segmentation methods from a single video use spatio-temporal object proposals [28,40], segments [31], motion [49] and foreground propagation [20]. Recently, cosegmentation methods were introduced to segment common objects in a collection of videos for a single object [19] or multiple objects [11,54].…”

Section: Semantic Segmentationmentioning

confidence: 99%

“…Sequential methods for frame-to-frame temporal coherence are prone to errors due to drift and rapid motion [6,46]. Previous work [54] has shown that semantic tracklets improve segmentation for single view video. To achieve robust temporally coherent reconstruction semantic tracklets are introduced linking instances of dynamic objects across wide-timeframes.…”

Section: Semantic Trackletsmentioning

confidence: 99%

“…Comparison of segmentation on public datasets against state-of-the-art methods: TcMVS [42] (Region in red represents region missing from ground-truth and green represents region not present in ground-truth), CRF-RNN [57] and SCV [54].…”

Section: Multi-view Evaluationmentioning

confidence: 99%

“…Multi tion (MVVS) [13], Joint segmentation and reconstruction (TcMVS) [42], and (b) Semantic methods: Semantic cosegmentation in videos (SCV) [54] and Conditional random field as recurrent neural networks (CRF-RNN) [57]. Single view methods MVC [11] and ObMiC [19] are also applied independently on each view for comparison.…”

Section: Datasetsmentioning

confidence: 99%

“…Single view methods MVC [11] and ObMiC [19] are also applied independently on each view for comparison. Comparison against MVVS [13] is shown in Figure 7 and evaluation against TcMVS [42], SCV [54] and CRF-RNN [57] are shown in Figure 8 for dynamic datasets. Quantitative evaluation against state-of-the-art methods is measured by Intersection-over-Union with ground-truth, shown in the Table 1.…”

Section: Datasetsmentioning

confidence: 99%

See 4 more Smart Citations

Semantically Coherent Co-Segmentation and Reconstruction of Dynamic Scenes

Mustafa

Hilton

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

In this paper we propose a framework for spatially and temporally coherent semantic co-segmentation and reconstruction of complex dynamic scenes from multiple static or moving cameras. Semantic co-segmentation exploits the coherence in semantic class labels both spatially, between views at a single time instant, and temporally, between widely spaced time instants of dynamic objects with similar shape and appearance. We demonstrate that semantic coherence results in improved segmentation and reconstruction for complex scenes. A joint formulation is proposed for semantically coherent object-based co-segmentation and reconstruction of scenes by enforcing consistent semantic labelling between views and over time. Semantic tracklets are introduced to enforce temporal coherence in semantic labelling and reconstruction between widely spaced instances of dynamic objects. Tracklets of dynamic objects enable unsupervised learning of appearance and shape priors that are exploited in joint segmentation and reconstruction. Evaluation on challenging indoor and outdoor sequences with hand-held moving cameras shows improved accuracy in segmentation, temporally coherent semantic labelling and 3D reconstruction of dynamic scenes.

show abstract

Section: Semantic Segmentationmentioning

confidence: 99%

Section: Semantic Trackletsmentioning

confidence: 99%

Section: Multi-view Evaluationmentioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

See 3 more Smart Citations

Semantically Coherent Co-Segmentation and Reconstruction of Dynamic Scenes

Mustafa

Hilton

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

Unseen Object Segmentation in Videos via Transferable Representations

Chen

Tsai²,

Yang

et al. 2019

Computer Vision – ACCV 2018

Self Cite

View full text Add to dashboard Cite

In order to learn object segmentation models in videos, conventional methods require a large amount of pixel-wise ground truth annotations. However, collecting such supervised data is time-consuming and labor-intensive. In this paper, we exploit existing annotations in source images and transfer such visual information to segment videos with unseen object categories. Without using any annotations in the target video, we propose a method to jointly mine useful segments and learn feature representations that better adapt to the target frames. The entire process is decomposed into two tasks: 1) solving a submodular function for selecting object-like segments, and 2) learning a CNN model with a transferable module for adapting seen categories in the source domain to the unseen target video. We present an iterative update scheme between two tasks to self-learn the final solution for object segmentation. Experimental results on numerous benchmark datasets show that the proposed method performs favorably against the state-of-the-art algorithms.

show abstract