Unsupervised Video Object Segmentation with Motion-Based Bilateral Networks

Li, Siyang; Seybold, Bryan; Vorobyov, Alexey; Lei, Xuejing; Kuo, C.-C. Jay

doi:10.1007/978-3-030-01219-9_13

Cited by 140 publications

(104 citation statements)

References 34 publications

Supporting

Mentioning

104

Contrasting

Order By: Relevance

“…In Table 3, several other deep learning based state-ofthe-art UVOS methods [9,52,24,53,33] leverage both appearance as well as extra motion information to improve the performance. Different from these methods, the proposed COSNet only utilizes appearance information but achieves superior performance.…”

Section: Quantitative and Qualitative Resultsmentioning

confidence: 99%

“…We also perform experiments on the FBMS dataset for completeness. Table 4 shows that our COSNet performs better (75.6% in mean J ) than stateof-the-art methods [14,42,24,21,30,32,33,49,9]. In most competing methods, except for the RGB input, additional optical flow information is utilized to estimate the segmentation mask.…”

Section: Quantitative and Qualitative Resultsmentioning

confidence: 99%

“…For example, Tokmakov et al [52] proposed to separate independent object and camera motion using a learnable motion pattern network [52]. Li et al learned an instance embedding network [32] from static images to better locate the object(s), and later they combined motion-based bilateral networks for identifying the background [33]. Two-stream fully convolution networks are also a popular choice [9,24,53,32] to fuse motion and appearance information together for object inference.…”

Section: Video Object Segmentationmentioning

confidence: 99%

See 2 more Smart Citations

See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks

Yang

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

470

226

View full text Add to dashboard Cite

We introduce a novel network, called CO-attention Siamese Network (COSNet), to address the unsupervised video object segmentation task from a holistic view. We emphasize the importance of inherent correlation among video frames and incorporate a global co-attention mechanism to improve further the state-of-the-art deep learning based solutions that primarily focus on learning discriminative foreground representations over appearance and motion in short-term temporal segments. The co-attention layers in our network provide efficient and competent stages for capturing global correlations and scene context by jointly computing and appending co-attention responses into a joint feature space. We train COSNet with pairs of video frames, which naturally augments training data and allows increased learning capacity. During the segmentation stage, the co-attention model encodes useful information by processing multiple reference frames together, which is leveraged to infer the frequently reappearing and salient foreground objects better. We propose a unified and end-to-end trainable framework where different co-attention variants can be derived for mining the rich context within videos. Our extensive experiments over three large benchmarks manifest that COSNet outperforms the current alternatives by a large margin.

show abstract

Section: Quantitative and Qualitative Resultsmentioning

confidence: 99%

Section: Quantitative and Qualitative Resultsmentioning

confidence: 99%

Section: Video Object Segmentationmentioning

confidence: 99%

See 1 more Smart Citation

See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks

Yang

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

470

226

View full text Add to dashboard Cite

show abstract

“…Recent deep learning based methods learn more powerful video object features from large-scale training data, yielding a zero-shot solution [63] (still no annotation used for any testing frame). Many of these [7,57,21,58,31,55] employ two-stream networks to combine local motion and appearance information, and apply recurrent neural networks to model the dynamics in a frame-by-frame manner. Though these methods greatly promoted the development of this field and gained promising results, they generally suffer from two limitations.…”

Section: Introductionmentioning

confidence: 99%

Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

Yang

Shen

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

278

133

View full text Add to dashboard Cite

This work proposes a novel attentive graph neural network (AGNN) for zero-shot video object segmentation (ZVOS). The suggested AGNN recasts this task as a process of iterative information fusion over video graphs. Specifically, AGNN builds a fully connected graph to efficiently represent frames as nodes, and relations between arbitrary frame pairs as edges. The underlying pair-wise relations are described by a differentiable attention mechanism. Through parametric message passing, AGNN is able to efficiently capture and mine much richer and higher-order relations between video frames, thus enabling a more complete understanding of video content and more accurate foreground estimation. Experimental results on three video segmentation datasets show that AGNN sets a new state-ofthe-art in each case. To further demonstrate the generalizability of our framework, we extend AGNN to an additional task: image object co-segmentation (IOCS). We perform experiments on two famous IOCS datasets and observe again the superiority of our AGNN model. The extensive experiments verify that AGNN is able to learn the underlying semantic/appearance relationships among video frames or related images, and discover the common objects.

show abstract

“…Existing VOS methods can be divided into two settings based on the degrees of human involvement, namely, unsupervised and semi-supervised. The unsupervised VOS methods [49,44,17,32,29] do not require any manual annotation, while the semi-supervised * Corresponding author. methods [47,6,9,18] rely on the annotated mask for objects in the first frame.…”

Section: Introductionmentioning

confidence: 99%

Spatiotemporal CNN for Video Object Segmentation

Xu¹,

Wen²,

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

In this paper, we present a unified, end-to-end trainable spatiotemporal CNN model for VOS, which consists of two branches, i.e., the temporal coherence branch and the spatial segmentation branch. Specifically, the temporal coherence branch pretrained in an adversarial fashion from unlabeled video data, is designed to capture the dynamic appearance and motion cues of video sequences to guide object segmentation. The spatial segmentation branch focuses on segmenting objects accurately based on the learned appearance and motion cues. To obtain accurate segmentation results, we design a coarse-to-fine process to sequentially apply a designed attention module on multi-scale feature maps, and concatenate them to produce the final prediction. In this way, the spatial segmentation branch is enforced to gradually concentrate on object regions. These two branches are jointly fine-tuned on video segmentation sequences in an end-to-end manner. Several experiments are carried out on three challenging datasets (i.e., DAVIS-2016, DAVIS-2017 and Youtube-Object) to show that our method achieves favorable performance against the stateof-the-arts. Code is available at https://github. com/longyin880815/STCNN .

show abstract

Unsupervised Video Object Segmentation with Motion-Based Bilateral Networks

Cited by 140 publications

References 34 publications

See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks

See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks

Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks

Spatiotemporal CNN for Video Object Segmentation

Contact Info

Product

Resources

About