Salient object detection in video using deep non-local neural networks

Shokri, Mohammad; Harati, Ahad; Taba, Kimya

doi:10.1016/j.jvcir.2020.102769

Cited by 29 publications

(13 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, a common strategy has been to extend static saliency models to the video scenario by incorporating motion features (Wang et al 2017;Shokri et al 2020;Sun 2018). For example, (Wang et al 2017) proposes a two-model architecture to exploit spatio-temporal features: the first module performs frame-level saliency prediction; the second module, instead, takes pairs of frames with saliency predicted by the first module, and generates a dynamic saliency map.…”

Section: Related Workmentioning

confidence: 99%

Hierarchical Domain-Adapted Feature Learning for Video Saliency Prediction

et al. 2021

View full text Add to dashboard Cite

In this work, we propose a 3D fully convolutional architecture for video saliency prediction that employs hierarchical supervision on intermediate maps (referred to as conspicuity maps) generated using features extracted at different abstraction levels. We provide the base hierarchical learning mechanism with two techniques for domain adaptation and domain-specific learning. For the former, we encourage the model to unsupervisedly learn hierarchical general features using gradient reversal at multiple scales, to enhance generalization capabilities on datasets for which no annotations are provided during training. As for domain specialization, we employ domain-specific operations (namely, priors, smoothing and batch normalization) by specializing the learned features on individual datasets in order to maximize performance. The results of our experiments show that the proposed model yields state-of-the-art accuracy on supervised saliency prediction. When the base hierarchical model is empowered with domain-specific modules, performance improves, outperforming state-of-the-art models on three out of five metrics on the DHF1K benchmark and reaching the second-best results on the other two. When, instead, we test it in an unsupervised domain adaptation setting, by enabling hierarchical gradient reversal layers, we obtain performance comparable to supervised state-of-the-art. Source code, trained models and example outputs are publicly available at https://github.com/perceivelab/hd2s.

show abstract

Section: Related Workmentioning

confidence: 99%

Hierarchical Domain-Adapted Feature Learning for Video Saliency Prediction

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The core idea of non-local attention is calculating the correlation between the block of pixels to be computed and rest blocks, highlighting concerned areas, and eliminating noise effectively. Non-local attention has a tremendous performance in other vision tasks, such as object detection [41], image restoration [42,43], semantic segmentation [44,45] and person re-identification [46,47]. For MVS reconstruction, the application of non-local attention has not been found yet.…”

Section: Non-local Attentionmentioning

confidence: 99%

PA-MVSNet: Sparse-to-Dense Multi-View Stereo With Pyramid Attention

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In 2020, Mohammad Shokri et al [1], developed a new non-local fully CNN model in order to capture the global dependencies additional competently as well as examine employ of newly developed non-local NN in video SOD. The consequence of non-local operations was examined individually on dynamic as well as static saliency detection to employ both motions as well as appearance features.…”

Section: Literature Reviewmentioning

confidence: 99%

“…The SOD intends to accurately and consistently recognize a large amount of visually distinctive areas in a video or image [1]. It has to turn out to be an extremely vigorous research issue in computer vision, since it is capable to sustain high-level visual tasks, like visual tracking, segmentation, photo cropping, and thumbnail creation.…”

Section: Introductionmentioning

confidence: 99%

Improved Deer Hunting Optimization Algorithm for Video- based Salient Object Detection

Puttaswamy¹

2020

View full text Add to dashboard Cite

Recent days, many researchers have concentrated on the development in salient object detection which is important in numerous computer vision applications. Nevertheless, the main confront is the competent SOD employing the still images. Hence, this work presents the SOD method employing the developed Improved-Deer Hunting Optimization (Improved-DHO) approach. The approach experiences 3 steps that include saliency mapping, keyframe extraction, contourlet mapping, subsequently, a combination of attained outputs by exploiting the optimal coefficients. Initially, extracted frames are given to saliency as well as contourlet mapping concurrently to decide each pixel quality. Subsequently, the outcomes attained from the contourlet mapping and saliency mapping are combined by exploiting the random coefficients in order to attain the last consequence which is used to recognize the salient objects. Moreover, the developed Improved-DHO is used to select the optimal coefficients in order to detect salient objects. The investigational analysis of developed Improved-DHO based on the performance measures exposes that the developed Improved-DHO obtained a maximum accuracy, specificity, and sensitivity.

show abstract

Salient object detection in video using deep non-local neural networks

Cited by 29 publications

References 66 publications

Hierarchical Domain-Adapted Feature Learning for Video Saliency Prediction

Hierarchical Domain-Adapted Feature Learning for Video Saliency Prediction

PA-MVSNet: Sparse-to-Dense Multi-View Stereo With Pyramid Attention

Improved Deer Hunting Optimization Algorithm for Video- based Salient Object Detection

Contact Info

Product

Resources

About