Cross Complementary Fusion Network for Video Salient Object Detection

Abstract

Recently, optical flow guided video saliency detection methods have achieved high performance. However, the computation cost of optical flow is usually expensive, which limits the applications of these methods in time-critical scenarios. In this paper, we propose an end-to-end cross complementary network (CCNet) based on fully convolutional network for video saliency detection. The CCNet consists of two effective components: single-image representation enhancement (SRE) module and spatiotemporal information learning (STIL) module. The SRE module provides robust saliency feature learning for a single image through a pyramid pooling module followed by a lightweight channel attention module. As an effective alternative operation of optical flow to extract spatiotemporal information, the STIL introduces a spatiotemporal information fusion module and a video correlation filter to learn the spatiotemporal information, the inner collaborative and interactive information between consecutive input groups. In addition to enhancing the feature representation of a single image, the combination of SRE and STIL can learn the spatiotemporal information and the correlation between consecutive images well. Extensive experimental results demonstrate the effectiveness of our method in comparison with 14 state-of-the-art approaches. INDEX TERMS Video saliency detection, pyramid pooling, self-attention mechanism, multi-channel concatenation, structural information.

Video saliency-recognition by applying custom spatio temporal fusion technique

“…Cluster saliency is measured using spatial, corresponding and contrast and the results are obtained by fusing the single and multi-image saliency maps. There is another research [28]- [34] where computation of robust geodesic measurement is done to get the saliency mapping. In [35]- [40] has used a super pixel-based strategy and this helps in formulating our proposed custom spatio-temporal fusion saliency detection method.…”