SCOM: Spatiotemporal Constrained Optimization for Salient Object Detection

Abstract

This paper presents a novel model for video salient object detection called spatiotemporal constrained optimization model (SCOM), which exploits spatial and temporal cues, as well as a local constraint, to achieve a global saliency optimization. For a robust motion estimation of salient objects, we propose a novel approach to modeling the motion cues from optical flow field, the saliency map of the prior video frame and the motion history of change detection, which is able to distinguish the moving salient objects from diverse changing background regions. Furthermore, an effective objectness measure is proposed with intuitive geometrical interpretation to extract some reliable object and background regions, which provided as the basis to define the foreground potential, background potential, and the constraint to support saliency propagation. These potentials and the constraint are formulated into the proposed SCOM framework to generate an optimal saliency map for each frame in a video. The proposed model is extensively evaluated on the widely used challenging benchmark data sets. Experiments demonstrate that our proposed SCOM substantially outperforms the state-of-the-art saliency models.

Evaluating salient object detection in natural images with multiple objects having multi‐level saliency

“…Note that the works proposing the SOD and PASCAL-S datasets do consider unequal importance of objects in their dataset creation, but do not use it for ground truth generation and evaluation. Assuming equal importance of salient objects, the use saliency for object detection has progressed to object-level abstraction [10] and video salient object detection [11]. Firstly, in this paper, with the help of subjective experiments we corroborate the findings of [6,7] that objects in natural images are seen and perceived to have varying levels of importance.…”

Overview of deep-learning based methods for salient object detection in videos

“…According to the common concepts used in deep learning methods, firstly, the global framework for each method is described in 2.1, then the deep network in each method is analyzed in 2.2, and finally an overview of the categorization of methods is shown at a functional level in 2.3. As a matter of convenience, the describled methods, are denoted as SCOMd [12], NRF [13], DHSNet [14], OSVOS [15], NLDF [16], LMP [17], SFCN [18], SegFlow [19], LVO [20], WSS [21], SCNN [22], DSS [23], SPD [24], AFNet [25] and CPD [26].…”

“…Moving objects attract large attention and thus can be regarded as salient objects in videos. As in the methods[12,22,18], we also use the 30 test videos with the provided ground truth. The DAVIS 2016 dataset[27] is a popular video dataset for video foreground segmentation.It is divided into two splits: the training (30 sequences) part used for training only and the validation (20 sequences) part for the inference.…”

Research on Salient Object Detection using Deep Learning and Segmentation Methods

“…Zhu et al, [25] presented a latent hierarchical structural learning method for object detection. The nodes can move spatially to allow both local and global shape deformations.…”