Markovian Framework for Foreground-Background-Shadow Separation of Real World Video Scenes

Abstract

Abstract. In this paper we give a new model for foreground-background-shadow separation. Our method extracts the faithful silhouettes of foreground objects even if they have partly background like colors and shadows are observable on the image. It does not need any a priori information about the shapes of the objects, it assumes only they are not point-wise. The method exploits temporal statistics to characterize the background and shadow, and spatial statistics for the foreground. A Markov Random Field model is used to enhance the accuracy of the separation. We validated our method on outdoor and indoor video sequences captured by the surveillance system of the university campus, and we also tested it on well-known benchmark videos.

Color Models of Shadow Detection in Video Scenes

“…First, we developed a deterministic method which classifies the pixels independently, since that way, we could perform a relevant quantitative comparison of the different color spaces. After that we gave a probabilistic interpretation to this model and we inserted it into a MRF framework which we developed earlier (Benedek and Szirányi, 2006). We compared the different results after MRF optimization qualitatively and observed similar relative performance of the color spaces to the deterministic model.…”

“…For these reasons, we used an elliptical shadow domain descriptor having parallel axes with the xyz coordinate axes: (5) where {a i , b i | i = 0, 1, 2} are the shadow domain parameters. For these parameters, a similar update procedure can be constructed to that we introduced in (Benedek and Szirányi, 2006). We found in the experiments, that the parameters of the 'chrominance' components are approximately constant in time.…”

“…The definition of the density functions p bg (s) and p fg (s) s ∈ S is the same, as we defined in (Benedek and Szirányi, 2006). We use a mixture of Gaussian model for the pixel values in the background, where the parameters are determined using (Stauffer and Grimson, 2000).…”

“…The flexibility of this MRF model comes from the fact that we defined ψ(s) shadow descriptors for different color spaces differently in Section 3. The method sets the parameters of the class models adaptively, similarly to (Benedek and Szirányi, 2006). We compared the segmentation results using different color spaces in the MRF model (Fig.…”

“…During the development of this framework, we have carefully considered the main approaches in the state-of-the art. Our presented model is the generalization of our previous work (Benedek and Szirányi, 2006).…”