Flexibly regularized mixture models and application to image segmentation

Unsupervised Video Segmentation Algorithms Based On Flexibly Regularized Mixture Models

“…We compare with this model to evaluate the effects of time propagation on image segmentation and on temporal consistency. In addition, we also compare to the static model of [1], handling each frame independently and propagating class information between hidden units of deep CNNs. This hierarchical model uses features from the 16 first layers of VGG19 and slightly outperforms comparable unsupervised algorithms [26,27,28], though it does not match state-of-the-art and deep learning based algorithms.…”

“…We compared these algorithms using two databases: the DAVIS database [13], a widely used database for natural video segmentation, and the MPI-Sintel database [29], gathering synthetic videos with complex motion patterns used to evaluate OF estimation and video segmentation algorithms. Figure 4 presents the segmentation maps for three sequences, produced by Temp prop, Temp + OF prop and the hierarchical model of [1]. First, temporal propagation across the iterations of the EM algorithm, used in our two models, seems to increase spatial smoothing.…”

“…The model presented here builds on FlexMM [1], and extends it to capture sequences of images and spatially segment them. Consider an image domain of size N and a number of frames T .…”

“…P (1 ) n B (1 ) n C (1) n P (1 ) m B (1 ) m C (1) m X (1) m X (1) n a (1) P (2 ) n B (2 ) n C (2) n P (2 ) m B (2 ) m C (2) m X (2) m X (2) n a (2) P (3 ) n B (3 ) n C (3) n P (3 ) m B (3 ) m C (3) m X (3) m X (3) n a (3) 0 1 Fig. 3: Graphical model for our Temp-prop model: the prior of the mixing probabilities depends on the class labels of the neighboring pixels and of the previous and next frames.…”

“…Here we propose a family of probabilistic segmentation algorithms that share grouping information across visual features in space and time, and rely on a generative model that captures static and dynamic natural image statistics. Our framework adopts flexibly regularized mixture models (FlexMM) [1], an efficient method to combine mixture distributions across different data sources. FlexMMs of Student-t distributions capture the statistics of wavelet coefficients and hidden units of deep convolutional neural networks (CNNs), and successfully segment static natural images, through uncertainty-based information sharing between hidden layers of CNNs.…”