Improved Video Segmentation by Adaptive Combination of Depth Keying and Mixture-of-Gaussians

Abstract

Video segmentation or matting, the separation of foreground objects from background in video sequences, is a demanding task and is needed for a broad range of applications. The most widespread method for video segmentation is chroma-keying using a known background color for which a controlled environment is required. Recently a different method of keying fore-and background has been proposed in which the chroma-keying is replaced by depth-keying using a Time-of-Flight (ToF) camera. The current ToF-cameras suffer from noise and low resolution sensors, which results in unsatisfying segmentation results. We propose to combine the segmentation of dynamic objects in depth with a segmentation in the color domain using adaptive background models. We weight the two measures depending on the actual depth values using either the variance of the depth images of the ToF-camera or the amplitude image of the ToF-camera as reliability measure. We show that both methods significantly improve the segmentation results.

Improving Video Segmentation by Fusing Depth Cues and the ViBe Algorithm

“…Section 4 presents experiments performed without post-processing and the results obtained using three RGB-D algorithms (MOG4D [17], DECB [18] and DEVB) as well as the color-based ViBe algorithm.…”

“…Crabb et al [16] and Schiller et al [17] focused on combining the depth and color information obtained by low-resolution TOF cameras, but their methods are not well suited for video surveillance. For example, the method of Crabb et al [16] requires the definition of a distance plane where no foreground object is located behind any part of the background.…”

“…We compared the results of our method with those obtained using two state-of-the-art RGB-D fusion-based background subtraction algorithms, namely, MOG4D [17] and DECB [18], as well as the ViBe algorithm on the color images [7] (ViBe) and the ViBe algorithm on the depth images (ViBe1D). To evaluate these algorithms objectively through a quantitative analysis, we required a benchmark that would provide information on both color and depth images.…”

3D people surveillance on range data sequences of a rotating Lidar

“…Range image sequences offer significant advantages versus conventional video flows for scene analysis, since geometrical information is directly available (Schiller and Koch, 2011), which can provide more reliable features than intensity, color or texture values (Wang et al, 2006;Benedek and Szirányi, 2008). Using Time-of-Light (ToF) cameras (Schiller and Koch, 2011) or scanning Lidar sensors (Kaestner et al, 2010) enable recording range images independently of the illumination conditions and we can also avoid artifacts of stereo vision techniques.…”

“…Using Time-of-Light (ToF) cameras (Schiller and Koch, 2011) or scanning Lidar sensors (Kaestner et al, 2010) enable recording range images independently of the illumination conditions and we can also avoid artifacts of stereo vision techniques. From the point of view of data analysis, ToF cameras record depth image sequences over a regular 2D pixel lattice, where established image processing approaches, such as Markov Random Fields (MRFs) can be adopted for smooth and observation consistent segmentation and recognition (Benedek and Szirányi, 2008).…”