Dynamic tree-structured sparse RPCA via column subset selection for background modeling and foreground detection

Abstract

Video analysis often begins with background subtraction, which consists of creation of a background model that allows distinguishing foreground pixels. Recent evaluation of background subtraction techniques demonstrated that there are still considerable challenges facing these methods. Processing per-pixel basis from the background is not only time-consuming but also can dramatically affect foreground region detection, if region cohesion and contiguity is not considered in the model. We present a new method in which we regard the image sequence to be made up of the sum of a low-rank background matrix and a dynamic tree-structured sparse matrix, and solve the decomposition using our approximated Robust Principal Component Analysis method extended to handle camera motion. Furthermore, to reduce the curse of dimensionality and scale, we introduce a low-rank background modeling via Column Subset Selection that reduces the order of complexity, decreases computation time, and eliminates the huge storage need for large videos.

On the Applications of Robust PCA in Image and Video Processing

“…However, RPCA methods immediately provided a very promising solution towards moving object detection. However, because of the wellknown challenges such as dynamic backgrounds, illumination conditions, color saturation, shadows, etc., the state-of-the-art RPCA methods do not often provide accurate segmentation [69], [72], [73], [74], [112], [113], [114], [115], [264], [297].…”

“…Other loss functions can be used such as σ -loss function [305], Least Squares (LS) loss function (|| • || 2 F ), Huber loss function [7], M -estimator based loss functions [121], and the generalized fused Lasso loss function [294], [295]. norm 2 is usually taken to force spatial homogeneous fitting in the matrix S, that is for example the norm 2,1 with p 2 = 1 [69], [72], [73], [74], [112], [113], [115], [114], [264]. It is important to note that the first part of (26) concerns mainly the decomposition into low-rank plus sparse and noise matrices and second part concerns mainly the application of backgroundforeground separation.…”

“…and Ω(S) are the gradient [113], [114], [115], [287], the total variation [41], [111], [113], [114], [288] and the static/dynamic tree structured sparsity norm [70], [71], [74], [180], [267] applied on the matrix S to enforce the spatial and/or temporal constraints, respectively.…”

“…• A weighting matrix W [256], [301], [308] or a transformation τ [68], [69], [70], [71], [72], [73], [74], [119], [120], [219] can also be used as a constraint in (26) to enforce the recovery of the background that appears at only a few frames and to eliminate the influence of light conditions, camouflages, and dynamic backgrounds, and to model potential global motion that the foreground region undergoes, respectively.…”

Foreground Segmentation with Tree-Structured Sparse RPCA

“…Based on the presented methodology, we perform the background modeling using the output of CSSP algorithm, where a lot of redundant information is discarded, as it does not contribute to the background model, if even worse, does not contaminate it [30]. 6] results: top row is the original image, second row is the ground truth, the third row is DBSS results, and the last row is DSPSS output.…”