Background Subtraction in Real Applications: Challenges, Current Models and Future Directions

Abstract

Computer vision applications based on videos often require the detection of moving objects in their first step. Background subtraction is then applied in order to separate the background and the foreground. In literature, background subtraction is surely among the most investigated field in computer vision providing a big amount of publications. Most of them concern the application of mathematical and machine learning models to be more robust to the challenges met in videos. However, the ultimate goal is that the background subtraction methods developed in research could be employed in real applications like traffic surveillance. But looking at the literature, we can remark that there is often a gap between the current methods used in real applications and the current methods in fundamental research. In addition, the videos evaluated in large-scale datasets are not exhaustive in the way that they only covered a part of the complete spectrum of the challenges met in real applications. In this context, we attempt to provide the most exhaustive survey as possible on real applications that used background subtraction in order to identify the real challenges met in practice, the current used background models and to provide future directions. Thus, challenges are investigated in terms of camera, foreground objects and environments. In addition, we identify the background models that are effectively used in these applications in order to find potential usable recent background models in terms of robustness, time and memory requirements.

G-LBM:Generative Low-dimensional Background Model Estimation from Video Sequences

“…This is a binary classification task, which can be achieved successfully using a DNN. Different methods for the background subtraction have been developed, and we refer the reader to look at [4,3] for comprehensive details on these methods. While we urge the background subtraction process to be unsupervised given the background model, the well performing methods are mostly supervised.…”

GraphBGS: Background Subtraction via Recovery of Graph Signals

“…Background subtraction is an important topic in computer vision and video analysis. There are a large number of applications such as automatic surveillance of human activities in public spaces, intelligent transportation, video analysis, industrial vision, among others [47,7]. Background subtraction aims to separate the moving objects, or foreground, from the static scene called background [5,9].…”

Global Optimality Guarantees for Nonconvex Unsupervised Video Segmentation

“…For instance, semantic segmentation tackles the problem of assigning each extracted object to a certain cluster or predefined class, and supervised (or semi-supervised) methods are endowed with one or more ground truth extractions or annotations [1]. This wide range of methodologies makes video segmentation suitable for many applications, such as surveillance systems, traffic monitoring, and gesture recognition, and therefore video object segmentation remains an active and challenging area of research [2], [3].…”

“…Example 1. Suppose that a video has frames given by X (1) = [ 256 1 256 1 ] and X (2) = [ 1 256 256 256 ], where elements of 256 represent background. Then, the data matrix is Now, a remarkable property of the nonconvex and nonsmooth problem (2) is that, under certain conditions on the problem data, the optimization landscape is benign, i.e., there are no spurious local minima, and the global minimum is unique [9].…”