Human segmentation in surveillance video with deep learning

Abstract

Advanced intelligent surveillance systems are able to automatically analyze video of surveillance data without human intervention. These systems allow high accuracy of human activity recognition and then a high-level activity evaluation. To provide such features, an intelligent surveillance system requires a background subtraction scheme for human segmentation that captures a sequence of images containing moving humans from the reference background image. This paper proposes an alternative approach for human segmentation in videos through the use of a deep convolutional neural network. Two specific datasets were created to train our network, using the shapes of 35 different moving actors arranged on background images related to the area where the camera is located, allowing the network to take advantage of the entire site chosen for video surveillance. To assess the proposed approach, we compare our results with an Adobe Photoshop tool called Select Subject, the conditional generative adversarial network Pix2Pix, and the fully-convolutional model for real-time instance segmentation Yolact. The results show that the main benefit of our method is the possibility to automatically recognize and segment people in videos without constraints on camera and people movements in the scene (Video, code and datasets are available at http://graphics.unibas.it/www/HumanSegmentation/index.md.html).

Activation-Based Sampling for Pixel- to Image-Level Aggregation in Weakly-Supervised Segmentation

“…Societal impacts. Image segmentation has several applications, including autonomous driving [1], medical image analysis [3], and video surveillance [2]. Although we have not focused on any specific application in this work, we describe some potential societal impacts in regard to image segmentation in general.…”

“…The great advancements of deep learning methods in recent years have had a large impact on many computer vision tasks, with no exception for semantic segmentation. The ability to automatically segment images has been found useful in many applications, including autonomous driving [1], video surveillance [2], and medical image analysis [3]. Fully-supervised segmentation frameworks have achieved remarkable results by utilizing large datasets of pixel-wise annotated images.…”

Selfie Segmentation in Video Using N-Frames Ensemble

“…They have segmented the person from the background on a video using a global background attenuation model. Monica Gruosso et al [8] showed the possibility of automatic human recognition and segmentation in a surveillance video system. The authors used SegNet [49] encoder-decoder Convolution Neural Network (NCNN) model for the experiment.…”

Invisible-to-Visible: Privacy-Aware Human Instance Segmentation using Airborne Ultrasound via Collaborative Learning Variational Autoencoder

“…Recently, human action recognition has attracted wide attention [10,44] because of its applications, such as automated surveillance [1], intelligent robots [23], and behavior monitoring in homes [3,15,16,46,60]. Several sensors, such as cameras [60], electromagnetic waves [31], and wearable devices [2,8,47,55,66], have been employed to detect human activities.…”