Super-pixel based crowd flow segmentation in H.264 compressed videos

Abstract

In this paper, we have proposed a simple yet robust novel approach for segmentation of high density crowd flows based on super-pixels in H.264 compressed videos. The collective representation of the motion vectors of the compressed video sequence is transformed to color map and super-pixel segmentation is performed at various scales for clustering the coherent motion vectors. The number of dynamically meaningful flow segments is determined by measuring the confidence score of the accumulated multi-scale super-pixel boundaries. The final crowd flow segmentation is obtained from the edges that are consistent across all the super-pixel resolutions. Hence, our major contribution involves obtaining the flow segmentation by clustering the motion vectors and determination of number of flow segments using only motion super-pixels without any prior assumption of the number of flow segments. The proposed approach was benchmarked on standard crowd flow dataset. Experiments demonstrated better accuracy and speedup for the proposed approach compared to the state-of-the-art methods.

Crowd flow segmentation in compressed domain using CRF

“…The videos of this dataset have dense flows in both traffic and crowd scenarios. Since these videos are not originally present in H.264 format, we have followed the same procedure as Biswas et al [12] for encoding. Specifically, the video is encoded into H.264 baseline with only I & P frames.…”

“…One reference frame is considered with the Group of Pictures length set to 30. As mentioned in [12], this baseline profile is ideal for extracting motion vectors onthe-fly with low latency. The motion vectors extracted from the encoded video can come from varying macro-block sizes (from 4x4 to 16x16).…”

A survey on compressed domain video analysis techniques

“…The major contribution of this paper involves obtaining the flow segmentation by clustering the motion vectors and determination of number of flow segments using only motion super-pixels without any prior Fig. 11 Crowd flow segmentation by Biswas et al [12] assumption of the number of flow segments. The segmentation result for a specific video is shown in Fig.…”

“…Biswas et al [12] use the collective representation of the motion vectors of the compressed video sequence, transform it to a color map and perform super-pixel segmentation at various scales for clustering the coherent motion vectors. The major contribution of this paper involves obtaining the flow segmentation by clustering the motion vectors and determination of number of flow segments using only motion super-pixels without any prior Fig.…”

“…The key to this classification is the uniqueness of human skin-tone colors. In the MB mask images, a 12 Face region detection results [83] pixel with value one corresponds to a MB whose average color is a skin-tone color. Second, the face regions in MB mask images were detected using binary template matching.…”