Extracting Spatio-Temporal Texture signatures for crowd abnormality detection

Abstract

Abstract-In order to achieve automatic prediction and warning of hazardous crowd behaviors, a Spatio-Temporal Volume (STV) analysis method is proposed in this research to detect crowd abnormality recorded in CCTV streams. The method starts from building STV models using video data. STV slices -called Spatio-Temporal Textures (STT) -can then be analyzed to detect crowded regions. After calculating the Gray Level Co-occurrence Matrix (GLCM) among those regions, abnormal crowd behavior can be identified, including panic behaviors and other behavioral patterns. In this research, the proposed STT signatures have been defined and experimented on benchmarking video databases. The proposed algorithm has shown a promising accuracy and efficiency for detecting crowd-based abnormal behaviors. It has been proved that the STT signatures are suitable descriptors for detecting certain crowd events, which provide an encouraging direction for real-time surveillance and video retrieval applications. Keywords-Crowd abnormality, Spatio-Temporal Volume, STT Signature I. BACKGROUNDThe data obtained from CCTV cameras are 2D image sequences called frames. Each frame of these video data contains spatial information of various visual patterns. However, a single frame doesn't contain useful temporal information such as motions and trajectories of those patterns. In order to explore the temporal features, two or more consecutive frames need to be analyzed together. Horn and Schunck [1] first introduced Optical Flow Field to describe instant motion between two consecutive frames, fellow researchers [2] have conducted experiments using optical flow features to realize crowd abnormality detection. However, the computational complexity of optical flow is high, which limits its real-time performance. Other approaches such as motion field have been used to extract spatio-temporal information, yet suffering from object tracking problems caused by occlusion [3]. In this research, low-level and 2D texture features are utilized to avoid excessive computation because they only require pixel-wise calculation. This approach has ensured the computational time is independent from the actual crowds' density and event types. Following sections will explain how to reconstruct and utilize texture features for abnormal crowd behavior detection in details.

Detecting Anomalies from Video-Sequences: a Novel Descriptor

“…The raw features are extracted from the sensors, such as one or more videocameras. Raw features, such as the RGB level of each pixels, can be used, but the extraction of specific features is often done, such as for the optical flow-based features [18] from the frames sequence or the textural-based features [19] from the single frame.…”

A Graphical Simulator for Modeling Complex Crowd Behaviors

“…In previous research, the Spatial-Temporal Texture (STT) is extracted from the video [30], and Gray Level Cooccurrence Matrix (GLCM) is obtained from the STT. Then a signature is modeled from the GLCM patterns to decide if current texture contains anomalies [31]. Thus the signature value of simulated crowd in this research is compared to some real-life crowd videos.…”

“…In the experiment, patches of empty background, pedestrian with same direction and pedestrian with opposite direction are selected. Then the GLCM patterns including Contrast, Dissimilarity, Homogeneity, Similarity, Angular Second Moment, Energy, Entropy, Variance and Correlation are calculated using the approach in [31]. The result is shown as Table 1.…”

An effective video processing pipeline for crowd pattern analysis

“…Future work will target at further improving the detection accuracy through adaptive thresholding. Feature signatures of the ribbon-like motion textures will be refined based on the earlier work [20] to reveal the true natures and severity of the crowd events.…”