Learning Rare Behaviours

Abstract

Abstract. We present a novel approach to detect and classify rare behaviours which are visually subtle and occur sparsely in the presence of overwhelming typical behaviours. We treat this as a weakly supervised classification problem and propose a novel topic model: Multi-Class Delta Latent Dirichlet Allocation which learns to model rare behaviours from a few weakly labelled videos as well as typical behaviours from uninteresting videos by collaboratively sharing features among all classes of footage. The learned model is able to accurately classify unseen data. We further explore a novel method for detecting unknown rare behaviours in unseen data by synthesising new plausible topics to hypothesise any potential behavioural conflicts. Extensive validation using both simulated and real-world CCTV video data demonstrates the superior performance of the proposed framework compared to conventional unsupervised detection and supervised classification approaches.

Topic-based Video Analysis

“…Anomaly detection in an automated surveillance system [132] uses a multi-class approach known as multi-class delta LDA that generates new unseen topics regarded as abnormal behavior. Multi-class LDA is also useful to find rare activities [91]. In one of the recent works [42], researchers have described an LDA model for streaming video dataset and then used it to detect anomalous events by an underwater robot.…”

Low‐rank structured sparse representation and reduced dictionary learning‐based abnormity detection

“…To overcome the difficulties mentioned above, low‐level features such as pixel‐level motion and appearance have been widely applied [1, 9, 13, 15, 19, 20, 23, 26, 29, 33, 35–37, 43–47, 52]. Optical flow is one of the most popular features for motion pattern, which can be used to describe the position and motion direction of each local patch and generate bag‐of‐words representation [15, 33, 35–37, 46].…”

“…To overcome the difficulties mentioned above, low‐level features such as pixel‐level motion and appearance have been widely applied [1, 9, 13, 15, 19, 20, 23, 26, 29, 33, 35–37, 43–47, 52]. Optical flow is one of the most popular features for motion pattern, which can be used to describe the position and motion direction of each local patch and generate bag‐of‐words representation [15, 33, 35–37, 46]. Histograms of local optical flow [13], a histogram of gradient and optical flow [20], and a multi‐scale histogram of optical flow [23, 47] can encode the flow into a histogram and generate rotation and translation invariant representation.…”

“…For DBNs based methods such as hidden Markov model (HMM) [2, 44, 45] and Markov random field (MRF) [14, 19, 43], the cost of multiple objects modelling increases exponentially according to the number of objects, and rendering them is intractable for the busy scenes [35]. PTMs based methods such as probabilistic latent semantic analysis (pLSA) [15, 29] and latent Dirichlet allocation [33, 35], mainly focus on the co‐occurrence visual words in space but ignore the temporal information; therefore, the anomalies cannot be located in space and time. Recently, sparse coding‐based models, which use learnt dictionaries to represent the data, have attracted more attentions but the video data's structure redundancy and low‐rank property have not been fully exploited.…”

“…Detecting the abnormal events from a video sequence is a very active research field in computer vision. It has numerous applications in video surveillance [1][2][3][4][5][6][7][8][9], social security [10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27], traffic monitoring [28][29][30][31][32][33][34][35][36][37] and many others [38][39][40][41][42]. Owing to the huge amount of video data, identifying the abnormities by human labour is almost intractable.…”