Multi-timescale Trajectory Prediction for Abnormal Human Activity Detection

Abstract

A classical approach to abnormal activity detection is to learn a representation for normal activities from the training data and then use this learned representation to detect abnormal activities while testing. Typically, the methods based on this approach operate at a fixed timescale -either a single timeinstant (e.g. frame-based) or a constant time duration (e.g. videoclip based). But human abnormal activities can take place at different timescales. For example, jumping is a short term anomaly and loitering is a long term anomaly in a surveillance scenario. A single and pre-defined timescale is not enough to capture the wide range of anomalies occurring with different time duration. In this paper, we propose a multi-timescale model to capture the temporal dynamics at different timescales. In particular, the proposed model makes future and past predictions at different timescales for a given input pose trajectory. The model is multi-layered where intermediate layers are responsible to generate predictions corresponding to different timescales. These predictions are combined to detect abnormal activities.In addition, we also introduce an abnormal activity dataset for research use that contains 4,83,566 annotated frames. Our experiments show that the proposed model can capture the anomalies of different time duration and outperforms existing methods.