Recognizing Humans in Motion: Trajectory-based Aerial Video Analysis

Abstract

We propose a novel method for recognizing people in aerial surveillance videos. Aerial surveillance images cover a wide area at low resolution. In order to detect objects (e.g., pedestrians) from such videos, conventional methods either utilize appearance information from raw videos or extract blob information from background subtraction results. However, people seen in low resolution images have less appearance information, and hence are very difficulty to classify based on their appearance or blob size. In addition, due to heavy camera movements caused by aerial vehicle ego-motion and wind, the system is expected to generate many noisy false detections including parallax. The idea presented in this paper is to detect and classify objects from aerial videos based on their motion: we analyze a trajectory of each object candidate, deciding whether it is a person-of-interest or simple noise based on how it moved. After objects are tracked by a Kalman filter-based tracking, we represent their motion as multi-scale histograms of 'orientation changes', which efficiently captures movements displayed by objects. Random forest classifiers are applied to our new representation to make the decision. The experimental results illustrate that our approach recognizes objects-of-interest (i.e., humans) even when there exist a large number of false detection/tracking, and it does it more reliably compared to the approaches with previous paradigm.

Real-Time Detection, Tracking and Classification of Multiple Moving Objects in UAV Videos

“…Quantitative analyses of our image registration, moving object detection and tracking performance have yielded greater average stationary state registration PSNR values and detection F1 scores than those from [18]. The overall classification accuracy was greater than in [2] and [3]. Considering major application differences, such as our use of feature based methods aided with different descriptors for image registration, our real-time approach using a sampled video with lower fps, use of different datasets and learning techniques, and possible differences in our camera setting such as UAV related hardware, it is difficult to evaluate the significance of these results.…”

“…In their study Máttyus et al [1] accomplished object detection and tracking for vehicles and humans in realtime from low altitude aerial surveillance. In addition to the detection and tracking, Iwashita et al [2] also performed classification with 80% accuracy for vehicles and humans but the operation was not in real-time. Another similar study is conducted by Oreifej et al [3] where they performed detection and classification for humans but not tracking.…”

Learning to Discriminate Information for Online Action Detection: Analysis and Application

“…In contrast, online action detection aims to identify ongoing actions from streaming videos at every moment in time. This task is useful for many real-world applications, such as autonomous driving [6], robot assistants [7], and surveillance systems [8], [9]. Also, online action detection in these applications can be developed to more challenging tasks (e.g., action anticipation).…”

SRG: Snippet Relatedness-Based Temporal Action Proposal Generator

“…process includes localizing temporal boundaries and classifying the class of each instance. Temporal action detection can be used in many applications such as smart surveillance [1], [2], video summarization [3], [4], [5], and video retrieval [6], [7], [8]. Similar to object proposals for object detection [9], [10], [11], temporal action proposals play an important role in temporal action detection [12], [13], [14].…”