Activity recognition in wide aerial video surveillance using entity relationship models

Abstract

We present the design and implementation of an activity recognition system in wide area aerial video surveillance using Entity Relationship Models (ERM). In this approach, finding an activity is equivalent to sending a query to a Relational DataBase Management System (RDBMS). By incorporating reference imagery and Geographic Information System (GIS) data, tracked objects can be associated with physical meanings, and several high levels of reasoning, such as traffic patterns or abnormal activity detection, can be performed. We demonstrate that different types of activities, with hierarchical structure, multiple actors, and context information, are effectively and efficiently defined and inferred using the ERM framework. We also show how visual tracks can be better interpreted as activities by using geo information. Experimental results on both real visual tracks and GPS traces validate our approach.

Big Data Scalability Issues in WAAS

“…Our activity inference method is simple but very efficient [5]. Activities are defined as vehicular tracks associated with certain properties (e.g., U-turn, 3-point turn, loop, convoy, following, speeding).…”

“…However, because our tracker connects the identities, we can group long-range tracks, distribute into multiple processors, and infer the global activity in independent clusters. Many of multiple actor activities, which need more than two relationships in the inference, can be efficiently decomposed into a series of 2-actor activities [5]. For instance, Nvehicle convoy can be decomposed to a series of 2-vehicle convoy and the computation scales linearly with the number of vehicles.…”

“…Finally, we insert the estimated trajectories into a database and propose to use a scalable ERM framework for activity representation and inference. The ERM-based activity inference framework [5] enables us to utilize highly optimized scalable RDBMSs.…”

Aerial Implicit 3D Video Stabilization Using Epipolar Geometry Constraint

“…It provides important inputs for surveillance applications such as vehicles detection and tracking [18,21]. The trajectories of moving objects inferred from the tracking process, in turn, are used to recognize activities [4]. The output of stabilization process helps improve the optical flow calculation, which is an essential task in most 3D reconstruction applications [15].…”

Learning symbolic descriptions of activities from examples in WAAS

“…In the previous research, we have developed EAR (Entity relationship models-based Activity Recognition) framework [Choi et al 2012]. We extract a set of atomic portions of a track ('tracklets') from video input, along with physical attributes, and store them into a standard RDBMS (Relational Database Management System).…”