Attribute-based vehicle search in crowded surveillance videos

Abstract

We present a novel application for searching for vehicles in surveillance videos based on semantic attributes. At the interface, the user specifies a set of vehicle characteristics (such as color, direction of travel, speed, length, height, etc.) and the system automatically retrieves video events that match the provided description. A key differentiating aspect of our system is the ability to handle challenging urban conditions such as high volumes of activity and environmental factors. This is achieved through a novel multi-view vehicle detection approach which relies on what we call motionlet classifiers, i.e. classifiers that are learned with vehicle samples clustered in the motion configuration space. We employ massively parallel feature selection to learn compact and accurate motionlet detectors. Moreover, in order to deal with different vehicle types (buses, trucks, SUVs, cars), we learn the motionlet detectors in a shape-free appearance space, where all training samples are resized to the same aspect ratio, and then during test time the aspect ratio of the sliding window is changed to allow the detection of different vehicle types. Once a vehicle is detected and tracked over the video, fine-grained attributes are extracted and ingested into a database to allow future search queries such as "Show me all blue trucks larger than 7ft length traveling at high speed northbound last Saturday, from 2pm to 5pm". See video demos of our system at:

Learning Detectors from Large Datasets for Object Retrieval in Video Surveillance

“…The work of Feris et al [1], [5] also splits the training data into motionlet clusters to better deal with non-linearities in the dataset. Training in each cluster is done with largescale feature selection, but only few thousands of training examples are considered.…”

“…The first step of our learning algorithm consists of automatically partitioning the dataset into motionlet clusters [1], i.e., clusters of vehicle images that share similar 2D motion direction. The motion information of a vehicle is directly related to its pose, therefore this operation provides a semantic partitioning of the dataset.…”

“…In this context, we are particularly interested in enabling automatic object retrieval based on attributes from surveillance videos, with focus on vehicles. We have built a search infrastructure similar to the work of Feris et al [1], involving video processing, database ingestion, and web service interface to support user queries such as "Show me all blue vehicles traveling at high speed northbound, last Saturday, from 2pm to 4pm". This paper addresses an important component of this framework: how to develop a robust and efficient approach for vehicle detection in surveillance videos, under varying pose and lighting changes.…”

“…Figure 1 shows the architecture of our proposed system. We use motionlets [1] to automatically split our training dataset into semantic partitions related to vehicle pose. For each partition, we create a set of compact, complementary detectors, each trained in a deep cascade structure, using hundreds of thousands of selected negative examples.…”

Boosting object detection performance in crowded surveillance videos

“…Specifically for applications that require real-time processing, cascade detectors based on Haar-like features have been widely used for detection of faces [22], pedestrians [23] and vehicles [6]. Although significant progress has been made in this area, state-of-the-art object detectors are still not able to generalize well to different camera angles and lighting conditions.…”