Temporal mapping of surveillance video for indexing and summarization

Abstract

This work converts the surveillance video to a temporal domain image called temporal profile that is scrollable and scalable for quick searching of long surveillance video by human operators. Such a profile is sampled with linear pixel lines located at critical locations in the video frames. It has precise time stamp on the target passing events through those locations in the field of view, shows target shapes for identification, and facilitates the target search in long videos. In this paper, we first study the projection and shape properties of dynamic scenes in the temporal profile so as to set sampling lines. Then, we design methods to capture target motion and preserve target shapes for target recognition in the temporal profile. It also provides the uniformed resolution of large crowds passing through so that it is powerful in target counting and flow measuring. We also align multiple sampling lines to visualize the spatial information missed in a single line temporal profile. Finally, we achieve real time adaptive background removal and robust target extraction to ensure long-term surveillance. Compared to the original video or the shortened video, this temporal profile reduced data by one dimension while keeping the majority of information for further video investigation. As an intermediate indexing image, the profile image can be transmitted via network much faster than video for online video searching task by multiple operators. Because the temporal profile can abstract passing targets with efficient computation, an even more compact digest of the surveillance video can be created.

Person Recognition System Based on a Combination of Body Images from Visible Light and Thermal Cameras

“…In the conventional setup, a surveillance system uses one or more cameras to collect sequences of images of an observation scene to automatically monitor the people and/or their actions that appear in the scene. Because of this characteristic, the surveillance system has been widely used in security systems to monitor private houses, workplaces, and public areas, or in business to collect customer information [1,2,3,4,5,6]. In a surveillance system, various image processing algorithms can be implemented to extract information from the observation scene such as the detection of incoming persons, and recognition of their age, gender, and actions.…”

Video Indexing Using Face Appearance and Shot Transition Detection

“…Several approaches have also been proposed in order to achieve efficient video indexing and retrieval like keyframes texture, edge and motion features [29], temporal mapping [2], or temporal patterns combined with sequence matching techniques [21]. Faces have been automatically annotated by clustering methods with a weighted feature fusion in [7], dealing with colour information, but a training set is needed in order to perform a general-learning (GL) training scheme.…”

A combined multiple action recognition and summarization for surveillance video sequences

“…From a video containing changing scenes, static-based methods consist of selecting keyframes and dynamic-based methods consist of selecting short video clips. Since the scenes can change, the cameras can move, the summarization, in this case, is carried out by determining the video sequences (shots) that represent the same scenes [8][9][10][11][12][13][14][15]. This allows the keyframe to be selected using extracted features and appropriate clustering methods.…”