Multicamera Joint Video Synopsis

Abstract

Due to an increasing demand of video surveillance, there is an explosive growth of surveillance videos, which causes a big challenge in video storage, browsing and retrieval. The video synopsis technique is thus developed to extract and rearrange moving objects so as to handle the massive video browsing challenge. However, the traditional video synopsis (TVS) method only considers processing videos captured by a single camera, ignoring object interactions in multi-camera videos. To address this issue, we propose a novel multi-camera joint video synopsis (JVS) algorithm for multi-camera surveillance videos. Firstly, a key time stamp (KTS) selection method is designed to find an object's appearing, merging, splitting, and disappearing moments in the frame sequence, called tube, of that object. Secondly, tubes are rearranged by minimizing a global energy function that involves the overall camera views. Compared with the energy function used in TVS, the proposed global energy function considers the chronological orders of tubes not only in the same camera view, but also among different camera views. Moreover, the chronological disorder cost term is formulated based on the KTS labels, and improved by considering the visual similarity between two tubes. Finally, the multi-camera synopsis videos are separately generated by stitching together the globally rearranged tubes and background images of the same camera view. Extensive experiments show that the proposed JVS method is better than the traditional single-camera video synopsis method in preserving chronological orders of moving objects among multi-camera synopsis videos.

A comprehensive survey of multi-view video summarization

“…2. Overall distribution of MVS literature (a) citation wise distribution of MVS research papers P1 [26], P2 [10], P3 [33], P4 [31], P5 [50], P6 [34], P7 [37], P8 [11], P9 [35], P10 [27], P11 [43], P12 [41], P13 [40], P14 [39], P15 [45], P16 [48], P17 [49], (b) year-wise MVS publications to date, (c) publisher-wise distribution of MVS research paper, and (d) distribution of MVS on the basis of research paper's type. The main theme is to reduce compression and transmission power.…”

“…16 [39] 2016 ✔ In this paper, authors presented a novel technique based on joint embedding and sparse coding for summarization of multi-view videos. 17 [40] 2016 ✔ A multi-camera joint video synopsis is presented that finds object's appeariance, merging, splitting, and disappearing moments in the frame sequence called as tube from each view. These tubes are joined by rearranging them so that temporal ordering remains same for all the cameras.…”

DeepQAMVS: Query-Aware Hierarchical Pointer Networks for Multi-Video Summarization

“…Applications range from multi-view summarization aiming at summarizing videos captured for the same scene with several dynamically moving cameras (e.g., in surveillance) [21,38,43,44,82], and summarizing of user-devices' videos [1,41,72,73,[77][78][79] (e.g., for cities hotspot preview [78] or city navigation [77]) to topicrelated MVS (QAMVS) [22,25,28,41,46,66]. Early attempts to solve the QAMVS task applied techniques optimizing for diversity [9, 20, 26, 31-35, 46, 47, 65, 66].…”

Multi‐camera video synopsis of a geographic scene based on optimal virtual viewpoint

“…Another method type, motion region transformation (Nie, Xiao, Sun, & Li, 2013), reconstructs the area of motion to achieve a spatiotemporal uniform distribution representation of video objects. Accordingly, some researchers have focused on the spatiotemporal association of multi‐camera video content and synopsis (Zhu, Liao, & Li, 2016) by adjusting the order of the same video objects shot by different cameras. These studies can effectively reduce the playback time of video synopsis (Mahapatra, Sa, Majhi, & Padhy, 2016); however, they do not consider the spatial relationship between different cameras and their fields of view (Milosavljević, Rančić, Dimitrijević, Predić, & Mihajlović, 2016), nor can they effectively describe the cross‐camera motion of video objects in multi‐camera geographical scenes.…”