Compact Video Synopsis via Global Spatiotemporal Optimization

Abstract

Video synopsis aims at providing condensed representations of video data sets that can be easily captured from digital cameras nowadays, especially for daily surveillance videos. Previous work in video synopsis usually moves active objects along the time axis, which inevitably causes collisions among the moving objects if compressed much. In this paper, we propose a novel approach for compact video synopsis using a unified spatiotemporal optimization. Our approach globally shifts moving objects in both spatial and temporal domains, which shifting objects temporally to reduce the length of the video and shifting colliding objects spatially to avoid visible collision artifacts. Furthermore, using a multilevel patch relocation (MPR) method, the moving space of the original video is expanded into a compact background based on environmental content to fit with the shifted objects. The shifted objects are finally composited with the expanded moving space to obtain the high-quality video synopsis, which is more condensed while remaining free of collision artifacts. Our experimental results have shown that the compact video synopsis we produced can be browsed quickly, preserves relative spatiotemporal relationships, and avoids motion collisions.

Surveillance Video Synopsis via Scaling Down Objects

“…Lu et al [40] edited timelines of moving objects to accelerate or slow objects, which can reduce the overlapped areas of moving objects. In spatiotemporal analysis approaches [8], [41], moving objects are shifted not only along the time axis, but also in the spatial domain. Moreover, to avoid producing unpleasant artifacts such as a car running on the water, an approach was proposed in [8] to expand the background to fit the shifted objects.…”

“…These approaches tried to preserve the most useful information and skip frames of low interest. However, framebased approaches tend to lose the dynamic aspect of the original video and retain large empty spaces of the background in the synopsis [1], [8]. To reduce most empty spaces of the original video, a number of different approaches have been proposed to extract informative video portions from the video volume and montage these portions together [9]- [12].…”

“…To reduce collisions and further improve the compression ratio, some spatiotemporal analysis approaches were proposed. Nie et al [8] and Kang et al [10] indicated that there are still some empty spaces in the synopsis video where no moving objects spread over, and then shifted the moving objects in the spatiotemporal domain by exploiting both spatial and temporal information. Although these approaches are capable to avoid most collisions, both temporal and location information of objects are violated.…”

Surveillance Video Synopsis in GIS

“…This section describes the main process of surveillance video synopsis in GIS and provides details for the technical implementation of each step (as shown in Figure 1), including georeferencing and clustering of moving objects, virtual field construction, moving object display trajectory generation, and moving object expression. expansion method [37]. Under many circumstances, ontology optimizations of moving objects and display sequence simultaneously exist.…”

“…Typical methods for spatial-temporal positioning optimization include the sub-channel offset method [36] and the moving area expansion method [37]. Under many circumstances, ontology optimizations of moving objects and display sequence simultaneously exist.…”