Hierarchical Video Representation with Trajectory Binary Partition Tree

Abstract

As early stage of video processing, we introduce an iterative trajectory merging algorithm that produces a regionbased and hierarchical representation of the video sequence, called the Trajectory Binary Partition Tree (BPT). From this representation, many analysis and graph cut techniques can be used to extract partitions or objects that are useful in the context of specific applications.In order to define trajectories and to create a precise merging algorithm, color and motion cues have to be used. Both types of informations are very useful to characterize objects but present strong differences of behavior in the spatial and the temporal dimensions. On the one hand, scenes and objects are rich in their spatial color distributions, but these distributions are rather stable over time. Object motion, on the other hand, presents simple structures and low spatial variability but may change from frame to frame. The proposed algorithm takes into account this key difference and relies on different models and associated metrics to deal with color and motion information. We show that the proposed algorithm outperforms existing hierarchical video segmentation algorithms and provides more stable and precise regions.

Temporally Coherent 3D Point Cloud Video Segmentation in Generic Scenes

“…Hierarchical representations are employed in some methods, such as [2], [40], [28], [15], to represent the raw data from coarse to fine. A hierarchical representation usually starts from the segments at a relatively fine level, such as superpixels or over-segmented regions recovered from a contour probability map [2] for RGB data, or super-voxels [30], [6] in the case of RGBD data.…”

“…On the other hand, exploiting temporal coherence also helps to better construct the hierarchical representation at each frame. For instance, in [28], long term trajectories are leveraged to help building BPTs.…”

Learning Must-Link Constraints for Video Segmentation Based on Spectral Clustering

“…The literature on the topic has become prolific [7,43,2,28,27,19,11,10,4,29] and a number of techniques have become available, e.g. generative layered models [25,26], graph-based models [20,46,36] and spectral techniques [39,8,15,18,32,35,16].…”

Robust video segment proposals with painless occlusion handling

“…Among unsupervised multiple segmentation methods that generate an exhaustive list of video segments, agglomerative or spectral clustering on superpixels/supervoxels has been popular [10,15,19,22,24,27,36,40,43]. Some approaches utilize tracked feature points [5,6,14,29,34].…”

“…Video segmentation has been defined by different researchers as separating foreground from background [4,28,37,45], identifying moving objects [12,29,34], creating segmentation proposals [3,28,31,45], computing hierarchical sets of coarse-tofine video segments [19,24,36,42], or generating motion segmentations [24,35]. Each of these definitions has its own merits and applications.…”