Object Segmentation and Ontologies for MPEG-2 Video Indexing and Retrieval

Abstract

Abstract.A novel approach to object-based video indexing and retrieval is presented, employing an object segmentation algorithm for the real-time, unsupervised segmentation of compressed image sequences and simple ontologies for retrieval. The segmentation algorithm uses motion information directly extracted from the MPEG-2 compressed stream to create meaningful foreground spatiotemporal objects, while background segmentation is additionally performed using color information. For the resulting objects, MPEG-7 compliant low-level indexing descriptors are extracted and are automatically mapped to appropriate intermediatelevel descriptors forming a simple vocabulary termed object ontology. This, combined with a relevance feedback mechanism, allows the qualitative definition of the high-level concepts the user queries for (semantic objects, each represented by a keyword) and the retrieval of relevant video segments. Experimental results demonstrate the effectiveness of the proposed approach.

Implicit Motion-Compensated Network for Unsupervised Video Object Segmentation

“…To focus more on the primary object(s) in I t , we delve into the intra-frame features supported by neighboring frames. In detail, we build a key encoder module to extract independently a key feature for each frame and is symmetric in the center and neighboring frames 1 . The rationales are that 1) Global dependence is computationally efficient to extract from key features than the value (V 5 i ), and 2) Global dependence exists in key features without redundancy, and there is a lot of distraction on value features.…”

“…1) Frame number: Our IMCNet simultaneously takes 2N + 1 frames as inputs and generates the segmentation mask of the center frame (i = t). It is of interest to assess the influence of the number of input frames 2N + 1 (N ∈ [1,3]) on the final performance. Table II shows the results for this.…”

“…T HE goal of unsupervised video object segmentation (UVOS) is to identify and segment the most visually prominent object(s) from the background in videos. The UVOS has attracted significant attention owing to its widespread applications in content-based video retrieval [1], video matting [2], video editing [3], video analysis [4], and video compression [5]- [10]. Different from the semisupervised [11]- [14] and weakly-supervised [15], [16] video object segmentation, which has a full mask of the object(s) of interest in the first frame of a video sequence, the UVOS is more challenging because of the lack of user interactions.…”