Embedded Indexing in Scalable Video Coding

Abstract

Effective encoding and indexing of audiovisual documents are two key aspects for enhancing the multimedia user experience. In this paper we propose the embedding of low-level content descriptors into a scalable video coding bit-stream by jointly optimizing encoding and indexing performance. This approach provides a new type of bit-stream where part of the information is used for both content encoding and content description, allowing the so called ”Midstream Content Access”. To support this concept, a novel technique based on the appropriate combination of Vector Quantization and Scalable Video Coding has been developed and evaluated. More specifically, the key-pictures of each video GOP are encoded at a first draft level by using an optimal visual-codebook, while the residual errors are encoded using a conventional approach. The same visual-codebook is also used to encode all the key pictures of a video shot, which boundaries are dynamically estimated. In this way, the visual-codebook is freely available as an efficient visual descriptor of the considered video shot. Moreover, since a new visual-codebook is introduced every time a new shot is detected, also an implicit temporal segmentation is provided

Image coding with face descriptors embedding

“…Descriptors have been used in [6] and [7] for obtaining respectively a scalable image encoder and a shot-based video encoder. In both solutions, the description of the image is composed by the visual codebook, obtained by vector quantization of image blocks.…”

JPSearch: New international standard providing interoperable framework for image search and sharing

“…The effort of research on image search and retrieval was greatly increased with the popularity of digital cameras and phone cameras. Some efforts, which may not cover the complete scope of JPSearch Frame-work, related to parts of JPSearch standard can be found in [3] [4] [5] [6] [7] [8].…”