Video retrieval with IRIS

Abstract

The large amount of available multimedia information (e.g. videos, audio, images) requires efficient and effective annotation and retrieval methods.The System IRIS (Image Retrieval for Information Systems), which was developed at the Univers~ty of Bremen m the Al group. is designed for content-based retrieval of single images.As videos become a more important role in the frame of multimedia, we want to make videos available for IRIS [1].The basic idea to include wdeos mto the IRIS-System is to separate the whole video into shots -sequences with a sirntlar content and to create a single image for each detected shot. This image will contain the full information of the shot and can be processed with IRIS.The first part of the video analysis. This IS done by a color-histogram based method [4], where a Large difference in the histograms of two successive images indicates the beginning of a new shot.For every frame in the shot the dominant camera motion is estimated by deterring the optical flow for each pair of mccessive images m the sequence. This leads to the coordinate transformations from one image to the next one m the sequence. By appiying the appropriate transformations via a warping operation and merging the overlapping regions of the warped images, a single panoratmc mosaic image covering the entire visible area of the scene can be constructed [2] [3].The second part of the video analysis. After this process the videos are reduced to still images. A textual description can be created using IRIS.The basic concept of IRIS is, that it is more sophisticated and usual for human beings to use natural language concepts, e.g. sky, than syntactical features, e.g. b[ue region righ~up. Thts leads to a content-based image retrieval.Furthermore, it is unreasonable for any human being to make the content description for thousands of images manually. IRIS combines methods and techniques from computer vision and ArtificialIntelligence (AI) in a new way to generate content descriptions of images in a textual form automatically. The text retrieval can be done by an ordinary text-retrieval system.The two dominating goals of the IRIS-system are:q The images should be processed automatically. q The system shotdd offer a comfortable user interface with a comfortable query vocabtdary to formulate more complex queries by using concepts.To realize these goals, the IIUS-system is divided into two modules.The first main-module M divided in four submodules for image analysis, And the second main-module deals with the retrieval.There arc three modules for the low level feature extraction. The extracted features are: color, Texture, and Contour,The solor-segmentation makes use of color histograms in the HLS-color space. Second order statistical features trigger a classifying neural network to determine the texturesegmentation of the image. With a gradient-based edge detection and shape analysls, the contour-segmentation of the image is performed.Each of these sub-modules extracts segments concerning one of the features mentioned above. Th...

VORTEX: Video retrieval and tracking from compressed multimedia databases-visual search engine

“…For P and B frames, an approximation to extracting the exact DC values is used since extracting exact DC values is costly. For the IRIS system developed by Alshuth et al [8], the basic idea is to separate the video sequence into shots and to create a single representative image for each shot. This is the input to IRIS which creates a textual description automatically.…”