Image understanding and the web: a state-of-the-art review

Abstract

Image understanding and the web: a state-of-the-art review. ABSTRACTThe contextual information of Web images is investigated to address the issue of characterizing their content with semantic descriptors and therefore bridge the semantic gap, i.e. the gap between their automated low-level representation in terms of colors, textures, shapes… and their semantic interpretation. Such characterization allows for understanding the image content and is crucial in important Web-based tasks such as image indexing and retrieval. Although we are highly motivated by the availability of rich knowledge on the Web and the relative success achieved by commercial search engines in automatically characterizing the image content using contextual information in Web pages, we are aware that the unpredictable quality of the contextual information is a major limiting factor. Among the reasons explaining the difficulty to leverage on the image contextual information, some problems are related to the characterization and extraction of this information. Indeed, the first issue is the lack of large-scale studies to highlight what is considered the relevant contextual information of an image, where it is located in a Web page and whether it is consistent across Web pages of different types, content layouts and domains. Also, the matter related to the extraction of this contextual information is topical as state-of-the-art automated extraction tools are unable to handle the heterogeneous Web. As far as the processing of the contextual information is concerned, problems linked to the syntactic and semantic characterizations of the textual components are important to address in order to tackle the semantic gap. Furthermore, questions pertaining to the organization of these textual components into coherent structures that are usable in image indexing and retrieval frameworks shall arise. To address these issues, we lay down the anatomy of a generic context-based Web image understanding framework and propose its stage-based decomposition, covering topical issues from information indexing and retrieval, image description models, natural language processing, Web page segmentation and automated information extraction. For each of the identified stages, we review state-of-the-art solutions in the literature categorized and analyzed under the light of the techniques used.Image retrieval systems are defined as computer systems for searching and retrieving images from large collections of digital images according to the users' information needs. Examples of image corpora include medical images, satellite images, photo galleries, the Web, etc.(a) The three rows shown are the first rows of the retrieved search results of the Google, Yahoo and Bing search engines respectively for the query "Baby boy in blue". 6 images (framed in red) out of 22 images are relevant.(b) The three rows shown are the first rows of the retrieved search results of the Google, Yahoo and Bing search engines respectively for the query "Car hits lorry". 4 images (framed in red) out ...

Improved support vector machine classification algorithm based on adaptive feature weight updating in the Hadoop cluster environment

“…However, although semantic features can effectively express the content of an image, semantic features are very complicated, and their extraction is a relatively difficult endeavor. At present, most semantic feature extraction algorithms are based on the low-level visual features of images [14]. Therefore, researchers have recently begun to consider image classification methods that integrate a variety of features.…”

A Novel Image Annotation Method based on Kernel Methods for Structured Prediction

“…For huge volumes of images, the tasks of classification and annotation have been attracted much attention since they may provide useful information for quick image search and content-based image retrieval (CBIR) [1]. For classification tasks, images are classified into several predefined categories by a trained model [2].…”

Integrating Visual and Textual Features for Web Image Clustering

“…F word "apple" could mean the fruit "apple" "Apple". So when we search for this word, t would be returned in a mixture of different top b and multimedia promotion on the a [1]. As a critical the multimedia, many applications ce of Web image lgorithms.…”