Exploring statistical correlations for image retrieval

Abstract

Bridging the cognitive gap in image retrieval has been an active research direction in recent years, of which a key challenge is to get enough training data to learn the mapping functions from low-level feature spaces to high-level semantics. In this paper, image regions are classified into two types: key regions representing the main semantic contents and environmental regions representing the contexts. We attempt to leverage the correlations between types of regions to improve the performance of image retrieval. A Context Expansion approach is explored to take advantages of such correlations by expanding the key regions of the queries using highly correlated environmental regions according to an image thesaurus. The thesaurus serves as both a mapping function between image low-level features and concepts and a store of the statistical correlations between different concepts. It is constructed through a data-driven approach which uses Web data (images, their surrounding textual annotations) as training data source to learn the region concepts and to explore the statistical correlations. Experimental results on a database of 10,000 general-purpose images show the effectiveness of our proposed approach in both improving search precision (i.e. filter irrelevant images) and recall (i.e. retrieval relevant images whose context may be varied). Several major factors which have impact on the performance of our approach are also studied.

A query expansion framework in image retrieval domain based on local and global analysis

“…Until recently only a few techniques have been investigated along this direction [28–31]. For example, a context expansion approach has been explored recently in [28] to take advantages of the correlations between key regions (representing the main semantic contents) and environmental regions (representing the contexts) by expanding the key regions of the queries using highly correlated environmental regions according to an image thesaurus. The thesaurus is constructed through a data-driven approach, serves as both a mapping function between image low-level features and concepts and a store of the statistical correlations between different concepts.…”

“…The approaches mentioned above are either data dependent on the entire collection [28] or relied on keywords from image captions [29–31]. Our proposed approach of query expansion is fundamentally different from the above approaches.…”

Implicit visual concept modeling in image / video annotation

“…To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ARTEMIS ' Figure 1: System Overview sources in order to be used for training concept classifiers [10]- [12]. Such data sources include content that has been annotated by user-defined tags (e.g., Picasa, Flickr, Yahoo!…”

Image annotation using clickthrough data

“…To compensate for the high cost in manually labelling training samples, research has recently moved towards the use of alternative data sources that are automatically acquired from the Web in order to be used for training concept classifiers [22,13,4,19]. Such data sources include usergenerated multimedia content annotated with user-defined tags (e.g., YouTube and Flickr) [13,4,19], as well as images and videos annotated with keywords automatically extracted from the text that surrounds them in the Web pages they are embedded in [22]. In this paradigm shift, Web communities unknowingly share in the generation of large amounts of labelled data.…”