Semantic Multimedia Retrieval using Lexical Query Expansion and Model-Based Reranking

Haubold, Alexander; Natsev, Apostol; Naphade, Milind

doi:10.1109/icme.2006.262892

Cited by 26 publications

(20 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[34] and [32]. In more recent work, the Lesk-based similarity measure [4] is demonstrated as one of the best measures for lexical relatedness and is employed in [13] for lexical query expansion. WordNet-based concept extraction is also investigated in [14] to evaluate the effectiveness of highlevel concepts used in video retrieval where it achieved results comparable to user-selected query concepts.…”

Section: Related Workmentioning

confidence: 99%

Semantics-based selection of everyday concepts in visual lifelogging

Wang

Smeaton

2012

Int J Multimed Info Retr

View full text Add to dashboard Cite

Concept-based indexing, based on identifying various semantic concepts appearing in multimedia, is an attractive option for multimedia retrieval and much research tries to bridge the semantic gap between the media's low-level features and high-level semantics. Research into concept-based multimedia retrieval has generally focussed on detecting concepts from high quality media such as broadcast TV or movies, but it is not well addressed in other domains like lifelogging where the original data is captured with poorer quality. We argue that in noisy domains such as lifelogging, the management of data needs to include semantic reasoning in order to deduce a set of concepts to represent lifelog content for applications like searching, browsing or summarisation. Using semantic concepts to manage lifelog data relies on the fusion of automaticallydetected concepts to provide a better understanding of the lifelog data. In this paper, we investigate the selection of semantic concepts for lifelogging which includes reasoning on semantic networks using a density-based approach. In a series of experiments we compare different semantic reasoning approaches and the experi-

show abstract

Section: Related Workmentioning

confidence: 99%

Semantics-based selection of everyday concepts in visual lifelogging

Wang

Smeaton

2012

Int J Multimed Info Retr

View full text Add to dashboard Cite

show abstract

“…This method can be used to search visual content without any textual metadata, or it can be used to supplement alternative retrieval approaches, such as textbased, speech-based, or visual content-based retrieval. For more information on concept-based retrieval approaches, the reader is referred to [20][21][22][23][24][25]. For the experiments in this paper, we largely adopt the approach of [25] but instead of the Lesk similarity measure, we use the Jiang-Conrath similarity with Web-based IC values.…”

Section: Concept-based Video Retrieval Applicationmentioning

confidence: 99%

Web-based information content and its application to concept-based video retrieval

Haubold

Natsev

2008

Proceedings of the 2008 International Conference on Content-Based Image and Video Retrieval

Self Cite

View full text Add to dashboard Cite

Semantic similarity between words or phrases is frequently used to find matching correlations between search queries and documents when straightforward matching of terms fails. This is particularly important for searching in visual databases, where pictures or video clips have been automatically tagged with a small set of semantic concepts based on analysis and classification of the visual content. Here, the textual description of documents is very limited, and semantic similarity based on WordNet's cognitive synonym structure, along with information content derived from term frequencies, can help to bridge the gap between an arbitrary textual query and a limited vocabulary of visual concepts. This approach, termed concept-based retrieval, has received significant attention over the last few years, and its success is highly dependent on the quality of the similarity measure used to map textual query terms to visual concepts.In this paper, we consider some issues of semantic similarity measures based on Information Content (IC), and propose a way to improve them. In particular, we note that most IC-based similarity measures are derived from a small and relatively outdated corpus (the Brown corpus), which does not adequately capture the usage pattern of many contemporary terms: for example, out of more than 150,000 WordNet terms, only about 36,000 are represented. This shortcoming reflects very negatively on the coverage of typical search query terms. We therefore suggest using alternative IC corpora that are larger and better aligned with the usage of modern vocabulary. We experimentally derive two such corpora using the WWW Google search engine, and show that they provide better coverage of vocabulary, while showing comparable frequencies for Brown corpus terms. Finally, we evaluate the two proposed IC corpora in the context of a concept-based video retrieval application using the TRECVID 2005, 2006, and 2007 datasets, and we show that they increase average precision results by up to 200%.

show abstract

“…They discuss several types of methods to expand the query with visual keywords. Another approach to query expansion in multimedia retrieval by Haubold et al [6], uses lexical expansions of the queries. Semantic distances between words is also explored by Smeaton and Quigley [23] to perform query expansion.…”

Section: Keyword Based Multimedia Similaritymentioning

confidence: 99%

Exploring multimedia in a keyword space

Magalhães

Ciravegna

Rüger

2008

Proceedings of the 16th ACM International Conference on Multimedia

View full text Add to dashboard Cite

We address the problem of searching multimedia by semantic similarity in a keyword space. In contrast to previous research we represent multimedia content by a vector of keywords instead of a vector of low-level features. This vector of keywords can be obtained through user manual annotations or computed by an automatic annotation algorithm. In this setting, we studied the influence of two aspects of the search by semantic similarity process: (1) accuracy of user keywords versus automatic keywords and (2) functions to compute semantic similarity between keyword vectors of two multimedia documents. We consider these two aspects to be crucial in the design of a keyword space that can exploit social-media information and can enrich applications such as Flickr and YouTube. Experiments were performed on an image and a video dataset with a large number of keywords, with different similarity functions and with two annotation methods. Surprisingly, we found that multimedia semantic similarity with automatic keywords performs as good as or better than 95% accurate user keywords.

show abstract

Semantic Multimedia Retrieval using Lexical Query Expansion and Model-Based Reranking

Cited by 26 publications

References 4 publications

Semantics-based selection of everyday concepts in visual lifelogging

Semantics-based selection of everyday concepts in visual lifelogging

Web-based information content and its application to concept-based video retrieval

Exploring multimedia in a keyword space

Contact Info

Product

Resources

About