Fusing Visual and Textual Retrieval Techniques to Effectively Search Large Collections of Wikipedia Images

Lau, Cher Han; Tjondronegoro, Dian; Zhang, Jinglan; Geva, Shlomo; Liu, Y.

doi:10.1007/978-3-540-73888-6_34

Cited by 7 publications

(7 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The few studies that have examined Wikipedia images focused on their use for building image databases (Tsikrika, Kludas, and Popescu 2012). The intended target for these databases is in training and testing image retrieval algorithms (e.g., (Lau et al 2006;Tsikrika, Popescu, and Kludas 2011;Aletras and Stevenson 2013;Benavent et al 2013)), making these algorithms vulnerable to biases as discussed above. The most prominent research that centered on Wikipedia's images themselves is from Viegas (2007).…”

Section: Images On Wikipediamentioning

confidence: 99%

The_Tower_of_Babel.jpg: Diversity of Visual Encyclopedic Knowledge Across Wikipedia Language Editions

Lin²,

Adar

et al. 2018

ICWSM

View full text Add to dashboard Cite

Across all Wikipedia language editions, millions of images augment text in critical ways. This visual encyclopedic knowledge is an important form of wikiwork for editors, a critical part of reader experience, an emerging resource for machine learning, and a lens into cultural differences. However, Wikipedia research--and cross-language edition Wikipedia research in particular--has thus far been limited to text. In this paper, we assess the diversity of visual encyclopedic knowledge across 25 language editions and compare our findings to those reported for textual content. Unlike text, translation in images is largely unnecessary. Additionally, the Wikimedia Foundation, through the Wikipedia Commons, has taken steps to simplify cross-language image sharing. While we may expect that these factors would reduce image diversity, we find that cross-language image diversity rivals, and often exceeds, that found in text. We find that diversity varies between language pairs and content types, but that many images are unique to different language editions. Our findings have implications for readers (in what imagery they see), for editors (in deciding what images to use), for researchers (who study cultural variations), and for machine learning developers (who use Wikipedia for training models).

show abstract

Section: Images On Wikipediamentioning

confidence: 99%

The_Tower_of_Babel.jpg: Diversity of Visual Encyclopedic Knowledge Across Wikipedia Language Editions

Lin²,

Adar

et al. 2018

ICWSM

View full text Add to dashboard Cite

show abstract

“…Concerning multimedia fragment retrieval approaches, most of the proposals in literature combine textual XML retrieval with image content-based approaches (Iskandar, Pehcevski, Thom, & Tahaghoghi, 2006;Lau, Tjondronegoro, Zhang, Geva, & Liu, 2006;Mihajlovic et al, 2005;Tjondronegoro, Zhang, Gu, Nguyen, & Geva, 2006). Content-based algorithms are used to score images similar to the one in the query (queries are often composed of text and an example image), and image scores are then combined with the ones obtained on text with traditional XML retrieval systems.…”

Section: Using Structural Contextmentioning

confidence: 99%

Investigating the document structure as a source of evidence for multimedia fragment retrieval

Torjmen-Khemakhem

Pinel-Sauvagnat

2013

Information Processing & Management

View full text Add to dashboard Cite

International audienceMultimedia objects can be retrieved using their context that can be for instance the text surrounding them in documents. This text may be either near or far from the searched objects. Our goal in this paper is to study the impact, in term of effectiveness, of text position relatively to searched objects. The multimedia objects we consider are described in structured documents such as XML ones. The document structure is therefore exploited to provide this text position in documents. Although structural information has been shown to be an effective source of evidence in textual information retrieval, only a few works investigated its interest in multimedia retrieval. More precisely, the task we are interested in this paper is to retrieve multimedia fragments (i.e. XML elements having at least one multimedia object). Our general approach is built on two steps: we first retrieve XML elements containing multimedia objects, and we then explore the surrounding information to retrieve relevant multimedia fragments. In both cases, we study the impact of the surrounding information using the documents structure.Our work is carried out on images, but it can be extended to any other media, since the physical content of multimedia objects is not used. We conducted several experiments in the context of the Multimedia track of the INEX evaluation campaign. Results showed that structural evidences are of high interest to tune the importance of textual context for multimedia retrieval. Moreover, the proposed approach outperforms state of the art approaches

show abstract

“…Others filter adhoc results into multimedia ones. For example, combining textual and image retrieval in XML multimedia retrieval is studied in Tjondronegoro et al (2005), Mihajlovic et al (2005) and Lau et al (2006). Results show that the use of visual features degrades the system accuracy.…”

Section: Existing Ir Approaches Adapted To Xml-multimedia Retrievalmentioning

confidence: 99%

Using textual and structural context for searching Multimedia Elements

Torjmen

Pinel-Sauvagnat

2010

IJBIDM

View full text Add to dashboard Cite

We investigate in this paper the use of XML structure in multimedia retrieval, particularly in context-based image retrieval. We propose two methods to represent multimedia objects: the first one is based on an implicit use of textual and structural context of multimedia objects, whereas the second one is based on an explicit use of both sources. Experimental evaluation is carried out using the INEX MultimediaFragments Task 2006 and 2007. We show that there is a strong vocabulary relation between the query and the multimedia object representation, and that using XML structure improves significantly the effectiveness of multimedia retrieval.

show abstract

Fusing Visual and Textual Retrieval Techniques to Effectively Search Large Collections of Wikipedia Images

Cited by 7 publications

References 20 publications

The_Tower_of_Babel.jpg: Diversity of Visual Encyclopedic Knowledge Across Wikipedia Language Editions

The_Tower_of_Babel.jpg: Diversity of Visual Encyclopedic Knowledge Across Wikipedia Language Editions

Investigating the document structure as a source of evidence for multimedia fragment retrieval

Using textual and structural context for searching Multimedia Elements

Contact Info

Product

Resources

About