An Experimental Comparison of Explicit Semantic Analysis Implementations for Cross-Language Retrieval

Abstract

Abstract. Explicit Semantic Analysis (ESA) has been recently proposed as an approach to computing semantic relatedness between words (and indirectly also between texts) and has thus a natural application in information retrieval, showing the potential to alleviate the vocabulary mismatch problem inherent in standard Bag-of-Word models. The ESA model has been also recently extended to cross-lingual retrieval settings, which can be considered as an extreme case of the vocabulary mismatch problem. The ESA approach actually represents a class of approaches and allows for various instantiations. As our first contribution, we generalize ESA in order to clearly show the degrees of freedom it provides. Second, we propose some variants of ESA along different dimensions, testing their impact on performance on a cross-lingual mate retrieval task on two datasets (JRC-ACQUIS and Multext). Our results are interesting as a systematic investigation has been missing so far and the variations between different basic design choices are significant. We also show that the settings adopted in the original ESA implementation are reasonably good, which to our knowledge has not been demonstrated so far, but can still be significantly improved by tuning the right parameters (yielding a relative improvement on a cross-lingual mate retrieval task of between 62% (Multext) and 237% (JRC-ACQUIS) with respect to the original ESA model).

Econo-ESA in semantic text similarity

“…We compare our proposal to each improvement and summarize related studies based on Sorg and Cimiano’s framework (Sorg and Cimiano 2010). This framework considers the index matrix, association strength, similarity measurement, and dimension limitation as ESA enhancements.…”

“…Hassan and Mihalcea (2009) refined ESA association strength by considering the length of the articles. Sorg and Cimiano (2010) examined five association strength schemes: TF, TFIDF, TFIDF*, BM25, and cosine. They (Sorg and Cimiano 2012) also introduced TFICF, TFICF 2 , and TFICF 3 .…”

“…We do not propose a new association strength scheme such as (Aryafar and Shokoufandeh 2011; Fernandez et al 2011; Hassan and Mihalcea 2009; Sorg and Cimiano 2010; 2012). Econo-ESA uses an original ESA scheme, TFIDF, to highlight the improvement of our proposal to the original ESA.…”

“…Researchers have further developed ESA for use in different environments, including information retrieval (Hassan and Mihalcea 2009; Polajnar et al 2013; Potthast et al 2008; Potthast et al 2012; Scholl et al 2010; Sorg and Cimiano 2010; 2012; Tanase and Kapetanios 2012), image retrieval (Popescu and Grefenstette 2011; Zhang et al 2012), semantic text similarity (STS) (Aggarwal et al 2012; Martín et al 2013; Szarvas et al 2011), categorization (O’Banion et al 2012; Szarvas et al 2011; Szczuka et al 2011), machine translation (Matsuno and Ishida 2011), and question-answering (Walter et al 2012). It is also used in knowledge discovery (Yan and Jin 2012), music classification (Aryafar and Shokoufandeh 2011), learning systems (Schmidt et al 2011), text disambiguation (Fernandez et al 2011), and case based reasoning systems (Patelia et al 2011).…”

“…It is also used in knowledge discovery (Yan and Jin 2012), music classification (Aryafar and Shokoufandeh 2011), learning systems (Schmidt et al 2011), text disambiguation (Fernandez et al 2011), and case based reasoning systems (Patelia et al 2011). Thanks to Wikipedia’s language links, there are also numerous implementations in multilingual environments (Potthast et al 2008; Schönhofen et al 2008; Schmidt et al 2011; Sorg and Cimiano 2010; 2012; Tanase and Kapetanios 2012). …”