An evaluation dataset for the toponym resolution task

Much of the information stored on the web contains geographical context, but current search engines treat such context in the same way as all other content. In this paper the design, implementation and evaluation of a spatially-aware search engine are described which is capable of handling queries in the form of the triplet of . The process of identifying geographic references in documents and assigning appropriate footprints to documents, to be stored together with document terms in an appropriate indexing structure allowing real-time search is described. Methods allowing users to query and explore results which have been relevance ranked in terms of both thematic and spatial relevance have been implanted and a usability study indicates that users are happy with the range of spatial relationships available and intuitively understand how to use such a search engine. Normalised precision for 38 queries, containing four types of spatial relationships is significantly higher (p < 0.001) for search exploiting spatial information than pure text search.

show abstract

“…Martins et al, 2005;Leidner, 2006). The GeoCLEF evaluation campaign (Gey et al, 2005) addresses some of the issues concerning system evaluation.…”

Section: Discussionmentioning

confidence: 99%

The design and implementation of SPIRIT: a spatially aware search engine for information retrieval on the Internet

Purves

Clough

Jones

et al. 2007

International Journal of Geographical Information Science

141

View full text Add to dashboard Cite

show abstract

“…In these test collections, toponyms have been manually labelled with the correct referent. These resources include the the TR-CoNLL corpus [8], GeoSemCor [7], the SpatialML [9] and the LGL [10] corpora.…”

Section: Discussionmentioning

confidence: 99%

Approaches to disambiguating toponyms

Buscaldi

2011

SIGSPATIAL Special

View full text Add to dashboard Cite

Many approaches have been proposed in recent years in the context of Geographic Information Retrieval (GIR), mostly in order to deal with geographically constrained information in un-structured texts. Most of these approaches share a common scheme: in order to disambiguate a toponymtwithnpossible referents in a documentd, they find a certain number of context toponymsc0,...,ckthat are contained ind.A score for each referent is calculated according to the context toponyms, and the referent with the highest score is selected. According to the method used to calculate the score, Toponym Disambiguation (TD) methods may be grouped into three main categories, as proposed by [7]:•map-based:methods that use an explicit representation of toponyms on a map, for instance to calculate the average distance of unambiguous context toponyms from referents;•knowledge-based:methods that exploit external knowledge sources such as gazetteers, Wikipedia or ontologies to find disambiguation clues;•data-drivenorsupervised:methods based on machine learning techniques.

show abstract

“…Parameters such as the gazetteer's scope, coverage, correctness, granularity, balance and richness of annotation influence the outcome of any evaluation experiment (Leidner, 2006). Therefore, even standardized evaluation corpora such as the one designed by Leidner require geo-taggers to use a fixed gazetteer to provide comparable results.…”

Section: Introductionmentioning

confidence: 99%

“…Leidner (Leidner, 2006) provides such an evaluation data set and describes the process of designing evaluation corpora. Nevertheless evaluating geographical data mining is still a tricky task.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Utility Centered Approach for Evaluating and Optimizing Geo-Tagging

Weichselbraun

2009

Proceedings of the International Conference on Knowledge Discovery and Information Retrieval

View full text Add to dashboard Cite

Abstract:Geo-tagging is the process of annotating a document with its geographic focus by extracting a unique locality that describes the geographic context of the document as a whole (Amitay et al., 2004). Accurate geographic annotations are crucial for geospatial applications such as Google Maps or the IDIOM Media Watch on Climate Change (Hubmann-Haidvogel et al., 2009), but many obstacles complicate the evaluation of such tags. This paper introduces an approach for optimizing geo-tagging by applying the concept of utility from economic theory to tagging results. Computing utility scores for geo-tags allows a fine grained evaluation of the tagger's performance in regard to multiple dimensions specified in use case specific domain ontologies and provides means for addressing problems such as different scope and coverage of evaluation corpora. The integration of external data sources and evaluation ontologies with user profiles ensures that the framework considers use case specific requirements. The presented model is instrumental in comparing different geo-tagging settings, evaluating the effect of design decisions, and customizing geo-tagging to a particular use cases.

show abstract

An evaluation dataset for the toponym resolution task

Cited by 32 publications

References 4 publications

The design and implementation of SPIRIT: a spatially aware search engine for information retrieval on the Internet

The design and implementation of SPIRIT: a spatially aware search engine for information retrieval on the Internet

Approaches to disambiguating toponyms

A Utility Centered Approach for Evaluating and Optimizing Geo-Tagging

Contact Info

Product

Resources

About