Evaluating Entity Summarization Using a Game-Based Ground Truth

Abstract

Abstract. In recent years, strategies for Linked Data consumption have caught attention in Semantic Web research. For direct consumption by users, Linked Data mashups, interfaces, and visualizations have become a popular research area. Many approaches in this field aim to make Linked Data interaction more user friendly to improve its accessibility for nontechnical users. A subtask for Linked Data interfaces is to present entities and their properties in a concise form. In general, these summaries take individual attributes and sometimes user contexts and preferences into account. But the objective evaluation of the quality of such summaries is an expensive task. In this paper we introduce a game-based approach aiming to establish a ground truth for the evaluation of entity summarization. We exemplify the applicability of the approach by evaluating two recent summarization approaches.

ESBM: An Entity Summarization BenchMark

“…As shown in Table 1, some benchmarks are no longer available. Others are available [22,7,8] but they are small and have limitations. Specifically, [22] has a taskspecific nature, and [7,8] exclude classes and/or literals.…”

Novel Node Importance Measures to Improve Keyword Search over RDF Graphs

“…The second possible path to follow is another research field, called Entity Summarization (Cheng et al 2011;Thalhammer et al 2012), which aims at providing meaningful descriptions of entities. This field is an extension of the Ontology Summarization field (Zhang et al 2007;Peroni et al 2008).…”

Entity Summarization: State of the Art and Future Challenges

“…The WhoKnows?Movies! game [53] was designed to collect ground-truth importance of triples. This online quiz game attracted 217 players who answered 8,308 questions about 2,829 triples in the descriptions of 60 movies taken from Freebase.…”