The Semantic Level of Shannon Information: Are Highly Informative Words Good Keywords? A Study on German

Kölbl, Max; Kyogoku, Yuki; Philipp, J. Nathanael; Richter, Michael; Rietdorf, Clemens; Yousef, Tariq

doi:10.1007/978-3-030-63787-3_5

Cited by 2 publications

(3 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bharti et al ( 2017 ) follow the definition in Zhang ( 2008 ) that keywords reflect the “core sentiment” of a document, and they are utilized for access and recovery of information and documents (Bharti et al, 2017 ). Due to the descriptive nature of keywords, they are either nouns or noun phrases, i.e., proper names, which was confirmed in previous work from Kölbl et al ( 2021 ). In this paper we maintain this classification.…”

Section: Introductionsupporting

confidence: 70%

“…We can define multiple subsets, firstly we define the subset that contains only keywords that also occur in the text, K T , ∩ , and secondly we define the subset that contains all keywords that do not occur in the text, K T , \ . As shown in Kölbl et al ( 2021 ) most keywords are names, in most cases either names of persons or organizations, or their abbreviations. This is mostly due to the fact that keywords most often are used in information retrieval systems.…”

Section: Theoretical Foundationmentioning

confidence: 99%

“…In the end, this TextRank algorithm computes scores of the text units by the iteration until convergence and based on the final scores; the relevant text units are extracted. Kölbl et al ( 2021 ) have shown, that TextRank performs very poorly for German texts.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Beyond the Failure of Direct-Matching in Keyword Evaluation: A Sketch of a Graph Based Solution

Kölbl

Kyogoku

Philipp

et al. 2022

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

The starting point of this paper is the observation that methods based on the direct match of keywords are inadequate because they do not consider the cognitive ability of concept formation and abstraction. We argue that keyword evaluation needs to be based on a semantic model of language capturing the semantic relatedness of words to satisfy the claim of the human-like ability of concept formation and abstraction and achieve better evaluation results. Evaluation of keywords is difficult since semantic informedness is required for this purpose. This model must be capable of identifying semantic relationships such as synonymy, hypernymy, hyponymy, and location-based abstraction. For example, when gathering texts from online sources, one usually finds a few keywords with each text. Still, these keyword sets are neither complete for the text nor are they in themselves closed, i.e., in most cases, the keywords are a random subset of all possible keywords and not that informative w.r.t. the complete keyword set. Therefore all algorithms based on this cannot achieve good evaluation results and provide good/better keywords or even a complete keyword set for a text. As a solution, we propose a word graph that captures all these semantic relationships for a given language. The problem with the hyponym/hyperonym relationship is that, unlike synonyms, it is not bidirectional. Thus the space of keyword sets requires a metric that is non-symmetric, in other words, a quasi-metric. We sketch such a metric that works on our graph. Since it is nearly impossible to obtain such a complete word graph for a language, we propose for the keyword task a simpler graph based on the base text upon which the keyword sets should be evaluated. This reduction is usually sufficient for evaluating keyword sets.

show abstract

Section: Introductionsupporting

confidence: 70%

Section: Theoretical Foundationmentioning

confidence: 99%