Creating a Phrase Similarity Graph from Wikipedia

Stanchev, Lubomir

doi:10.1109/icsc.2014.22

Cited by 8 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The reason why our system has higher value for the MAP measure than the Apache Lucene system is because we find not only documents that contain words from the input query, but also documents that contain words and phrases that are semantically similar to those in the input query. In comparison with our previous work ( [39]), the presented system performs better because our old algorithm considers only disjoint paths between the input query and each of the documents and it does not take into account the complex interweaving network of edges that can exist in the probabilistic graph.…”

Section: Introductionmentioning

confidence: 89%

“…Second, let us quickly examine the algorithm from [39]. Given a document d and a query q, the scoring function is defined as shown in Equations 4 and 5.…”

Section: P Pmentioning

confidence: 99%

“…Human subjects have determined the documents that are relevant for every query. We compare our algorithm with the TF-IDF algorithms that is implemented in Apache Lucene and the algorithm from [39], which looks only at disjoint paths when comparing two nodes. The experimental section shows that our semantic search algorithm produces higher value for the mean average precision (MAP) over all queries than the other two algorithms.…”

Section: Introductionmentioning

confidence: 99%

“…Section V presents an approximation algorithm for finding the top-N relevant documents. Section VI shows how the algorithm compares with the Apache Lucene algorithm that is based on keywordsmatching and our old algorithm from [39] on the Cranfiled benchmark [6], while concluding remarks and areas for future research are outlined in Section VII.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Implementing Semantic Document Search Using a Bounded Random Walk in a Probabilistic Graph

Stanchev

2017

2017 IEEE 11th International Conference on Semantic Computing (ICSC)

Self Cite

View full text Add to dashboard Cite

Given a set of documents and an input query that is expressed using natural language, the problem of document search is retrieving all relevant documents ordered by the degree of relevance. Semantic document search fetches not only documents that contain words from the input query, but also documents that are semantically relevant. For example, the query "friendly pets" will consider documents that contain the words "dog" and "cat", among others. One way to implement semantic search is to use a probabilistic graph in which the input query is connected to the documents through paths that contain semantically similar words and phrases, where we use WordNet to initially populate the graph. Each edge in the graph is labeled with the conditional probability that the destination node is relevant given that the source node is relevant. Our semantic document search algorithm works in two phases. In the first phase, we find all documents in the graph that are close to the input query and create a bounded subgraph that includes the query, the found documents, and the paths that connect them. In the second phase, we simulate multiple random walks. Each random walk starts at the input query and continues until a document is reached, a jump outside the bounding subgraph is made, or the number of allowed jumps is exhausted. This allows us to rank the documents based on the number of random walks that terminated in them. We experimentally validated the algorithm on the Cranfield benchmark that contains 1400 documents and 225 natural language queries. We show that we achieve higher value for the mean average precision (MAP) measure than a keywords-based search algorithm and a previously published algorithm that relies on a variation of the probabilistic graph.

show abstract

Section: Introductionmentioning

confidence: 89%

“…Second, let us quickly examine the algorithm from [39]. Given a document d and a query q, the scoring function is defined as shown in Equations 4 and 5.…”

Section: P Pmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Implementing Semantic Document Search Using a Bounded Random Walk in a Probabilistic Graph

Stanchev

2017

2017 IEEE 11th International Conference on Semantic Computing (ICSC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…We also showed that the ¯ne-tuned algorithm improves these results even further and also gives us improved results on the entropy measure as compare to the cosine similarity algorithm. One area for future research is using an extended version of the similarity graph that contains information from Wikipedia [37] to perform document clustering. One challenge in this area is that the extended graph is relatively big (more than 10 GB) and computing the distance between documents can be computationally expensive.…”

Section: Conclusion and Future Researchmentioning

confidence: 99%

Fine-Tuning an Algorithm for Semantic Document Clustering Using a Similarity Graph

Stanchev

2016

Int. J. Semantic Computing

Self Cite

View full text Add to dashboard Cite

In this article, we examine an algorithm for document clustering using a similarity graph. The graph stores words and common phrases from the English language as nodes and it can be used to compute the degree of semantic similarity between any two phrases. One application of the similarity graph is semantic document clustering, that is, grouping documents based on the meaning of the words in them. Since our algorithm for semantic document clustering relies on multiple parameters, we examine how fine-tuning these values affects the quality of the result. Specifically, we use the Reuters-21578 benchmark, which contains [Formula: see text] newswire stories that are grouped in 82 categories using human judgment. We apply the k-means clustering algorithm to group the documents using a similarity metric that is based on keywords matching and one that uses the similarity graph. We evaluate the results of the clustering algorithms using multiple metrics, such as precision, recall, f-score, entropy, and purity.

show abstract