Neural Network Guided Fast and Efficient Query-Based Stemming by Predicting Term Co-occurrence Statistics

Singh, Pankaj; Bhowmick, Plaban Kumar

doi:10.1007/s42979-022-01081-5

Cited by 2 publications

(1 citation statement)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A recently proposed method by Singh & Bhowmick (2022) uses neural network to predict co-occurrence similarity between a query term and its potential morphological variant. Potential or candidate variants are initially determined by using lexical similarity of the words in the corpus.…”

Section: Related Workmentioning

confidence: 99%

A selective approach to stemming for minimizing the risk of failure in information retrieval systems

Göksel

Arslan

Dinçer

2023

PeerJ Computer Science

View full text Add to dashboard Cite

Stemming is supposed to improve the average performance of an information retrieval system, but in practice, past experimental results show that this is not always the case. In this article, we propose a selective approach to stemming that decides whether stemming should be applied or not on a query basis. Our method aims at minimizing the risk of failure caused by stemming in retrieving semantically-related documents. The proposed work mainly contributes to the IR literature by proposing an application of selective stemming and a set of new features that derived from the term frequency distributions of the systems in selection. The method based on the approach leverages both some of the query performance predictors and the derived features and a machine learning technique. It is comprehensively evaluated using three rule-based stemmers and eight query sets corresponding to four document collections from the standard TREC and NTCIR datasets. The document collections, except for one, include Web documents ranging from 25 million to 733 million. The results of the experiments show that the method is capable of making accurate selections that increase the robustness of the system and minimize the risk of failure (i.e., per query performance losses) across queries. The results also show that the method attains a systematically higher average retrieval performance than the single systems for most query sets.

show abstract

Section: Related Workmentioning

confidence: 99%

A selective approach to stemming for minimizing the risk of failure in information retrieval systems

Göksel

Arslan

Dinçer

2023

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

Building a Multilevel Inflection Handling Stemmer to Improve Search Effectiveness for Urdu Language

Jabbar,

Iqbal,

Alaulamie

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Stemming is an essential step in various Natural Language Processing (NLP) applications and is used to reduce different variants of the query words to a standard form to avoid the vocabulary mismatch issue in Information Retrieval (IR) systems. Due to specific grammatical rules and complex morphological structures, finding an effective stemming algorithm in Urdu is challenging task. Although, several stemming algorithms have been proposed for the Urdu text stemming; however, none of them extract the stem from multilevel inflected forms. In this context, according to the best of our knowledge, this is a first effort towards the proposition and evaluation of a novel Urdu Text Stemmer (UTS) that can deal with multi-level inflection forms in Urdu text. The experimental evaluation of the proposed scheme has been conducted on the text-based and word-based custom-developed corpus. The proposed stemming technique is rigorously evaluated and compared with state-of-the-art stemming algorithms. Experimental results demonstrate that UTS outperforms existing Urdu stemmers and achieves an accuracy of 94.92% and 91.8% on word corpus and text corpus, respectively. We also evaluated our proposed system in an Information Retrieval application for Urdu, using the Collection for Urdu Retrieval Evaluation (CURE) dataset. Our approach for information retrieval outperformed and improved both recall and precision metrics.

show abstract

Neural Network Guided Fast and Efficient Query-Based Stemming by Predicting Term Co-occurrence Statistics

Cited by 2 publications

References 48 publications

A selective approach to stemming for minimizing the risk of failure in information retrieval systems

A selective approach to stemming for minimizing the risk of failure in information retrieval systems

Building a Multilevel Inflection Handling Stemmer to Improve Search Effectiveness for Urdu Language

Contact Info

Product

Resources

About