A selective approach to index term weighting for robust information retrieval based on the frequency distributions of query terms

Arslan, Ahmet; Dinçer, Bekir Taner

doi:10.1007/s10791-018-9347-9

Cited by 5 publications

(16 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous attempts to use a selective search strategy have considered more than two configurations. Arslan and Dinçer [3] used eight configurations consisting of eight term-weighting models in which the hyperparameters were first optimized by using Grid search. In this approach, the selection of a configuration to process a new query is based on the frequency distributions of query terms on the document collection.…”

Section: Related Workmentioning

confidence: 99%

“…In this approach, the selection of a configuration to process a new query is based on the frequency distributions of query terms on the document collection. By considering more configurations in selective search strategy the system performance improved [3,54] when compared to SQE.…”

Section: Related Workmentioning

confidence: 99%

“…The most effective selective search strategy approaches make the configuration selection decision based on training by machine learning from queries [16,23,40,53] rather than making the decision based on a query score as was done previously [3,14,54]. He and Ounis [23] introduced a model selection approach in which queries were clustered considering three pre-retrieval features and the best-performing retrieval model was attributed to each cluster.…”

Section: Related Workmentioning

confidence: 99%

“…Either the configuration pool is limited to 2-8 configurations [2,3,23,53] and hence needs no further limitation, or it contains 20, 000-80, 000 configurations [15,16,40].…”

Section: Overviewmentioning

confidence: 99%

“…SQE's use of only two search strategies did not improve search engine performance very much, however, thus it is no longer used. Subsequently, selective search strategy approaches have increased the number of candidate configurations (i.e., the set of chosen component processes and their hyperparameter values), the system can choose from to process a new query [3,8,16,23,54].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Defining an Optimal Configuration Set for Selective Search Strategy - A Risk-Sensitive Approach

Mothe

Ullah

2021

Proceedings of the 30th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

A search engine generally applies a single search strategy to any user query. The search combines many component processes (e.g., indexing, query expansion, search-weighting model, document ranking) and their hyperparameters, whose values are optimized based on past queries and then applied to all future queries. Even an optimized system may perform poorly on some queries, however, whereas another system might perform better on those queries. Selective search strategy aims to select the most appropriate combination of components and hyperparameter values to apply for each individual query. The number of candidate combinations is huge. To adapt best to any query, the ideal system would use many combinations. In the real world it would be too costly to use and maintain thousands of configurations. A trade-off must therefore be found between performance and cost. In this paper, we describe a risksensitive approach to optimize the set of configurations that should be included in a selective search strategy. This approach solves the problem of which and how many configurations to include in the system. We show that the use of 20 configurations results in significantly greater effectiveness than current approaches when tested on three TREC reference collections, by about 23% when compared to L2R documents and about 10% when compared to other selective approaches, and that it offers an appropriate trade-off between system complexity and system effectiveness. CCS CONCEPTS• Information systems → Retrieval models and ranking; Retrieval effectiveness; Information retrieval query processing.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…Either the configuration pool is limited to 2-8 configurations [2,3,23,53] and hence needs no further limitation, or it contains 20, 000-80, 000 configurations [15,16,40].…”

Section: Overviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Defining an Optimal Configuration Set for Selective Search Strategy - A Risk-Sensitive Approach

Mothe

Ullah

2021

Proceedings of the 30th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

show abstract

Selective Query Processing: A Risk-Sensitive Selection of Search Configurations

Mothe

Zia

2023

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

In information retrieval systems, search parameters are optimized to ensure high effectiveness based on a set of past searches and these optimized parameters are then used as the search configuration for all subsequent queries. A better approach, however, would be to adapt the parameters to fit the query at hand. Selective query expansion is one such an approach, in which the system decides automatically whether or not to expand the query, resulting in two possible search configurations. This approach was extended recently to include many other parameters, leading to many possible search configurations where the system automatically selects the best configuration on a per-query basis. One problem with this approach is the system training which requires evaluation of each training query with every possible configuration. In real-world systems, so many parameters and possible values must be evaluated that this approach is impractical, especially when the system must be updated frequently, as is the case for commercial search engines. In general, the more configurations, the greater the effectiveness when configuration selection is appropriate but also the greater the risk of decreasing effectiveness in the case of an inappropriate configuration selection. To determine the ideal configurations to use on a per-query basis in real-world systems we developed a method in which a restricted number of possible configurations is pre-selected and then used in a meta-search engine that decides the best search configuration on a per query basis. We define a risk-sensitive approach for configuration pre-selection that considers the risk-reward trade-off between the number of configurations kept, and system effectiveness. We define two alternative risk functions to apply to different goals. For final configuration selection, the decision is based on query feature similarities. We compare two alternative risk functions on two query types: ad hoc and diversity and compare these to more sophisticated machine learning-based methods. We find that a relatively small number of configurations (20) selected by our risk-sensitive model is sufficient to obtain results close to the best achievable results for each query. Effectiveness is increased by about 15% according to the P@10 and nDCG@10 evaluation metrics when compared to traditional grid search using a single configuration and by about 20% when compared to learning to rank documents. Our risk-sensitive approach works for both diversity- and ad hoc-oriented searches. Moreover, the similarity-based selection method outperforms the more sophisticated approaches. Thus, we demonstrate the feasibility of developing per-query information retrieval systems, which will guide future research in this direction.

show abstract

Towards Robust Neural Rankers with Large Language Model: A Contrastive Training Approach

Pan,

Fan,

Liu

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Pre-trained language model-based neural rankers have been widely applied in information retrieval (IR). However, the robustness issue of current IR models has not received sufficient attention, which could significantly impact the user experience in practical applications. In this study, we focus on the defensive ability of IR models against query attacks while guaranteeing their retrieval performance. We discover that improving the robustness of IR models not only requires a focus on model architecture and training methods but is also closely related to the quality of data. Different from previous research, we use large language models (LLMs) to generate query variations with the same intent, which exhibit richer and more realistic expressions while maintaining consistent query intent. Based on LLM-generated query variations, we propose a novel contrastive training framework that substantially enhances the robustness of IR models to query perturbations. Specifically, we combine the contrastive loss in the representation space of query variations with the ranking loss in the retrieval training stage to improve the model’s ability to understand the underlying semantic information of queries. Experimental results on two public datasets, WikiQA and ANTIQUE, demonstrate that the proposed contrastive training approach effectively improves the robustness of models facing query attack scenarios while outperforming baselines in retrieval performance. Compared with the best baseline approach, the improvements in average robustness performance of Reranker IR models are 24.9%, 26.5%, 27.0%, and 75.0% on WikiQA and 8.7%, 1.9%, 6.3%, and 13.6% on ANTIQUE, in terms of the MAP (Mean Average Precision), MRR (Mean Reciprocal Rank), nDCG@10 (Normalized Discounted Cumulative Gain) and P@10 (Precision), respectively.

show abstract

A selective approach to index term weighting for robust information retrieval based on the frequency distributions of query terms

Cited by 5 publications

References 45 publications

Defining an Optimal Configuration Set for Selective Search Strategy - A Risk-Sensitive Approach

Defining an Optimal Configuration Set for Selective Search Strategy - A Risk-Sensitive Approach

Selective Query Processing: A Risk-Sensitive Selection of Search Configurations

Towards Robust Neural Rankers with Large Language Model: A Contrastive Training Approach

Contact Info

Product

Resources

About