Field features: The impact in learning to rank approaches

Yang, Hua; Gonçalves, Teresa

doi:10.1016/j.asoc.2023.110183

Cited by 4 publications

(3 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our study, we investigate the faithfulness of different variants of Local Interpretable Model-Agnostic Explanation (LIME) techniques for explaining Neural Rank GAM models. We propose our variation of LIME with different sampling techniques such as Gaussian, SMOTE, Latin Hypercube Sampling (LHS), and Deterministic LIME (DLIME) 2 . We then evaluate our proposed techniques against the point-wise explanations of Locally Interpretable Ranking Model Explanation (LIRME) [10] and Explainable Search (EXS) [9], and non-LTR explanations of LIME in its official implementation [14] and SHapley Additive exPlanations (SHAP) [15] on the Yahoo!…”

Section: *Equal Contributionmentioning

confidence: 99%

“…To the best of our knowledge, there have not been studies on local point-wise explanations of Neural Rank GAM models. 2 These sampling techniques are described in Section IV-A. 3 These measures are defined in Section VI-C.…”

Section: Related Workmentioning

confidence: 99%

“…These models learn a ranking function to increase the relevance of each document to its corresponding query. LTR models are often complex since they are trained using many parameters to achieve high accuracy [2]. The complexity of ranking models can sometimes undermine their efficacy, as humans struggle to comprehend the rationale behind a particular order [3].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Local Interpretable Model-Agnostic Explanations for Neural Ranking Models

Rahnama,

Alfaro,

Wang

et al. 2024

Linköping Electronic Conference Proceedings

View full text Add to dashboard Cite

Neural Ranking Models have shown state-of-the-art performance in Learning-To-Rank (LTR) tasks. However, they are considered black-box models. Understanding the logic behind the predictions of such black-box models is paramount for their adaptability in the real-world and high-stake decision-making domains. Local explanation techniques can help us understand the importance of features in the dataset relative to the predicted output of these black-box models. This study investigates new adaptations of Local Interpretable Model-Agnostic Explanation (LIME) explanation for explaining Neural ranking models. To evaluate our proposed explanation, we explain Neural GAM models. Since these models are intrinsically interpretable Neural Ranking Models, we can directly extract their ground truth importance scores. We show that our explanation of Neural GAM models is more faithful than explanation techniques developed for LTR applications such as LIRME and EXS and non-LTR explanation techniques for regression models such as LIME and KernelSHAP using measures such as Rank Biased Overlap (RBO) and Overlap AUC. Our analysis is performed on the Yahoo! Learning-To-Rank Challenge dataset.

show abstract

Section: *Equal Contributionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Local Interpretable Model-Agnostic Explanations for Neural Ranking Models

Rahnama,

Alfaro,

Wang

et al. 2024

Linköping Electronic Conference Proceedings

View full text Add to dashboard Cite

show abstract

Dual cycle generative adversarial networks for web search

Lin,

Ying,

et al. 2024

Applied Soft Computing

View full text Add to dashboard Cite

A Self-Distilled Learning to Rank Model for Ad Hoc Retrieval

Keshvari,

Saeedi,

Sadoghi Yazdi

et al. 2024

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

Learning to rank models are broadly applied in ad-hoc retrieval for scoring and sorting documents based on their relevance to textual queries. The generalizability of the trained model in the learning to rank approach, however, can have an impact on the retrieval performance, particularly when data includes noise and outliers, or is incorrectly collected or measured. In this paper, we introduce a Self-Distilled Learning to Rank (SDLR) framework for ad-hoc retrieval, and analyze its performance over a range of retrieval datasets and also in the presence of features’ noise. SDLR assigns a confidence weight to each training sample, aiming at reducing the impact of noisy and outlier data in the training process. The confidence wight is approximated based on the feature’s distributions derived from the values observed for the features of the documents labeled for a query in a listwise training sample. SDLR includes a distillation process that facilitates passing on the underlying patterns in assigning confidence weights from the teacher model to the student one. We empirically illustrate that SDLR outperforms state-of-the-art learning to rank models in ad-hoc retrieval. We thoroughly investigate the SDLR performance in different settings including when no distillation strategy is applied; when different portion of data is used for training the teacher and the student models, and when both teacher and student models are trained over identical data. We show that SDLR is more effective when training data is split between a teacher and a student model. We also show that SDLR’s performance is robust when data features are noisy.

show abstract

Field features: The impact in learning to rank approaches

Cited by 4 publications

References 34 publications

Local Interpretable Model-Agnostic Explanations for Neural Ranking Models

Local Interpretable Model-Agnostic Explanations for Neural Ranking Models

Dual cycle generative adversarial networks for web search

A Self-Distilled Learning to Rank Model for Ad Hoc Retrieval

Contact Info

Product

Resources

About