Click models inspired learning to rank

Keyhanipour, Amir Hosein; Oroumchian, Farhad

doi:10.1108/ijwis-03-2021-0017

Cited by 4 publications

(14 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motivated by the reasonable performance of Keyhanipour and Oroumchian's (2021) findings, this paper extends the previous work by proposing a novel learning to rank (L2R) algorithm using reinforcement learning (RL) methods. This algorithm, called SeaRank, utilizes the calculated Attractiveness and Examination factors for the prediction of the User Satisfaction which is considered here to be proportional to the relevance level.…”

Section: Introductionmentioning

confidence: 86%

“…This brief review of the major CMs shows that these models are built upon common factors such as Attractiveness, Examination and User Satisfaction . The base paper of this research (Keyhanipour and Oroumchian, 2021) has introduced the idea of L2R space transformation using the aforementioned common elements of the CMs. Authors of that paper have applied exponential ordered weighted averaging (OWA) operators in order to aggregate the Attractiveness and Examination values and used this aggregated value to approximate the User Satisfaction , which is an estimator of the actual relevance label.…”

Section: Related Workmentioning

confidence: 99%

“…Compared with the aforementioned works, this paper proposes an extension of Keyhanipour and Oroumchian's (2021) work by applying the RL techniques to the L2R problem through the use of the CMs. The proposed algorithm, SeaRank, accomplishes the feature generation and space reduction using a limited subset of the IR features.…”

Section: Related Workmentioning

confidence: 99%

“…These common elements are Attractiveness, Examination and User Satisfaction . A recent research has shown that by accurate estimation of the Attractiveness and Examination values, it would be possible to approximate the User Satisfaction , which is proportional to the relevance level (Keyhanipour and Oroumchian, 2021). Attractiveness of a document is an indicator of its click rate.…”

Section: Introductionmentioning

confidence: 99%

“…In this way, one may assume that Examination of a document is related to the average dwell time of the users regarding a specific query. Keyhanipour and Oroumchian (2021) have shown that there is a strong relationship between these two factors and that the actual relevance could be estimated by the User Satisfaction .…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

SeaRank: relevance prediction based on click models in a reinforcement learning framework

Keyhanipour

Oroumchian

2022

DTA

Self Cite

View full text Add to dashboard Cite

PurposeUser feedback inferred from the user's search-time behavior could improve the learning to rank (L2R) algorithms. Click models (CMs) present probabilistic frameworks for describing and predicting the user's clicks during search sessions. Most of these CMs are based on common assumptions such as Attractiveness, Examination and User Satisfaction. CMs usually consider the Attractiveness and Examination as pre- and post-estimators of the actual relevance. They also assume that User Satisfaction is a function of the actual relevance. This paper extends the authors' previous work by building a reinforcement learning (RL) model to predict the relevance. The Attractiveness, Examination and User Satisfaction are estimated using a limited number of the features of the utilized benchmark data set and then they are incorporated in the construction of an RL agent. The proposed RL model learns to predict the relevance label of documents with respect to a given query more effectively than the baseline RL models for those data sets.Design/methodology/approachIn this paper, User Satisfaction is used as an indication of the relevance level of a query to a document. User Satisfaction itself is estimated through Attractiveness and Examination, and in turn, Attractiveness and Examination are calculated by the random forest algorithm. In this process, only a small subset of top information retrieval (IR) features are used, which are selected based on their mean average precision and normalized discounted cumulative gain values. Based on the authors' observations, the multiplication of the Attractiveness and Examination values of a given query–document pair closely approximates the User Satisfaction and hence the relevance level. Besides, an RL model is designed in such a way that the current state of the RL agent is determined by discretization of the estimated Attractiveness and Examination values. In this way, each query–document pair would be mapped into a specific state based on its Attractiveness and Examination values. Then, based on the reward function, the RL agent would try to choose an action (relevance label) which maximizes the received reward in its current state. Using temporal difference (TD) learning algorithms, such as Q-learning and SARSA, the learning agent gradually learns to identify an appropriate relevance label in each state. The reward that is used in the RL agent is proportional to the difference between the User Satisfaction and the selected action.FindingsExperimental results on MSLR-WEB10K and WCL2R benchmark data sets demonstrate that the proposed algorithm, named as SeaRank, outperforms baseline algorithms. Improvement is more noticeable in top-ranked results, which usually receive more attention from users.Originality/valueThis research provides a mapping from IR features to the CM features and thereafter utilizes these newly generated features to build an RL model. This RL model is proposed with the definition of the states, actions and reward function. By applying TD learning algorithms, such as the Q-learning and SARSA, within several learning episodes, the RL agent would be able to learn how to choose the most appropriate relevance label for a given pair of query–document.

show abstract

Section: Introductionmentioning

confidence: 86%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations