A POMDP model for content-free document re-ranking

Zhang, Sicong; Luo, Jiyun; Yang, Hui

doi:10.1145/2600428.2609529

Cited by 18 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As ranking is a key issue in practical recommendation problems, any improvements in ranking contribute significantly to reinforcement recommendation systems. Zhang et al [ 30 ] used a log-based document reranking modeled as a POMDP. Wei et al [ 6 ] proposed a novel LTR model based on a MDP, referred to as MDPRank, which directly optimizes a ranking using a MDP.…”

Section: Related Workmentioning

confidence: 99%

A Knowledge-Fusion Ranking System with an Attention Network for Making Assignment Recommendations

Jin

Zhou

Ying

et al. 2020

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

In recent decades, more teachers are using question generators to provide students with online homework. Learning-to-rank (LTR) methods can partially rank questions to address the needs of individual students and reduce their study burden. Unfortunately, ranking questions for students is not trivial because of three main challenges: (1) discovering students’ latent knowledge and cognitive level is difficult, (2) the content of quizzes can be totally different but the knowledge points of these quizzes may be inherently related, and (3) ranking models based on supervised, semisupervised, or reinforcement learning focus on the current assignment without considering past performance. In this work, we propose KFRank, a knowledge-fusion ranking model based on reinforcement learning, which considers both a student’s assignment history and the relevance of quizzes with their knowledge points. First, we load students’ assignment history, reorganize it using knowledge points, and calculate the effective features for ranking in terms of the relation between a student’s knowledge cognitive and the question. Then, a similarity estimator is built to choose historical questions, and an attention neural network is used to calculate the attention value and update the current study state with knowledge fusion. Finally, a rank algorithm based on a Markov decision process is used to optimize the parameters. Extensive experiments were conducted on a real-life dataset spanning a year and we compared our model with the state-of-the-art ranking models (e.g., ListNET and LambdaMART) and reinforcement-learning methods (such as MDPRank). Based on top- k nDCG values, our model outperforms other methods for groups of average and weak students, whose study abilities are relatively poor and thus their behaviors are more difficult to predict.

show abstract

Section: Related Workmentioning

confidence: 99%

A Knowledge-Fusion Ranking System with an Attention Network for Making Assignment Recommendations

Jin

Zhou

Ying

et al. 2020

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Similar to our modeling approach, multi-armed bandits have been utilized to model user preferences by learning diverse rankings for a single query based on clicking behavior [56,88] and learning rankings from pair-wise document comparisons derived from implicit feedback [52,126]. Other similar techniques include POMDPs that have been recently proposed for re-ranking [128] and session search [74].…”

Section: Complex Tasks and Search Personalizationmentioning

confidence: 99%

Interactive Intent Modeling for Exploratory Search

Ruotsalo

Peltonen

Eugster

et al. 2018

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

Exploratory search requires the system to assist the user in comprehending the information space and expressing evolving search intents for iterative exploration and retrieval of information. We introduce interactive intent modeling, a technique that models a user's evolving search intents and visualizes them as keywords for interaction. The user can provide feedback on the keywords, from which the system learns and visualizes an improved intent estimate and retrieves information. We report experiments comparing variants of a system implementing interactive intent modeling to a control system. Data comprising search logs, interaction logs, essay answers, and questionnaires indicate significant improvements in task performance, information retrieval performance over the session, information comprehension performance, and user experience. The improvements in retrieval effectiveness can be attributed to the intent modeling and the effect on users' task performance, breadth of information comprehension, and user experience are shown to be dependent on a richer visualization. Our results demonstrate the utility of combining interactive modeling of search intentions with interactive visualization of the models that can benefit both directing the exploratory search process and making sense of the information space. Our findings can help design personalized systems that support exploratory information seeking and discovery of novel information.

show abstract

“…One line is session search. In [11,20], a collaborative search process between the user and the search engine is defined as an MDP. The other line mainly focuses on reinforcement learning to rank.…”

Section: Background 21 Related Workmentioning

confidence: 99%

MarlRank

Zou

Akbari

et al. 2019

Proceedings of the 28th ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

When estimating the relevancy between a query and a document, ranking models largely neglect the mutual information among documents. A common wisdom is that if two documents are similar in terms of the same query, they are more likely to have similar relevance score. To mitigate this problem, in this paper, we propose a multi-agent reinforced ranking model, named MarlRank. In particular, by considering each document as an agent, we formulate the ranking process as a multi-agent Markov Decision Process (MDP), where the mutual interactions among documents are incorporated in the ranking process. To compute the ranking list, each document predicts its relevance to a query considering not only its own query-document features but also its similar documents' features and actions. By defining reward as a function of NDCG, we can optimize our model directly on the ranking performance measure. Our experimental results on two LETOR benchmark datasets show that our model has significant performance gains over the state-of-art baselines. We also find that the NDCG shows an overall increasing trend along with the step of interactions, which demonstrates that the mutual information among documents helps improve the ranking performance. CCS CONCEPTS• Information systems → Learning to rank; Novelty in information retrieval.

show abstract

A POMDP model for content-free document re-ranking

Cited by 18 publications

References 11 publications

A Knowledge-Fusion Ranking System with an Attention Network for Making Assignment Recommendations

A Knowledge-Fusion Ranking System with an Attention Network for Making Assignment Recommendations

Interactive Intent Modeling for Exploratory Search

MarlRank

Contact Info

Product

Resources

About