Learning to Reinforce Search Effectiveness

Luo, Jiyun; Dong, Xuchu; Yang, Hui

doi:10.1145/2808194.2809468

Cited by 7 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work is closely related to question answering, conversational question answering, session search [27,26,56], and weak supervision and data augmentation [24,3]. We highlight the related works on QA and ConvQA as follows.…”

Section: Related Workmentioning

confidence: 99%

Open-Retrieval Conversational Question Answering

Liu

Chen

et al. 2020

Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

118

190

View full text Add to dashboard Cite

Recent studies on Question Answering (QA) and Conversational QA (ConvQA) emphasize the role of retrieval: a system first retrieves evidence from a large collection and then extracts answers. This open-retrieval ConvQA setting typically assumes that each question is answerable by a single span of text within a particular passage (a span answer). The supervision signal is thus derived from whether or not the system can recover an exact match of this ground-truth answer span from the retrieved passages. This method is referred to as spanmatch weak supervision. However, information-seeking conversations are challenging for this span-match method since long answers, especially freeform answers, are not necessarily strict spans of any passage. Therefore, we introduce a learned weak supervision approach that can identify a paraphrased span of the known answer in a passage. Our experiments on QuAC and CoQA datasets show that the span-match weak supervisor can only handle conversations with span answers, and has less satisfactory results for freeform answers generated by people. Our method is more flexible as it can handle both span answers and freeform answers. Moreover, our method can be more powerful when combined with the span-match method which shows it is complementary to the span-match method. We also conduct in-depth analyses to show more insights on open-retrieval ConvQA under a weak supervision setting.

show abstract

Section: Related Workmentioning

confidence: 99%

Open-Retrieval Conversational Question Answering

Liu

Chen

et al. 2020

Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

118

190

View full text Add to dashboard Cite

show abstract

“…Hofmann et al [12,14] seem to have been the first; they use RL for online evaluation and online learning to rank and define reward functions directly in terms of NDCG [13]. Later work on RL in IR predefined reward functions as the number of satisfied clicks in session search [22,23]. Odijk et al [26] use RL for query modeling and define reward in terms of retrieval performance (NDCG).…”

Section: Background and Related Workmentioning

confidence: 99%

Towards Learning Reward Functions from User Interactions

Kiseleva

Rijke

et al. 2017

Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval

View full text Add to dashboard Cite

In the physical world, people have dynamic preferences, e.g., the same situation can lead to satisfaction for some humans and to frustration for others. Personalization is called for. The same observation holds for online behavior with interactive systems. It is natural to represent the behavior of users who are engaging with interactive systems such as a search engine or a recommender system, as a sequence of actions where each next action depends on the current situation and the user reward of taking a particular action. By and large, current online evaluation metrics for interactive systems such as search engines or recommender systems, are static and do not reflect differences in user behavior. They rarely capture or model the reward experienced by a user while interacting with an interactive system. We argue that knowing a user's reward function is essential for an interactive system as both for learning and evaluation. We propose to learn users' reward functions directly from observed interaction traces. In particular, we present how users' reward functions can be uncovered directly using inverse reinforcement learning techniques. We also show how to incorporate user features into the learning process. Our main contribution is a novel and dynamic approach to restore a user's reward function. We present an analytic approach to this problem and complement it with initial experiments using the interaction logs of a cultural heritage institution that demonstrate the feasibility of the approach by uncovering different reward functions for different user groups. CCS CONCEPTS• Information systems → Users and interactive retrieval; • Computing methodologies → Inverse reinforcement learning; Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

show abstract

“…Session search has obtained increasing attentions in the IR community, for instance, in the recent dynamic information retrieval modeling approaches [164,165,261,262] and in the TExt Retrieval Conference (TREC) 2010-2014 Session Tracks [132,133]. e current approaches include (1) extending existing IR techniques, such as learning to rank and using large scale query logs, from ad-hoc retrieval for one-shot query to session search, and (2) emerging efforts in applying reinforcement learning (RL) to session search.…”

Section: Session Searchmentioning

confidence: 99%

“…Session search has been viewed as a cooperative game played between the user and the search engine in recent work [164,169]. e interactions between the two agents show the following characteristics:…”

Section: Two-way Communication In Sessionsmentioning

confidence: 99%

Dynamic Information Retrieval Modeling

HuiYang

MarcSloan

JunWang

2016

Synthesis Lectures on Information Concepts, Retrieval, and Serv

View full text Add to dashboard Cite

Synthesis Lectures on Information Concepts, Retrieval, and Services publishes short books on topics pertaining to information science and applications of technology to information discovery, production, distribution, and management. Potential topics include: data models, indexing theory and algorithms, classification, information architecture, information economics, privacy and identity, scholarly communication, bibliometrics and webometrics, personal information management, human information behavior, digital libraries, archives and preservation, cultural informatics, information retrieval evaluation, data fusion, relevance feedback, recommendation systems, question answering, natural language processing for retrieval, text summarization, multimedia retrieval, multilingual retrieval, and exploratory search. ABSTRACTBig data and human-computer information retrieval (HCIR) are changing IR. ey capture the dynamic changes in the data and dynamic interactions of users with IR systems. A dynamic system is one which changes or adapts over time or a sequence of events. Many modern IR systems and data exhibit these characteristics which are largely ignored by conventional techniques. What is missing is an ability for the model to change over time and be responsive to stimulus. Documents, relevance, users and tasks all exhibit dynamic behavior that is captured in data sets typically collected over long time spans and models need to respond to these changes. Additionally, the size of modern datasets enforces limits on the amount of learning a system can achieve. Further to this, advances in IR interface, personalization and ad display demand models that can react to users in real time and in an intelligent, contextual way.In this book we provide a comprehensive and up-to-date introduction to Dynamic Information Retrieval Modeling, the statistical modeling of IR systems that can adapt to change. We define dynamics, what it means within the context of IR and highlight examples of problems where dynamics play an important role. We cover techniques ranging from classic relevance feedback to the latest applications of partially observable Markov decision processes (POMDPs) and a handful of useful algorithms and tools for solving IR problems incorporating dynamics.e theoretical component is based around the Markov Decision Process (MDP), a mathematical framework taken from the field of Artificial Intelligence (AI) that enables us to construct models that change according to sequential inputs. We define the framework and the algorithms commonly used to optimize over it and generalize it to the case where the inputs aren't reliable. We explore the topic of reinforcement learning more broadly and introduce another tool known as a Multi-Armed Bandit which is useful for cases where exploring model parameters is beneficial. Following this we introduce theories and algorithms which can be used to incorporate dynamics into an IR model before presenting an array of state-of-the-art research that already does, such as in the areas of se...

show abstract

Learning to Reinforce Search Effectiveness

Cited by 7 publications

References 17 publications

Open-Retrieval Conversational Question Answering

Open-Retrieval Conversational Question Answering

Towards Learning Reward Functions from User Interactions

Dynamic Information Retrieval Modeling

Contact Info

Product

Resources

About