Capturing User Intent for Information Retrieval

Abstract

We study the problem of employing a cognitive user model for information retrieval in which knowledge about a user is captured and used for improving retrieval performance and user satisfaction. In this proposed research, we improve retrieval performance and user satisfaction for information retrieval by building a user model to capture user intent dynamically through analyzing behavioral information from retrieved relevant documents, and by combining captured user intent with the elements of an information retrieval system. We use decision theoretic principles and bayesian networks for building this model. The novelties of our approach lie with the fine-grained representation of the model, the ability to learn user knowledge incrementally and dynamically, the integration of user intent and system elements for improving retrieval performance and the unified evaluation framework to assess the accuracy of user intent captured and effectiveness of our model. Problem StatementWe study the problem of employing a cognitive user model for information retrieval in which knowledge about a user is captured and used for improving retrieval performance and user satisfaction. The problem of employing a user model for information retrieval has been investigated since the late 80s (Brajnik, Guida, & Tasso 1987) to address the difficulty of traditional information retrieval which is to satisfy a user's information needs by retrieving information with good quality quickly. It has many applications from information filtering, and text recommendation systems (e.g (Balabanovic 1998;Billsus & Pazzani 2000)). This problem is wellknown for the challenges it poses to Information Retrieval (IR), and User Modeling (UM) communities. We have identified four primary challenges of this problem which are (i) non-observability of user knowledge (ii) uncertainty in modeling user interactions (ii) vagueness of a user's information needs (iv) and dynamics of user knowledge which changes over time as a result of new information, and temporal factors. These challenges come from the main problem of modeling a user in information seeking. Unfortunately, traditional IR does not offer a way to overcome these challenges because its framework supported very little users' involvement (Saracevic 1996). The thesis of this proposed research is to improve retrieval performance and user satisfaction for IR by building a user model to capture user intent dynamically through analyzing behavioral information of retrieved relevant documents, and combine it with the elements of an IR system. Our methodologies for this research are (i) to combine system-centered approaches and user-centered approaches by taking advantage of well-established evaluation frameworks in IR, and using the strength of knowledge representation techniques in artificial intelligence (AI) to build this model. As pointed out in (Saracevic, Spink, & Wu 1997), so far there has been a little crossover between IR and AI communities with regards to building user models for IR; and (ii) to cr...

Layered evaluation of interactive adaptive systems: framework and formative methods

“…Since the first introduction of the term in 2000, the scientific community has adopted this concept in planning and conducting empirical studies. Many authors explicitly refer back to the foundational papers published on the topic to justify experimental designs, to provide rationale for goals or structure of their evaluation studies (Arruabarrena et al 2002;Ortigosa and Carro 2003;Petrelli and Not 2005;Cena et al 2006;Goren-Bar et al 2006;Glahn et al 2007;Kosba et al 2007;Nguyen and Santos Jr 2007;Carmagnola et al 2008;Limongelli et al 2008;Ley et al 2009;Popescu 2009;Santos and Boticario 2009), or to demonstrate methodological shortcomings of existing studies (Masthoff 2002;Gena 2005;Brusilovsky et al 2006;Yang and Huo 2008;Brown et al 2009). The fact that layered evaluation received such a high level of attention in the literature reaffirms the claim that the evaluation of adaptive systems implicates some inherent difficulties.…”

Effects of prior knowledge on the effectiveness of a hybrid user model for information retrieval

“…If the total number of retrieved relevant documents exceeds a user-defined threshold, a tool is considered helpful. For more information, please see 14,13 .…”

“…The Interests refers to what users are doing to achieve their goals, the Preferences captures how the users may go about achieving their goals, and the Context infers why they are trying to achieve these goals. We capture the Context, the Interests, and the Preferences aspects of a user's model with a context network (C), an interest set (I), and a preference network (P) 17,15,14,13 .…”

“…One of the misleading impressions regarding the differences between novice and experienced users is that the more experienced users would have more domain knowledge which leads to more successful retrieval tasks. Many intelligent information retrieval applications have been trying to use and capture a user's domain knowledge to help him/her to retrieve more relevant documents quickly (for example: 2,21,8,17,15 ). However, while there are many research projects focused on how to construct and use domain knowledge in an information retrieval application, very little effort has been paid to quantitatively evaluating how the captured user's domain knowledge affects his/her effectiveness.…”

A large-scale distributed framework for information retrieval in large dynamic search spaces

“…The DG representation has been successfully used in the development of a cognitive user modeling technology to enhance the performance of an IR system [24][25][26]. This cognitive user modeling technology has been shown to help the target IR system to retrieve more relevant documents earlier when compared to the traditional vector space model approach in evaluations with testbed collections found in the IR community such as CRANFIELD, CACM, and MEDLINE [18][19][20]. More importantly, it also helps human intelligence analysts to retrieve more uniquely relevant documents as compared to keyword-based retrieval systems as demonstrated in an experimental evaluation with human analysts at the National Institute of Standard and Technology [30].…”