Building Cost-Benefit Models of Information Interactions

Abstract

Modeling how people interact with search interfaces has been of particular interest and importance to the field of Interactive Information Retrieval. Recently, there has been a move to developing formal models of the interaction between the user and the system, whether it be to: (i) run a simulation, (ii) conduct an economic analysis, (iii) measure system performance, or (iv) simply to better understand the interactions and hypothesise about user behaviours. In such models, they consider the costs and the benefits that arise through the interaction with the interface/system and the information surfaced during the course of interaction. In this full day tutorial, we will focus on describing a series of cost (and benefit) models that have been proposed in the literature and how they have been applied in various scenarios. The tutorial will be structured into two parts. First, we will provide an overview of Decision Theory and Cost-Benefit Analysis techniques, and how they can and have be applied to a variety of Interactive Information Retrieval scenarios. For example, how do we reduce the cost browsing facets? how many results should we show per page? and how do we get users to pose longer queries? The second part of the day will be dedicated to building cost models where we will discuss different techniques to build and develop models from which one can draw testable hypotheses from. In this more practical session, we will also discuss how costs and benefits can be estimated, and how the models can help inform and guide experimentation. During the tutorial participants will be challenged to build cost models for a number of problems (or even bring their own problems to solve). We will then conclude with presentations of solutions followed by a summary and overview of challenges and future directions. This tutorial is aimed at participants wanting to know more about cost models and how they can be applied, interpreted and built in the context of information seeking, search and retrieval.

Information Foraging for Enhancing Implicit Feedback in Content-based Image Recommendation

“…We report the classification result in Table 1. We found that the F1-score of the "uninterested" collection of images is significantly higher than the "interested" list of items which is due to the diminishing return of information scent score (in line with [1]). It means that the user selected item in first glance put less effort (selection cost) and those followed several iteration on user preferences to select an image results in weak information scent (and ranked lower in the list of interesting items).…”

“…In Information Foraging Theory, the flux of information in an information environment perceives user plans for seeking, gathering and consumption based on their utility [16]. The view of animals 1 A board allows people to place all of their visual cues around diverse interests, ideas and plans 2 Foraging phenomena in this work refers to Information Foraging Theory following scents to forage is analogous to users following various kinds of cues, assessing information contents and navigating across information spaces depending on their information scent. Cues cater as proximal cues in order to emit information scent.…”

Bibliometric‐enhanced legal information retrieval: Combining usage and citations as flavors of impact relevance

“…To determine whether the boost function achieves this aim we use a cost model inspired by Järvelin (2009) and compare the cost before the introduction of this variable (the intervention) with the cost after the intervention. Azzopardi (2017) and Azzopardi and Zuccon (2016) have used such cost based models to determine the effectiveness of changes to the user interface. This model will be limited to cost without gain, as there are no relevance judgments to base gain on.…”

Building Economic Models of Human Computer Interaction

“…Leif and Guido have presented related courses at ACM SIGIR 2015 [5], ACM CIKM 2016, ACM ICTIR 2016 [6] and ACM CHIIR 2017 [4]. They have also recently contributed a chapter on building economic models of interaction to the book "Computational Interaction" published in 2018 [8] which this course draws upon.…”