Untangling Cost, Effort, and Load in Information Seeking and Retrieval

Abstract

When performing Information Seeking and Retrieval (ISR) activities, people submit queries, examine results, assess documents and engage with the information to make decisions and complete tasks. All these activities come at a "cost", but within the field of ISR there is no universally accepted definition of the concepts of Cost, Effort and Load (CEL). Instead, researchers have used the same terms interchangeably to describe similar but also different concepts. This lack of shared understanding has led to a disconnect between how these concepts are defined and discussed versus how they are interpreted and measured. Thus, the aim of this paper is twofold: (i) to review the meaning of CEL related concepts used within ISR, and (ii) to create a shared taxonomy of the concepts relating to CEL in ISR. To seed our analysis, we conducted a literature review, where 397 papers were reviewed, and twenty-six papers that explicitly proposed measures or definitions of CEL were selected for analysis. By drawing upon theory from Psychology and other fields, we present the common definitions of CEL in order to ground our discussion of these concepts in ISR. We also highlight the issues associated with CEL measurement in ISR to help researchers reflect on the validity and precision of existing methods. We hope this perspectives paper serves as a basis for a taxonomy of how CEL concepts are used within ISR-where we have provided a series of working definitions that clearly delineate the different concepts being used, investigated and measured in ISR research.

Bibliometric‐enhanced legal information retrieval: Combining usage and citations as flavors of impact relevance

“…To calculate the cost related to this assumed gain, we use a time‐oriented metric (McGregor et al, 2021). Because of the time pressure legal professionals work under this appears to be the most suitable representation of cost or effort.…”

“…This sum of costs is then offset to the gains of the relevant documents found. McGregor et al (2021) differentiate between load, effort, and cost. Load is taken to refer to the total amount of resources used to complete the task, internally and externally.…”

A narrative review of immersive virtual reality’s ergonomics and risks at the workplace: cybersickness, visual fatigue, muscular fatigue, acute stress, and mental overload

“…Those results seem moderated by expertise within VR and the task demands (Aksoy et al 2019;Luong et al 2019;Armougum et al 2019). For instance, outside of VR, when time and load on the resources are high, humans hit the maximum resource allocation capacity (McGregor et al 2021).…”

A Systematic Review of Cost, Effort, and Load Research in Information Search and Retrieval, 1972–2020

“…The demands imposed during the search process are well recognised, and numerous studies have demonstrated that the cost, effort, and load (CEL) experienced during the search process is affected by not only the search task [27,91,120], but also the system [21,23,37] and the individual characteristics [24,57] of the user. Despite this recognition, our recent perspectives paper [97] highlighted two prominent challenges facing the ISR community in regard to CEL research. The first challenge relates to the general lack of consensus regarding cost, effort, and load definition, which has ultimately left the field with no universal definitions to describe these constructs.…”