Personalised Search Time Prediction using Markov Chains

Tran, Vu Tuan; Maxwell, David; Fuhr, Norbert; Azzopardi, Leif

doi:10.1145/3121050.3121085

Cited by 9 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the dynamic nature of task-based search interactions, IIR researchers have explored several ways to characterize the sequence and transitions of users' search phases and tactics. One widelyused approach is to simplify the process by making it memoryless (assuming that the current state is dependent only on its previous state) [9,27,41]). In IIR context, this assumption is reasonable in the sense that users' often decide their search tactics in local, small steps (i.e.…”

Section: Adaptive Search Recommendationsmentioning

confidence: 99%

Leveraging user interaction signals and task state information in adaptively optimizing usefulness-oriented search sessions

Liu

Shah

2022

Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries

View full text Add to dashboard Cite

Current information retrieval (IR) systems still face plenty of challenges when applied in addressing complex search tasks (CSTs) that trigger multi-round search iterations. Existing relevance-oriented optimization algorithms and metrics are limited in helping users find documents that are useful for completing CSTs, rather than merely topically relevant. To address this gap, our work aimed to characterize CSTs from a process-oriented perspective and develop a state-based adaptive approach to simulating and evaluating search path recommendations. Based on the data collected from 80 journalism search sessions, we first extracted intention-based task states from participants' annotations to characterize temporal their temporal cognitive changes in searching and validated the state labels with expert assessments. Built upon the state labels and state distribution patterns, we then developed a simulated adaptive search path recommendation approach, aiming to help users find needed useful documents quicker. The results demonstrate that 1) different types of CSTs can be differentiated based on their distinct state distribution and transition patterns; 2) After a small number of iterative training, our adaptive recommendation algorithm can consistently outperform the best possible performance from individual participants in terms of the useful-based search efficiency across all CSTs. Going beyond traditional static viewpoint of task facets and relevance-focused evaluation approach, our work characterizes CSTs with a dynamic perspective and develops a domain-specific adaptive search algorithm that can help users find useful documents quicker and learn from online search logs. Our findings can facilitate future exploration of adaptive search path adjustments for similar types of CSTs in other domains and work task scenarios. CCS CONCEPTS• Information systems → Users and interactive retrieval.

show abstract

Section: Adaptive Search Recommendationsmentioning

confidence: 99%

Leveraging user interaction signals and task state information in adaptively optimizing usefulness-oriented search sessions

Liu

Shah

2022

Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries

View full text Add to dashboard Cite

show abstract

“…For example, Smucker and Clarke (2012) introduced time-biased gain, where probabilities of interacting with documents were included. Work by Tran et al (2017) considered the TREC searcher model from the standpoint of a Markov model. As such, this representation of the search process was complemented with a series of different transition probabilities, dictating the likelihood of a searcher switching from one state to another.…”

Section: From System To Searchermentioning

confidence: 99%

“…Two lists of rankings with one highly rel- Figure 2.16, assumes that a searcher will always examine the first result presented to him or her. The process can be likened to the previously discussed idea of using a Markov model to represent the search process (Tran et al, 2017), modelling the likelihood of a searcher reaching a given depth. Subsequent documents further down the ranking will then be examined with a decreasing likelihood.…”

Section: Cumulative Gain Measuresmentioning

confidence: 99%

Modelling search and stopping in interactive information retrieval

Maxwell

2019

SIGIR Forum

Self Cite

View full text Add to dashboard Cite

Searching for information when using a computerised retrieval system is a complex and inherently interactive process. Individuals during a search session may issue multiple queries, and examine a varying number of result summaries and documents per query. Searchers must also decide when to stop assessing content for relevance - or decide when to stop their search session altogether. Despite being such a fundamental activity, only a limited number of studies have explored stopping behaviours in detail, with a majority reporting that searchers stop because they decide that what they have found feels " good enough ". Notwithstanding the limited exploration of stopping during search, the phenomenon is central to the study of Information Retrieval, playing a role in the models and measures that we employ. However, the current de facto assumption considers that searchers will examine k documents - examining up to a fixed depth. In this thesis, we examine searcher stopping behaviours under a number of different search contexts. We conduct and report on two user studies, examining how result summary lengths and a variation of search tasks and goals affect such behaviours. Interaction data from these studies are then used to ground extensive simulations of interaction , exploring a number of different stopping heuristics (operationalised as twelve stopping strategies). We consider how well the proposed strategies perform and match up with real-world stopping behaviours. As part of our contribution, we also propose the Complex Searcher Model , a high-level conceptual searcher model that encodes stopping behaviours at different points throughout the search process (see Figure 1 below). Within the Complex Searcher Model, we also propose a new results page stopping decision point. From this new stopping decision point, searchers can obtain an impression of the page before deciding to enter or abandon it. Results presented and discussed demonstrate that searchers employ a range of different stopping strategies, with no strategy standing out in terms of performance and approximations offered. Stopping behaviours are clearly not fixed, but are rather adaptive in nature. This complex picture reinforces the idea that modelling stopping behaviour is difficult. However, simplistic stopping strategies do offer good performance and approximations, such as the frustration -based stopping strategy. This strategy considers a searcher's tolerance to non-relevance. We also find that combination strategies - such as those combining a searcher's satisfaction with finding relevant material, and their frustration towards observing non-relevant material - also consistently offer good approximations and performance. In addition, we also demonstrate that the inclusion of the additional stopping decision point within the Complex Searcher Model provides significant improvements to performance over our baseline implementation. It also offers improvements to the approximations of real-world searcher stopping behaviours. This work motivates a revision of how we currently model the search process and demonstrates that different stopping heuristics need to be considered within the models and measures that we use in Information Retrieval. Measures should be reformed according to the stopping behaviours of searchers. A number of potential avenues for future exploration can also be considered, such as modelling the stopping behaviours of searchers individually (rather than as a population), and to explore and consider a wider variety of different stopping heuristics under different search contexts. Despite the inherently difficult task that understanding and modelling the stopping behaviours of searchers represents, potential benefits of further exploration in this area will undoubtedly aid the searchers of future retrieval systems - with further work bringing about improved interfaces and experiences. Doctoral Supervisor Dr Leif Azzopardi (University of Strathclyde, Scotland) Examination Committee Professor Iadh Ounis (University of Glasgow, Scotland) and Dr Suzan Verberne (Leiden University, The Netherlands). Thanks to both of you for your insightful and fair questioning during the defence! Availability This thesis is available to download from http://www.dmax.org.uk/thesis/, or the University of Glasgow's Enlighten repository - see http://theses.gla.ac.uk/41132/. A Quick Thank You Five years of hard work has got me to the point at which I can now submit the abstract of my doctoral thesis to the SIGIR Forum. There have been plenty of ups and downs, but I'm super pleased with the result! Even though there is only a single name on the front cover of this thesis, there are many people who have helped me get to where I am today. You all know who you are - from my friends and family, those who granted me so many fantastic opportunities to travel and see the world - and of course, to Leif. Thanks to all of you for confiding your belief and trust in me, even when I may have momentarily lost that belief and trust in myself. This thesis is for you all.

show abstract

“…In Digital Libraries (DLs), IIR researchers study concepts such as search strategies [28,7,25,12], search term suggestions [6,38,19,29], user modelling [41,16], communities' detection [2], personalisation of search results [35,26], recommendation's impact [19], user's information needs change [42] and many more topics. In addition, many interactive IR models have been proposed in the literature (e.g.…”

Section: Introductionmentioning

confidence: 99%

Analysis of search stratagem utilisation

Kacem

Mayr

2018

Scientometrics

View full text Add to dashboard Cite

In Interactive Information Retrieval, researchers consider the user behaviour towards systems and search tasks in order to adapt search results and to improve the search experience of users. Analysing the users' past interactions with the system is one typical approach. In this paper, we analyse the user behaviour in retrieval sessions towards Marcia Bates' search stratagems such as "Footnote Chasing", "Citation Searching", "Keyword Searching", "Author Searching" and "Journal Run" in a real-life academic search engine. In fact, search stratagems represent high-level search behaviour as the users go beyond simple execution of queries and investigate more of the system functionalities. We performed analyses of these five search stratagems using two datasets extracted from the social sciences search engine sowiport. A specific focus was the detection of the search phase and frequency of the usage of these stratagems. In addition, we explored the impact of these stratagems on the whole search process performance. We addressed mainly the usage patterns' observation of the stratagems, their impact on the conduct of retrieval sessions and explore whether they are used similarly in both datasets. From the observation and metrics proposed, we can conclude that the utilisation of search stratagems in real retrieval sessions leads to an improvement of the precision in terms of positive interactions. For both datasets (SUSS 14-15 and

show abstract

Personalised Search Time Prediction using Markov Chains

Cited by 9 publications

References 15 publications

Leveraging user interaction signals and task state information in adaptively optimizing usefulness-oriented search sessions

Leveraging user interaction signals and task state information in adaptively optimizing usefulness-oriented search sessions

Modelling search and stopping in interactive information retrieval

Analysis of search stratagem utilisation

Contact Info

Product

Resources

About