Understanding User Behavior in Digital Libraries Using the MAGUS Session Visualization Tool

Bogaard, Tessel; Wielemaker, Jan; Hollink, Laura; Hardman, Lynda; Ossenbruggen, Jacco van

doi:10.1007/978-3-030-54956-5_13

Cited by 2 publications

(4 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As this is a publicly available knowledge resource, with many possible applications in different domains for different topics, it contributes to the general applicability of our methods. Our topics of interest correspond to the existing Wikipedia categories for WWII 6 and for feminism 7 . We have selected the top-300 Wikipedia articles in these categories, based on the popularity within the same period as the logs (October 2015-March 2016).…”

Section: Knowledge Sources For Topic Representationsmentioning

confidence: 99%

“…The raters judge whether one of the information needs of the user in that session is to find newspaper documents about a topic that is directly related to the topic of interest. To do this, the rater can inspect the session, using a visualization that includes the search interactions with the queries and selected facets [6], and the clicked documents and their metadata and content. We use inter-rater reliability to check the agreement among the raters.…”

Section: Manually Evaluating Retrieved Sessionsmentioning

confidence: 99%

See 1 more Smart Citation

Comparing Methods for Finding Search Sessions on a Specified Topic: A Double Case Study

Bogaard

Bilgin

Wielemaker

et al. 2021

Linking Theory and Practice of Digital Libraries

Self Cite

View full text Add to dashboard Cite

Users searching for different topics in a collection may show distinct search patterns. To analyze search behavior of users searching for a specific topic, we need to retrieve the sessions containing this topic. In this paper, we compare different topic representations and approaches to find topic-specific sessions. We conduct our research in a double case study of two topics, World War II and feminism, using search logs of a historical newspaper collection. We evaluate the results using manually created ground truths of over 600 sessions per topic. The two case studies show similar results: The query-based methods yield high precision, at the expense of recall. The document-based methods find more sessions, at the expense of precision. In both approaches, precision improves significantly by manually curating the topic representations. This study demonstrates how different methods to find sessions containing specific topics can be applied by digital humanities scholars and practitioners.

show abstract

Section: Knowledge Sources For Topic Representationsmentioning

confidence: 99%

Section: Manually Evaluating Retrieved Sessionsmentioning

confidence: 99%

Comparing Methods for Finding Search Sessions on a Specified Topic: A Double Case Study

Bogaard

Bilgin

Wielemaker

et al. 2021

Linking Theory and Practice of Digital Libraries

Self Cite

View full text Add to dashboard Cite

show abstract

“…Writing contributions and feedback on the text were provided by all authors, especially on the parts they contributed to. chapter 5 is previously published as Understanding User Behavior in Digital Libraries Using the MAGUS Session Visualization Tool presented at the 24th International Conference on Theory and Practice of Digital Libraries, TPDL 2020, Lyon, France, August 25-27, 2020, by Tessel Bogaard, Jan Wielemaker, Laura Hollink, Lynda Hardman, and Jacco van Ossenbruggen [14]. All authors contributed to the text.…”

Section: Publicationsmentioning

confidence: 99%

“…The raters judge whether one of the information needs of the user in that session is to find newspaper documents about a topic that is directly related to the topic of interest. To do this, the rater can inspect the session, using a visualization that includes the search interactions with the queries and selected facets [14], and the clicked documents and their metadata and content. We use inter-rater reliability to check the agreement among the raters.…”

Section: Manually Evaluating Retrieved Sessionsmentioning

confidence: 99%

Using Metadata to Understand Search Behavior in Digital Libraries

Bogaard¹

View full text Add to dashboard Cite

This thesis explores how search log analysis can be used to gain a deeper understanding of online search behavior in curated collections by leveraging the metadata. For this, we use both the metadata of facets selected in the search interface and the metadata of the clicked documents. Our research is conducted using data from the National Library of the Netherlands, a typical digital library with a richly annotated historical newspaper collection and a faceted search interface. We investigate how to leverage metadata in three analytical settings. First, we analyze the logs using specific metadata values, to study search within specified parts of the collection. The analysis shows that faceted search is common, we observe distinct search patterns in different parts of the collection, and are able to formulate concrete suggestions for improvement of the search system and collection management. This shows how metadata can be used to analyze search behavior in specific parts of a collection. Second, we uncover user interests by clustering over the combined metadata values. We apply a clustering algorithm grouping sessions based on the metadata of selected facets and clicked documents. To evaluate resulting clusters, their stability over a six-month period is measured. The results show that user interests are stable, with the same interests reoccurring and the related search behavior varies per cluster. This demonstrates that a partitioning of sessions based on metadata, and an investigation of the related search behavior reveals specific user needs in specific parts of a collection, where in an overall analysis these patterns would disappear. Third, we explore how to identify search for specific topics when no metadata directly describes these. We look into different, consecutive ways to build a term list as a topic representation: (i) using a knowledge resource, (ii) using local word embeddings trained on the collection, and (iii) by manual curation. Then we look into how to match the different term lists to search sessions: matching the terms to a) user queries, or b) clicked documents. We investigate two topics of societal relevance, WWII and feminism, and compare and discuss the combined methods in terms of number of retrieved sessions as well as estimated precision scores computed using manually created ground truths. With this work we provide insights into how different topic representations and matching approaches perform when retrieving topic-specific sessions. Finally, we examine how to communicate results of such an analysis to collection owners and domain experts. We introduce MAGUS, a session visualization tool combining graphs to visualize search behavior with colors to visualize relevant metadata. Our design is new in combining both search interactions and metadata in a single visualization, allowing researchers and professionals to recognize different interaction patterns while at the same time providing insights into the parts of the collection a user is interested in. For the evaluation we conduct a user study comparing MAGUS with a table representation in three tasks completed by 12 participants from diverse backgrounds. Our study demonstrates that MAGUS enables participants to identify the part of the collection a user is interested in, and that it helps to distinguish different types of search behavior. We expect that the presented methods can be used for other collections. Vertical search engines are ubiquitous and the search interfaces providing access to these systems are complex. We show how leveraging metadata in a search log analysis can enhance our understanding of how users are searching within different parts of a digital library. We were able to provide collection owners with recommendations about how to improve access to the collection.

show abstract

Understanding User Behavior in Digital Libraries Using the MAGUS Session Visualization Tool

Cited by 2 publications

References 19 publications

Comparing Methods for Finding Search Sessions on a Specified Topic: A Double Case Study

Comparing Methods for Finding Search Sessions on a Specified Topic: A Double Case Study

Using Metadata to Understand Search Behavior in Digital Libraries

Contact Info

Product

Resources

About