Evaluating hierarchical organisation structures for exploring digital libraries

Abstract

Search boxes providing simple keyword-based search are insufficient when users have complex information needs or are unfamiliar with a collection, for example in large digital libraries. Browsing hierarchies can support these richer interactions, but many collections do not have a suitable hierarchy available. In this paper we present a number of approaches for automatically creating hierarchies and mapping items into them, including a novel technique which automatically adapts a Wikipedia-based taxonomy to the target collection. These approaches are applied to a large collection of cultural heritage items which is formed through the aggregation of other collections and for which no unified hierarchy is available. We investigate a number of novel user-evaluated metrics to quantify the hierarchies' quality and performance, showing that the proposed technique is preferred by users. From this we draw a number of conclusions as to what makes a hierarchy useful to the user.

Overview of the CLEF 2016 Social Book Search Lab

“…The main feature to support the free exploration is the hierarchy browsing component on the left, which shows a hierarchical tree of Amazon subject classifications. This was generated using the algorithm described in (Hall et al, 2014a), which uses the relative frequencies of the subjects to arrange them into the tree-structure with the most-frequent subjects at the top of the tree. The search result list is designed to be more compact to allow the user to browse books quickly and shows only the book's title, thumbnail image, and aggregate ratings (if available).…”

“Waiting for Carnot”: Information and complexity

“…The idea that complexity is associated with repeated division into subsystems was introduced by Simon's famous paper of 1962, “The Architecture of Complexity.” Ellis () and Mitchell () note, with examples, that many others have explored the idea of hierarchy as a way of quantifying complexity. Hierarchy is, of course, a central concept in several areas of library and information science (LIS), such as classification, resource description, and information architecture; as examples, see Niu (), Hall, Fernando, Clough, Soroa, Agirre, and Stevenson (), Neelameghan (), and Wright, Nardini, Aronson, and Rindflesch (). That such a vital tool, and well‐nigh universally used, principle for organizing and representing meaningful recorded information is also an objective measure of complexity is a further indication that complexity and information are closely linked.…”

Evaluation

“…Regardless of whether the hierarchies are created manually or automatically, they must be evaluated [26]. Evaluation approaches can be grouped into the following [17]: (i) the comparison of a hierarchy with an existing gold standard; (ii) the comparison of the hierarchy against a set of pre-defined criteria (e.g., consistency, completeness or clarity); (iii) evaluation of the hierarchy by a group of domain experts; and (iv) the use of statistical measures to automatically evaluate and compare hierarchies. Increasingly, there is a need to evaluate the hierarchies in work and task contexts.…”