Report on the INEX 2005 interactive track

Abstract

The overall motivation for the Interactive Track (iTrack) at INEX is twofold. First, to investigate the behaviour of users when interacting with components of XML documents, and secondly to investigate and develop approaches for XML retrieval which are effective in user-based environments. One of the major outcomes of the track in 2004 was the need to investigate methods that can be supportive during the search process based on features extracted from the XML formatting[1][2]. Problems that might be solved using such methods include overlapping components and the presentation of retrieved elements in the result list. In the experimental system that was offered in 2005, these two issues were addressed. This offered the opportunity to study how overall user search behaviour was affected by these changes when compared to the behaviour observed in 2004. In addition, the following issues were addressed in 2005, following the recommendations of the INEX Methodology Workshop at the Glasgow IR Festival 1 .

The effect of granularity and order in XML element retrieval

“…Thus only 13.5 percent of the documents searchers inspected were assessed on article level. From our findings, which replicate those of Malik, Larsen & Tombros, 2007) the conclusion that searchers prefer document elements to full documents is easy to draw. There are, however, indications in the transaction logs that conclusions must be regarded with a certain amount of scepticism.…”

“…The interactive track organisers Malik, Larsen & Tombros, 2007) have performed initial studies of the 2005 interactive search logs which suggest that users seem to prefer document elements rather than whole documents and that elements of short length are often assessed as not relevant. They also suggest that searchers prefer to use document metadata as their initial entry point.…”

Summarisation of the logical structure of XML documents

“…is it sections and sub-sections to be included or sections only, etc.) has been found to be unsatisfactory in user studies carried out as part of the INEX Interactive Track (Malik, Larsen, & Tombros, 2007), in the context of XML retrieval (Szlávik, 2006b). We also found in our study that a ToC should reflect the user's query and that it is not enough to determine ToC-worthiness only based on type (e.g.…”

Effects of granularity of search results on the relevance judgment behavior of engineers: Building systems for retrieval and understanding of context

“…Daffodil presents a list of relevant document elements or subparts in response to a user's query. Usually, these are organized by document (Figure 1) (for a review, see Malik et al, 2006).…”