Retrieval experiments using pseudo-desktop collections

Abstract

Desktop search is an important part of personal information management (PIM). However, research in this area has been limited by the lack of shareable test collections, making cumulative progress difficult. In this paper, we define desktop search as a semi-structured document retrieval problem and introduce a methodology to automatically build a reusable collection (the pseudo-desktop) that has many of the same properties as a real desktop collection.We then present a comprehensive evaluation of retrieval methods for semi-structured document retrieval on several pseudo-desktop collections and the TREC Enterprise collection. Our results show that a probabilistic retrieval model using the mapping relation between a query term and a document field (PRM-S) has the best performance in collections with more structure, such as email, and that the query-likelihood language model is better for other document types. We further analyze the observed differences using generated queries and suggest ways to improve PRM-S, which makes the performance gains more significant and consistent.

A scalable search engine for mass storage smart objects

“…This use-case is representative of situations where the indexing documents have a rich content (tens to hundreds of thousands of terms) and documents updates and deletes can be performed randomly. To capture the behavior of our solution in such context, we use the pseudo-desktop data collection and query set provided in [11] which is considered as representative of a personal desktop where searches, updates and deletes are performed.…”

“…The desktop search is an important topic in the IR community, but real personal collections of desktop files cannot be published for evident privacy issues. Instead, the authors in [11] propose a method to generate pseudo desktop collections and show that such collections have the same properties as real collections. As recommended in [11], we preprocess the files in this collection by removing the stop words and stemming the remaining terms using the Krovetz stemmer.…”

“…Instead, the authors in [11] propose a method to generate pseudo desktop collections and show that such collections have the same properties as real collections. As recommended in [11], we preprocess the files in this collection by removing the stop words and stemming the remaining terms using the Krovetz stemmer. The obtained number of terms in the vocabulary is large, i.e., 337952.…”

Generating queries from user-selected text

“…Another related research area involves the extraction of representative words from a document. Research on knownitem search [4,14] constructs (query, document) pairs by extracting important words from a document and formulating pseudo queries in a desktop search environment. This work, however, chooses words from a document based on some basic statistical indicators, which risks losing informative phrases and could be too simple to generate queries for long documents of 5000 words.…”

Evaluating Personal Information Retrieval

“…This latter issue creates problems for all aspects of the evaluation of search of personal collections. Current work on evaluation of personal collection search is exploring the development of simulated personal Cranfield type search test collections [2]. However, these collections do not represent the diversity of real users collections, items selected by an individual owning the collection that they actually want to retrieve from it, nor the query terms collection owners will use.…”