Constant interaction-time scatter/gather browsing of very large document collections

Abstract

The Scatter/Gather document browsing method uses fast document clustering to produce table-of-contentslike outlines of large document collections. Previous work [I] developed linear-time document clustering algorithms to establish the feasibility of this method over moderately large collections. However, even linear-time algorithms are too slow to support, interactive browsing of very large collections such as Tipster. the DARPA st,andard text retrieval evaluation collection. We present a scheme that supports constant interaction-time Scatter/Gather of arbitrarily large collections after nearlinear time preprocessing. This involves the construction of a cluster hzerarch,y. A modification of Scatter/Gather employing this scheme, and an example of its use over the Tipster collection are presented.

Grouper: a dynamic clustering interface to Web search results

“…Many document clustering algorithms rely on off-line clustering of the entire document collection [6,28]. Since the Web is a dynamic environment, static, pre-computed clusters would have to be constantly updated, and most clustering algorithms cannot do so incrementally.…”

Exact and Approximation Algorithms for Clustering

“…Clustering a set of points into a few groups is frequently used for statistical analysis and classi cation in numerous applications, including information retrieval 6,7,8,26], facility location 11,30], data mining 2,28], spatial data bases 9,20,25], data compression 23], image processing 19, 27], astrophysics 22], and scienti c computing 5]. Because of such a diversity of applications, several variants of clustering problems have been proposed and widely studied.…”

A Survey of Text Clustering Algorithms

“…One possibility is to perform a hierarchical clustering a-priori; however such an approach has the disadvantage that it is unable to merge and recluster related branches of the tree hierarchy on-the-fly when a user may need it. A method for constant-interaction time browsing with the use of the scatter-gather approach has been presented in [26]. This approach presents the keywords associated with the different keywords to a user.…”

“…Clearly, a larger value of M does not assume the cluster-refinement hypothesis quite as strongly, but also comes at a higher cost. The details of the algorithm are described in [26]. Some extensions of this approach are also presented in [85], in which it has been shown how this approach can be used to cluster arbitrary corpus subsets of the documents in constant time.…”

“…The problem of clustering can be very useful in the text domain, where the objects to be clusters can be of different granularities such as documents, paragraphs, sentences or terms. Clustering is especially useful for organizing documents to improve retrieval and support browsing [11,26].…”