Automatic construction of navigable concept networks characterizing text databases

Abstract

In this paper we present a comprehensive approach to conceptual structuring and intelligent navigation of text databases. Given any collection of texts, we first automatically extract a set of index terms describing each text. Next, we use a particular lattice conceptual clustering method to build a network of clustered texts whose nodes are described using the index terms. We argue that the resulting network supports an hybrid navigational approach to text retrieval -implemented into an actual user interface -that combines browsing potentials with good retrieval performance. We present the results of an experiment on subject searching where this approach outperformed a conventional Boolean retrieval system.

From Data Mining to Knowledge Mining

“…A classification quality criterion used in conceptual clustering may involve a variety of factors, such as the fit of a cluster description to the data (called sparseness), the simplicity of the description, and other properties of the entities or the concepts that describe them (Michalski and Stepp, 1983;Stepp and Michalski, 1986). Ideas on employing conceptual clustering for structuring text databases and creating concept lattices for discovering dependencies in data are described by Carpineto and Romano (1995a;1995b). The concepts created through the clustering are linked in lattice structures that can be traversed to represent generalization and specialization relationships.…”