Discovery of hierarchical thematic structure in text collections with adaptive resonance theory

Abstract

This paper investigates the abilities of Adaptive Resonance Theory (ART) neural networks as miners of hierarchical thematic structure in text collections. We present experimental results with binary ART1 on the benchmark Reuter-21578 corpus. Using both quantitative evaluation with the standard F 1 measure and qualitative visualization of the hierarchy obtained with ART, we discuss how useful ART built hierarchies would be to a user intending to use it as a means to find and access textual information. Our F 1 results show that ART1 produces hierarchical clustering that exhibit a quality exceeding k-means and a hierarchical clustering algorithm. However, we identify several critical problem areas that would make it rather impractical to actually use such a hierarchy in a real-life environment. These predicaments point to the importance of semantic feature selection. Our main contribution is to test in details the applicability of ART to the important domain of hierarchical document clustering, an application of Adaptive Resonance that had received little attention until now.

A survey of adaptive resonance theory neural network models for engineering applications

“…Elementary ART modules have been used as building blocks to construct both bottom-up (agglomerative) and top-down (divisive) hierarchical architectures. Typically, these follow one of two designs (Massey, 2009): (i) cascade (series connection) of ART modules in which the output of a preceding ART layer is used as the input of the succeeding one, or (ii) parallel ART modules enforcing different vigilance criteria while having a common input layer.…”

Distributed dual vigilance fuzzy adaptive resonance theory learns online, retrieves arbitrarily-shaped clusters, and mitigates order dependence

“…Particularly, ART has been used as the basis for several hierarchical clustering methods, which can be classified into bottom-up (agglomerative or merging methods) and top-down (divisive or splitting methods) [2]. Hierarchical ART architectures generally follow two main designs [20]: (a) a series/cascade of ART modules where the output of one ART (i.e., a prototype) is the input of the next [21][22][23][24][25][26][27][28][29][30][31] or (b) parallel ART modules sharing the same inputs and using different vigilance values [6,[32][33][34][35][36][37][38][39]. Generally, the hierarchical relationships between ART modules are defined implicitly by the input signal flow, explicitly by enforcing constraints or connections, and/or by the setting of multiple vigilance parameters to define hierarchies.…”

“…Alternatively, hierarchies within the same ART can be created by designing custom ART activation functions [40,41] or by analyzing its distributed activation patterns [42]. ART-based hierarchical approaches have been successfully applied, for instance, in text mining [20,43] and robotics [30,39].Another branch of clustering includes multi-prototype-based methods. These allow multiple prototypes to represent a single cluster and more accurately capture the data topology, thereby typically handling clusters with arbitrary shapes.Multi-prototype representations have been successfully used for clustering [44][45][46][47][48], visualization [46,49,50], and validation purposes [51,52].…”

“…Alternatively, hierarchies within the same ART can be created by designing custom ART activation functions [40,41] or by analyzing its distributed activation patterns [42]. ART-based hierarchical approaches have been successfully applied, for instance, in text mining [20,43] and robotics [30,39].…”