Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps

Abstract

A new neural network architecture is introduced for incremental supervised learning of recognition categories and multidimensional maps in response to arbitrary sequences of analog or binary input vectors. The architecture, called Fuzzy ARTMAP, achieves a synthesis of fuzzy logic and Adaptive Resonance Theory (ART) neural networks by exploiting a close formal similarity between the computations of fuzzy subsethood and ART category choice, resonance, and learning. Fuzzy ARTMAP also realizes a new Minimax Learning Rule that conjointly minimizes predictive error and maximizes code compression, or generalization. This is achieved by a match tracking process that increases the ART vigilance parameter by the minimum amount needed to correct a predictive error. As a result, the system automatically learns a minimal number of recognition categories, or "hidden units", to met accuracy criteria. Category proliferation is prevented by normalizing input vectors at a preprocessing stage. A normalization procedure called complement coding leads to a symmetric theory in which the MIN operator (A) and the MAX operator (v) of fuzzy logic play complementary roles. Complement coding uses on-cells and off-cells to represent the input pattern, and preserves individual feature amplitudes while normalizing the total on-cell/off-cell vector. Learning is stable because all adaptive weights can only decrease in time. Decreasing weights correspond to increasing sizes of category "boxes". Smaller vigilance values lead to larger category boxes. Improved prediction is achieved by training the system several times using different orderings of the input set. This voting strategy can also be used to assign probability estimates to competing predictions given small, noisy, or incomplete training sets. Four classes of simulations illustrate Fuzzy ARTMAP performance as compared to benchmark back propagation and genetic algorithm systems. These simulations include (i) finding points inside vs. outside a circle; (ii) learning to tell two spirals apart; (iii) incremental approximation of a piecewise continuous function; and (iv) a letter recognition database. The Fuzzy ARTMAP system is also compared to Salzberg's NGE system and to Simpson's FMMC system.

Data-partitioning using the Hilbert space filling curves: Effect on the speed of convergence of Fuzzy ARTMAP for large database problems

“…The Fuzzy ARTMAP neural network and its associated architecture was introduced by Carpenter and Grossberg in their seminal paper (Carpenter et al, 1992). Since its introduction, a number of Fuzzy ARTMAP variations and associated successful applications of this ART family of neural networks have appeared in the literature (for instance, ARTEMAP (Carpenter & Ross, 1995), ARTMAP-IC (Carpenter & Markuzon, 1998), Ellisoid-ART/ARTMAP ), Fuzzy Min-Max (Simpson, 1992), LAPART2 (Caudell & Healy, 1999), and σ-FLNMAP (Petridis, Kaburlasos, Fragkou, & Kehagais, 2001), to mention only a few.…”

“…Even one of the fastest (in terms of training speed) neural network algorithms, the Fuzzy ARTMAP algorithm ( (Carpenter, Grossberg, & Reynolds, 1991) and (Carpenter, Grossberg, Markuzon, Reynolds, & Rosen, 1992)), and its faster variations ( (Kasuba, 1993), (Taghi, Baghmisheh, & Pavesic, 2003)) tend to converge slowly to a solution as the size of the network increases.…”

“…One of the Fuzzy ARTMAP fast algorithmic variants presented in (Taghi et al, 2003) is called, SFAM2.0 and it has the same functionality as Fuzzy ARTMAP (Carpenter et al, 1992) for classification problems. From now we will refer to this variant of Fuzzy ARTMAP as FS-FAM (Fast Simplified Fuzzy ARTMAP).…”

“…Since the functionality of Fuzzy ARTMAP (Carpenter et al, 1992) and FS-FAM (Taghi et al, 2003) are the same for classification problems we will occasionally refer to FS-FAM as Fuzzy ARTMAP. We chose to demonstrate the effectiveness of our proposed data-partitioning approach on the performance of FS-FAM…”

Category regions as new geometrical concepts in Fuzzy-ART and Fuzzy-ARTMAP

“…Fuzzy-ART (FA) (Carpenter, Grossberg, & Rosen, 1991) and Fuzzy-ARTMAP (FAM) (Carpenter, Grossberg, Markuzon, Reynolds, & Rosen, 1992) are two selforganizing, neural network architectures based on the Adaptive Resonance Theory (ART) introduced by Grossberg (1976), which addresses the stability -plasticity dilemma occurring in learning systems. FA performs unsupervised clustering of its input data.…”