An improved multiple fuzzy NNC system based on mutual information and fuzzy integral

Abstract

Multiple nearest neighbor classifier system (MNNCS) is a popular method to relax the curse of dimensionality. In previous work, most of the MNNCSs are designed by random methods. Random methods may generate unstable component classifiers. In order to relax the randomness, large amount of component classifiers are needed. This paper first extends nearest neighbor classifier into fuzzy nearest neighbor classifier, and proposes a new multiple fuzzy nearest neighbor classifier system based on mutual information and fuzzy integral, called MIFI-MFNNCS. MIFI-MFNNCS adopts target perturbation. Target perturbation decomposes the original classification problem into several sub-problems, where one sub-problem represents one class data. Each sub-problem is described by the relevant data and features. Then it is classified by one component classifier. Therefore, the number of component classifiers can be fixed and reduced. For one component classifier, data may be selected according to its class. And feature is needed to be selected by mutual information. Mutual information can reduce the uncertainty of each component classifier. Feature selection by mutual information in MIFI-MFNNCS may be less affected by the interaction among different classes. The diversity decisions from sub-problem classifiers are combined by fuzzy integral to get the final decision. Here we propose a new method to compute density value according to mutual information, which is a simple method. To demonstrate the performance of the proposed MIFI-MFNNCS, we perform experimental comparisons using five UCI datasets. The results of component classifiers in MIFI-MFNNCS for Ionosphere are shown and analyzed. MIFI-MFNNCS is compared with (1) NNC (2) NNC after feature selection by mutual information (MI-FS-NNC). In multiple fuzzy nearest neighbor classifier system (MFNNCS), mutual information is compared with attribute bagging. And three combination methods are compared, including fuzzy integral, majority voting rule and average. The experimental results show that the accuracy of MIFI-MFNNCS is better than other methods. And mutual information is superior to attribute bagging. Fuzzy integral shows a better performance than majority voting rule and average.

Data-Informed Fuzzy Measures for Fuzzy Integration of Intervals and Fuzzy Numbers

“…Using s, we propose the GenA measure at (14), where the binomial coefficient term n K −1 accounts for the number of possible K-tuples inh. Equation (14) can be written in a similar spirit as (9),…”

“…The GenA FM for FN-valued inputs can be directly calculated by substituting a FN similarity function, such as those at (30), into the formulation at (14), where the input to the GenA measure is now the set of FNs i . Furthermore, the same recursive method proposed for the interval-valued GenA measure can be used for FN-valued inputs.…”

“…and type-2 fuzzy sets [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16]. While manual specification of the FM works for small sets of sources (there are already 16 possible combinations of sources in the power set of 4 sources), manually specifying the values of the FM for large collections of sources is all but impossible.…”

Objective information theory: A Sextuple model and 9 kinds of metrics

“…Large amount of following studies have successively introduced the concepts of the expansion of Hartley entropy and Shannon entropy [16], relative entropy [17], cumulative residual entropy [18][19][20][21], joint entropy [22,23], conditional entropy [24][25][26], mutual information [27][28][29][30][31][32], cross entropy [33][34][35][36][37][38], fuzzy entropy [15,39], maximum entropy principle [40,41] and minimum cross-entropy principle [42,43], and a series of achievements have been made in these aspects. Zhong makes use of general information functions to unify the methods of describing information metrics with Entropy formulas [4].…”

Eigenclassifiers for combining correlated classifiers

“…Tong and Mitram [39] use genetic algorithms to choose between neural network activation functions and features to increase classifier performance. Wang [43] uses mutual information for feature selection in combining nearest neighbor classifiers using the fuzzy integral. Ensemble algorithms have been proposed and applied to different problems in different domains [48,47,26,52].…”