Mutual information-based feature selection and partition design in fuzzy rule-based classifiers from vague data

Abstract

Algorithms for preprocessing databases with incomplete and imprecise data are seldom studied. For the most part, we lack numerical tools to quantify the mutual information between fuzzy random variables. Therefore, these algorithms (discretization, instance selection, feature selection, etc.) have to use crisp estimations of the interdependency between continuous variables, whose application to vague datasets is arguable. In particular, when we select features for being used in fuzzy rule-based classifiers, we often use a mutual information-based ranking of the relevance of inputs. But, either with crisp or fuzzy data, fuzzy rule-based systems route the input through a fuzzification interface. The fuzzification process may alter this ranking, as the partition of the input data does not need to be optimal. In our opinion, to discover the most important variables for a fuzzy rule-based system, we want to compute the mutual information between the fuzzified variables, and we should not assume that the ranking between the crisp variables is the best one. In this paper we address these problems, and propose an extended definition of the mutual information between two fuzzified continuous variables. We also introduce a numerical algorithm for estimating the mutual information from a sample of vague data. We will show that this estimation can be included in a feature selection algorithm, and also that, in combination with a genetic optimization, the same definition can be used to obtain the most informative fuzzy partition for the data. Both applications will be exemplified with the help of some benchmark problems.

Random Fuzzy Sets as Ill-Perceived Random Variables

“…Unfortunately, this kind of mathematical model is not useful in some problems where the images of the fuzzy random variable represent the imprecise observations of the outcomes of a random experiment [9,[64][65][66][67][68][69]. It does not reflect the available (imprecise) information about the "true" probability distribution that governs the random experiment.…”

Mining fuzzy association rules from low-quality data

“…. ; x m =l m g and according to Sanchez et al (2008) the corresponding lower bound of each linguistic label is implicit from this fuzzy subset (4):…”

“…; l n g; associated with a Ruspini (1969) fuzzy partition. According to Sanchez et al (2008), an inaccurate input will be represented as a fuzzy subset of L, where the discrete probability distribution is generalized to an imprecise probability distribution and, where this imprecise probability distribution can also be interpreted as a possibility distribution (Dubois and Prade 1992). Let us illustrate this with a example.…”

Classification based on Neighborhood from Datasets with Low Quality Data

“…Nevertheless, imperfect information or low quality data inevitably appear in real world applications [16], [17]. The errors in the instruments and/or the corruption due to noise during experiments may lead to the obtaining of information with incomplete data when a value of a specific attribute is being obtained.…”