Comparison of Fuzzy Integral-Fuzzy Measure Based Ensemble Algorithms with the State-of-the-Art Ensemble Algorithms

Abstract

Abstract. The Fuzzy Integral (FI) is a non-linear aggregation operator which enables the fusion of information from multiple sources in respect to a Fuzzy Measure (FM) which captures the worth of both the individual sources and all their possible combinations. Based on the expected potential of non-linear aggregation offered by the FI, its application to decision-level fusion in ensemble classifiers, i.e. to fuse multiple classifiers outputs towards one superior decision level output, has recently been explored. A key example of such a FI-FM ensemble classification method is the Decision-level Fuzzy Integral Multiple Kernel Learning (DeFIMKL) algorithm, which aggregates the outputs of kernel based classifiers through the use of the Choquet FI with respect to a FM learned through a regularised quadratic programming approach. While the approach has been validated against a number of classifiers based on multiple kernel learning, it has thus far not been compared to the state-of-the-art in ensemble classification. Thus, this paper puts forward a detailed comparison of FI-FM based ensemble methods, specifically the DeFIMKL algorithm, with state-of-the art ensemble methods including Adaboost, Bagging, Random Forest and Majority Voting over 20 public datasets from the UCI machine learning repository. The results on the selected datasets suggest that the FI based ensemble classifier performs both well and efficiently, indicating that it is a viable alternative when selecting ensemble classifiers and indicating that the non-linear fusion of decision level outputs offered by the FI provides expected potential and warrants further study.

Fuzzy Integral Driven Ensemble Classification using A Priori Fuzzy Measures

“…Ensemble Classification determines the class to which a new object belongs by integrating the results of multiple classifiers. In the previous studies [1], [6], [15], the researchers concluded that aggregation operator based ensemble classifiers worked well in a number of applications such as multi-criteria decision making (MCDM) [16], forensic science [17], software defect prediction [18], brain computer interface (BCI) [19], computer vision [20], [21] and explosive hazard detection [22]. Here, we use the application of ensemble classification to compare the a priori measure with the Sugeno λ-measure and the Uriz measure.…”

“…Majority Voting with SVM (MJSVM): Let x be an instance and S i (i = 1, 2, ..., k) Support Vector Machine (SVM) classifiers that output class labels m i (x, c j ). For each class label c j (where j = 1, ..., n) [15], the output of the final classifier y(x) for instance x is given by:…”

“…DeFIMKL algorithm: The Decision-level Fuzzy Integral Multiple Kernel Learning (DeFIMKL) is a state-of-the-art FI-FM ensemble classification algorithm, which aggregates the kernels through the use of the CFI with respect to a FM learned by a regularised quadratic programming approach [1]. Agrawal et al [15] showed that DeFIMKL is the best FI-FM ensemble classification algorithm, therefore it will be informative to compare DeFIMKL with the a priori FM based ensemble classifier.…”