Does Classifier Fusion Improve the Overall Performance? Numerical Analysis of Data and Fusion Method Characteristics Influencing Classifier Fusion Performance

Abstract

The reliability of complex or safety critical systems is of increasing importance in several application fields. In many cases, decisions evaluating situations or conditions are made. To ensure the high accuracy of these decisions, the assignments from different classifiers can be fused to one final decision to improve the decision performance in terms of given measures like accuracy or false alarm rate. Recent research results show that fusion methods not always outperform individual classifiers trained and optimized for a specific situation. Nevertheless fusion helps to ensure reliability and redundancy by combining the advantages of individual classifiers, even if some classifiers are not performing well for specific situations. Especially in unexpected (untrained) situations, fusion of more than one classifier allows to get a suitable decision, because of different behavior of classifiers in this case. Nevertheless, there are several examples, where fusion not always improves the overall accuracy of a decision. In this contribution fusion options are discussed to overcome the problem to overcome the aforementioned problem and to define influencing factors on overall fusion accuracy. As a results requirements for good or guaranteed or possibly increased fusion performance and also suggestions denoting those options not leading to any kind of improvement are given. For illustrating the effects a practical example based on three characteristics of fusion methods (type of classifier output, use of these outputs and necessity of training) and four data properties (number of classes, number of samples, entropy of classes and entropy of attributes) are considered and analyzed with 15 different benchmark data sets, which are classified with eight classification methods. The classification results are fused using seven fusion methods. From the discussion of the results it can be concluded, which fusion method performs best/worst for all data sets as well as which fusion method characteristic or data property has more or less positive/negative influence on the fusion performance in comparison to the best base classifier.Using this information, suitable fusion methods can be selected or data sets can be adapted to improve the reliability of decisions made in complex or safety critical systems.

Portable Near Infrared Spectrometer Dengan Sensor As7263 Untuk Pendugaan Sifat Kimia Jeruk Siam (Citrus Nobilis) Secara Non-Destruktif

“…Selain itu, perbedaan struktur BPNN juga dapat mempengaruhi output BPNN sehingga sulit dan memerlukan banyak waktu untuk menentukan satu arsitektur BPNN tunggal yang tepat untuk menyelesaikan sebuah persoalan (Ruta dan Gabrys, 2000). Untuk mengatasi permasalahan tersebut, dikembangkan algoritma neural network ensemble (NNE) dengan cara menggabungkan beberapa BPNN dengan arsitektur yang berbeda namun menggunakan input yang sama (Rothe et al, 2019). Beberapa penelitian juga menunjukkan bahwa penggabungan beberapa neural network terbukti dapat meningkatkan akurasi klasifikasi atau estimasi dibandingkan dengan menggunakan satu neural network tunggal (Zhou et al, 2002;Liang et al, 2014;Sulistyo et al, 2018).…”

A novel multi-classifier information fusion based on Dempster–Shafer theory: application to vibration-based fault detection

“…But not necessarily for all applications and datasets due to the presence of different levels of noise, outliers, nonlinearities, and data redundancy. 10 To deal with this problem, researchers suggested employing an ensemble of classification models to compensate for the weaknesses and boost the strength of each classifier. 11,12 This, however, leads to two important questions: (1) how to select classifiers from the pool of classifiers to keep the information and impose the diversity and (2) how to combine their outputs to make a final decision.…”

“…19–21 The last one is of interest in this article due to its proven advantages over other combination methods. 10,13,22 For instance, Rothe et al 10 conducted an extensive investigation to analyze the performance of all the fusion methods except for the Dempster–Shafer method. It was concluded that in most cases, although the fusion methods cannot outperform the best individual classifier, they decrease the sensitivity to the outliers.…”

“…These approaches can provide high accuracy for a range of classification applications. But not necessarily for all applications and datasets due to the presence of different levels of noise, outliers, nonlinearities, and data redundancy [23]. To deal with this problem, researchers suggested employing an ensemble of classification models to compensate for the weaknesses and boost the strength of each classifier [19] [11,16].…”

The Effects of Class Imbalance and Training Data Size on Classifier Learning: An Empirical Study

“…However, consistent with the no free lunch (NFL) theorem 1 [26], no single classifier seems to be capable of ensuring optimal results for all datasets. Data characteristics are known to have an intrinsic relationship with classifier performance [2,8,16,18]. This study focuses on two kinds of data characteristics: class imbalance and training data size.…”