Exploring the Hidden Challenges Associated with the Evaluation of Multi-class Datasets Using Multiple Classifiers

Abstract

-The optimization and evaluation of a pattern recognition system requires different problems like multi-class and imbalanced datasets be addressed. This paper presents the classification of multi-class datasets which present more challenges when compare to binary class datasets in machine learning. Furthermore, it argues that the performance evaluation of a classification model for multi-class imbalanced datasets in terms of simple "accuracy rate" can possibly provide misleading results. Other parameters such as failure avoidance, true identification of positive and negative instances of a class and class discrimination are also very important. We, in this paper, hypothesize that "misclassification of true positive patterns should not necessarily be categorized as false negative while evaluating a classifier for multi-class datasets", a common practice that has been observed in the existing literature. In order to address these hidden challenges for the generalization of a particular classifier, several evaluation metrics are compared for a multi-class dataset with four classes; three of them belong to different neurodegenerative diseases and one to control subjects. Three classifiers, linear discriminant, quadratic discriminant and Parzen are selected to demonstrate the results with examples.

A novel method for network traffic classification based on robust support vector machine

“…To verify the universal performance of proposed algorithm in the field of network classification, a series of experiments will also be conducted on the Moore dataset 18 . Moore dataset is an important dataset to study the classification of network traffic.…”

“…The simulation indicated the method had better performance on user experience. There are also lots of experiments simulate on outdated dataset, such as Moore and the PCAP 18‐20 …”