Data-driven adaptive selection of rule quality measures for improving rule induction and filtration algorithms

Abstract

This paper presents a proposal of a rule induction algorithm selecting a rule quality measure adaptively. The quality measure plays the role of an optimization criterion of the generated rules. Nine quality measures applied by the algorithm are presented and discussed in the paper. It is shown experimentally that the proposed algorithm provides us with obtaining a classifier of the best quality. During experiments, three criteria of the classifier quality were considered: overall accuracy, balanced accuracy (average accuracy of decision classes), and complexity of the classifier (understood to mean the number of induced rules). The experiments were carried out on 34 data sets coming from the UCI machine learning repository. Moreover, a proposal of four-rule filtration algorithms is presented in the paper. Their task is to limit the number of rules in the classifier. In particular, filtration influence on the classifier quality is studied.

Rule Quality Measures Settings in a Sequential Covering Rule Induction Algorithm - an Empirical Approach

“…It can be seen that the C2 measure leads, on average, to the highest overall classification accuracy, wLap to the highest balanced accuracy, and RSS to the smallest number of rules but with significantly worse classification qualities (in the paper [33] it was proven that this worsening is statistically relevant). What is more, the Corr measure can be a reasonable compromise between the number of generated rules and their classification abilities.…”

“…In addition, they demonstrate that, as the parameters change, certain measures begin to behave similarly to others (such an analysis was conducted for m-estimates and Klösgen measures). In our articles [26,33], in turn, we are analysing the measures which do not contain parameters and we demonstrate their efficiency on a data set similar to the one quoted in Fürnkranz's work. However, we use an algorithm which generates the rule set instead of a rule list.…”

“…there is another place (apart from rule growing and pruning) where a rule quality measure can be applied. This way it is possible to achieve better accuracy of the classification, particularly higher specificity and sensitivity of the classifier [33].…”

“…The selection is a result of a previous experiments -in already quoted works [26,33], about 30 measures were analysed with respect to their efficiency. To check the efficiency, the measures were used in the rule induction algorithm -the same measure was used at each induction stage and during the resolution of classification conflicts.…”

“…Additionally, the number of rules was analysed with respect to the interpretability of the obtained rule sets. Without going deeper in the measure selection methodology, which is presented for example in [33], 9 evaluation measures were selected. These measures achieved good results, at least from the point of view of one of the three above mentioned criteria -statistically they performed better than other measures in the majority of data sets.…”