Generating noisy monotone ordinal datasets

Abstract

Ordinal decision problems are very common in real-life. As a result, ordinal classification models have drawn much attention in recent years. Many ordinal problem domains assume that the output is monotonously related to the input, and some ordinal data mining models ensure this property while classifying. However, no one has ever reported how accurate these models are in presence of varying levels of non-monotone noise. In order to do that researchers need an easy-to-use tool for generating artificial ordinal datasets which contain both an arbitrary monotone pattern as well as user-specified levels of non-monotone noise. An algorithm that generates such datasets is presented here in detail for the first time. Two versions of the algorithm are discussed. The first is more time consuming. It generates purely monotone datasets as the base of the computation. Later, non-monotone noise is incrementally inserted to the dataset. The second version is basically similar, but it is significantly faster. It begins with the generation of almost monotone datasets before introducing the noise. Theoretical and empirical studies of the two versions are provided, showing that the second, faster, algorithm is sufficient for almost all practical applications. Some useful information about the two algorithms and suggestions for further research are also discussed.

Rank discrimination measures for enforcing monotonicity in decision tree induction

“…The quantification of non-monotonicity in a dataset has been recently investigated in [28]. For each pair x i ; x h 2 X we denote with NMPðx i ; x h Þ the function which is 1 if (i) or (ii) are satisfied and 0 otherwise.…”

“…The main difficulty when comparing monotone classifiers is the fact that real datasets are generally not monotone consistent and, furthermore, it is difficult to find a real dataset with a specified value of NMI1 index. This is why in [28] an algorithm to generate artificial datasets with a fixed value of NMI1 index is introduced. The same authors suggest to use the proposed algorithm as a test bed for comparing different monotone classification algorithms on sensitivity to different degrees of non-monotone noise.…”

“…The algorithm in [28] essentially consists in generating n m-tuples of values for attributes a 1 ; . .…”

“…We used the algorithm reported in [28] to generate 11 artificial datasets with increasing NMI1 from 0% to 10% with 1% steps. Each dataset consists of 500 examples described by five attributes a 1 ; a 2 ; a 3 ; a 4 ; a 5 ranging in f0; .…”

“…We firstly investigated the effect of non-monotone noise on the considered performance indices for the different measures, generating artificial datasets with an increasing degree of non-monotone noise [28]. In these tests no significant difference on classification accuracy has been detected between the tested measures, while an evident better response of our measures in terms of monotonicity has been noticed, especially for increasing degree of non-monotone noise.…”