The unimodal model for the classification of ordinal data

Abstract

Many real life problems require the classification of items into naturally ordered classes. These problems are traditionally handled by conventional methods intended for the classification of nominal classes where the order relation is ignored. This paper introduces a new machine learning paradigm intended for multi-class classification problems where the classes are ordered. The theoretical development of this paradigm is carried out under the key idea that the random variable class associated with a given query should follow a unimodal distribution. In this context, two approaches are considered: a parametric, where the random variable class is assumed to follow a specific discrete distribution; a nonparametric, where the random variable class is assumed to be distribution-free. In either case, the unimodal model can be implemented in practice by means of feedforward neural networks and support vector machines, for instance. Nevertheless, our main focus is on feedforward neural networks. We also introduce a new coefficient, r(int), to measure the performance of ordinal data classifiers. An experimental study with artificial and real datasets is presented in order to illustrate the performances of both parametric and nonparametric approaches and compare them with the performances of other methods. The superiority of the parametric approach is suggested, namely when flexible discrete distributions, a new concept introduced here, are considered.

An All-at-once Unimodal SVM Approach for Ordinal Classification

“…Here we recover the idea of the unimodal paradigm presented in [13], [14]. In the presence of a supervised multiclassification problem where the classes are ordered, like for instance the four classes in [15], Excellent > Good > Fair > Poor, if for a particular instance the class with highest a posteriori probability is Fair, then its neighbouring classes, Good and Poor, should have the second and third highest probabilities.…”

“…In simple words, it doesn't make sense that the most likely class is Fair and that the second most likely is Excellent; it should be one of the classes closest to Fair. This unimodal paradigm has already been introduced in the context of neural networks in [13], [14] and we propose in this work to extend it in another context, namely all-at-once support vector machines (SVM).…”

“…More recently, [11] proposed a cascade classification technique encompassing a decision tree classifier and a model tree algorithm. In [12], [13] two new methods were present towards ordinal classification. In [12] a new reduction technique is used allowing to solve the problem of ordinal classification using a single binary classifier.…”

“…In [12] a new reduction technique is used allowing to solve the problem of ordinal classification using a single binary classifier. In [13] the class order relation is taken into account by imposing an unimodal distribution to the class a posteriori probabilities.…”

“…< C K . Based on the work in [13], [14] where a new paradigm mainly in the context of neural networks was developed, here we propose to extend this paradigm for SVM. Basically, our paradigm assumes that the a posteriori probabilities of the K classes should follow an unimodal distribution, in order to take into account the order relationship.…”