Classification of Ordinal Data Using Neural Networks

Abstract

Abstract.Many real life problems require the classification of items in naturally ordered classes. These problems are traditionally handled by conventional methods for nominal classes, ignoring the order. This paper introduces a new training model for feedforward neural networks, for multiclass classification problems, where the classes are ordered. The proposed model has just one output unit which takes values in the interval [0,1]; this interval is then subdivided into K subintervals (one for each class), according to a specific probabilistic model. A comparison is made with conventional approaches, as well as with other architectures specific for ordinal data proposed in the literature. The new model compares favourably with the other methods under study, in the synthetic dataset used for evaluation.

Ordinal Regression Methods: Survey and Experimental Study

“…To adapt neural networks to the ordinal case structure, targets were reformulated following the OneVsFollowers approach and the prediction phase was accomplished by considering that, under its constrained entropic loss formulation, the output of the q-th output neuron estimates the probability that q and q − 1 events are both true. This methodology was further evaluated and compared in other works [64], [75], [76].…”

An All-at-once Unimodal SVM Approach for Ordinal Classification

“…Based on the unimodal paradigm presented in [13], [14] we extended it onto the SVM context using all-at-once strategies. This paradigm states that the probabilities outputted by a prediction method should increase monotonically until reaching a maximum value and then decrease monotonically.…”

“…Here we recover the idea of the unimodal paradigm presented in [13], [14]. In the presence of a supervised multiclassification problem where the classes are ordered, like for instance the four classes in [15], Excellent > Good > Fair > Poor, if for a particular instance the class with highest a posteriori probability is Fair, then its neighbouring classes, Good and Poor, should have the second and third highest probabilities.…”

“…In simple words, it doesn't make sense that the most likely class is Fair and that the second most likely is Excellent; it should be one of the classes closest to Fair. This unimodal paradigm has already been introduced in the context of neural networks in [13], [14] and we propose in this work to extend it in another context, namely all-at-once support vector machines (SVM).…”

“…< C K . Based on the work in [13], [14] where a new paradigm mainly in the context of neural networks was developed, here we propose to extend this paradigm for SVM. Basically, our paradigm assumes that the a posteriori probabilities of the K classes should follow an unimodal distribution, in order to take into account the order relationship.…”