Negative Correlation Ensemble Learning for Ordinal Regression

Abstract

In this paper, two neural network threshold ensemble models are proposed for ordinal regression problems. For the first ensemble method, the thresholds are fixed a priori and are not modified during training. The second one considers the thresholds of each member of the ensemble as free parameters, allowing their modification during the training process. This is achieved through a reformulation of these tunable thresholds, which avoids the constraints they must fulfill for the ordinal regression problem. During training, diversity exists in different projections generated by each member is taken into account for the parameter updating. This diversity is promoted in an explicit way using a diversity-encouraging error function, extending the well-known negative correlation learning framework to the area of ordinal regression, and inheriting many of its good properties. Experimental results demonstrate that the proposed algorithms can achieve competitive generalization performance when considering four ordinal regression metrics.

Ordinal Regression Methods: Survey and Experimental Study

“…The framework of negative correlation learning (where the ensemble members are learnt in such a way that the correlation between their responses is minimised) was used in the context of ordinal regression [17], [112] by calculating the correlation between the latent variable estimations or, alternatively, between the probabilities obtained by the ensemble members.…”

Ordinal Neural Networks Without Iterative Tuning

“…In the field of ensemble learning, Fernández-Navarro et al [38] recently propose a modified version of the negative correlation framework for OR. They study two versions of the base algorithm.…”

Multiple Ordinal Regression by Maximizing the Sum of Margins

“…Other recent ordinal regression approaches provide extensions for ensemble learning [29], [11], sampling problems [30], and semi-supervised learning [37]. [29] proposes an ensemble learning for ordinal regression by extracting multiple projection-based two class classifiers and three class ordinal regressors.…”

“…The ensemble of the probability scores obtained from these functions are used to rank the test samples. [11] addresses the feature selection for ensemble learning of ordinal regressors. Negative correlation is used to find new features that provide additional information to the ensemble.…”