Active Learning for Imbalanced Ordinal Regression

Abstract

Ordinal regression (OR), also called ordinal classification, is a special multi-classification designed for problems with ordered classes. Imbalanced data hinders the performance of classification algorithms, especially for OR algorithms, as imbalanced class distributions often arise in OR problems. In this paper, we address an active learning based solution for imbalanced OR problem. We propose an active learning algorithm for OR (AL-OR) to select the most informative samples from unlabeled samples, mark them and add them to the training set. Based on AL-OR, we put forward an improved active learning for imbalanced OR (IAL-IOR), which further adjust the sampling strategy of AL-OR dynamically to make the training data as valuable and balanced as possible. Recall rate for multi-classification and new mean absolute error are designed to measure the performance of the algorithms. To the best of our knowledge, our algorithm is the first algorithm for imbalanced OR in algorithm level. The experimental results show that the proposed algorithms have faster convergence and much better generalization ability than the classical methods and the state-of-the-art methods under the evaluation measurements for imbalance problems. In addition, we also proved the effectiveness of our algorithms by statistical analysis.

Active learning for ordinal classification based on expected cost minimization

“… LogitA 15 is the A-optimal experimental design method for ordinal classification, which tends to query representative instances. ALOR 16 is an uncertainty sampling-based AL method for ordinal classification based on the REDSVM model 57 . This method queries the instance with the smallest distance to the nearest separating hyperplane in each iteration.…”

“…However, the label acquisition for ordinal instances is usually expensive and time-consuming due to the dependence on human preference and domain expertise 10 , 11 , prohibiting the collection of a large number of labeled instances. In this situation, one can use the active learning (AL) technique 12 – 14 to train an ordinal classifier 15 , 16 . Active learning aims to reduce the labeling cost by selectively labeling a small set of valuable instances.…”

“…The prohibitive computational cost limits its usability in practice. In the imbalanced ordinal classification study, Ge et al 16 employed a margin-based uncertainty sampling strategy in ordinal classification to achieve oversampling. It is clear that this method is susceptible to the problems of uncertainty sampling, such as sampling redundancy, selecting outliers, and so on.…”

Active Learning for Ordinal Classification Based on Adaptive Diversity-Based Uncertainty Sampling

“…When the data dimension is large, the prohibitive computational cost will limit its usability. Ge et al [39] tried to solve the imbalanced ordinal classification problem by extending an uncertainty sampling criterion to a threshold-based ordinal classification model. One immediate problem with this method is that the uncertainty sampling criterion may suffer from sampling redundancy.…”

“…• ALOR [39] is an active ordinal classification method based on a reduced SVM model. This method selects the instance with the smallest distance from the nearest decision boundary.…”