Combining predictions in pairwise classification: An optimal adaptive voting strategy and its relation to weighted voting

Abstract

Weighted voting is the commonly used strategy for combining predictions in pairwise classification. Even though it shows good classification performance in practice, it is often criticized for lacking a sound theoretical justification. In this paper, we study the problem of combining predictions within a formal framework of label ranking and, under some model assumptions, derive a generalized voting strategy in which predictions are properly adapted according to the strengths of the corresponding base classifiers. We call this strategy adaptive voting and show that it is optimal in the sense of yielding a MAP prediction of the class label of a test instance. Moreover, we offer a theoretical justification for weighted voting by showing that it yields a good approximation of the optimal adaptive voting prediction. This result is further corroborated by empirical evidence from experiments with real and synthetic data sets showing that, even though adaptive voting is sometimes able to achieve consistent improvements, weighted voting is in general quite competitive, all the more in cases where the aforementioned model assumptions underlying adaptive voting are not met. In this sense, weighted voting appears to be a more robust aggregation strategy.

Construction of interval-valued fuzzy preference relations from ignorance functions and fuzzy preference relations. Application to decision making

“…This first technique is one of the simplest and most widely used aggregation method in pairwise learning [30]. The final class is assigned by computing the maximum vote by rows from the values of the strict fuzzy preference relation P.…”

Improving Pairwise Learning Classification in Fuzzy Rule Based Classification Systems Using Dynamic Classifier Selection

“…However, since all binary classifiers always give an output score for a query instance, those values which are not directly related with the actual class of the instance may lead to noise in the decision process, and therefore to an erroneous classification. This is known as the "non-competence classifier problem" [8], and addressing this issue can lead to the enhancement of the final system. The main hitch is that we cannot know a priori which are the competent classifiers for a given instance, but we can restrict the score matrix for a small subset of classes to whom membership is more probable.…”

“…In order to aggregate the output for all binary classifiers, the simplest and most widely used method in pairwise learning is applying a "Weighted Voting" (WV) [8] so that the final class is assigned by taking the maximum vote among the summation of the scores for the binary classifiers associated to the same class.…”

“…Therefore, these classifiers will submit an erroneous score that can be regarded as noise in the aggregation phase. This case is better known as the "noncompetent classifiers problem" [8], which can mislead the correct labeling of the query example.…”

Variable selection methods for multi-class classification using signomial function

“…Although there are many multi-class classification problems in practice, many embedded methods have been suggested for binary classification. Many algorithms for multiclass classification decompose a multi-class classification problem into a set of multiple binary classification problems (Clark and Boswell, 1991;Anand et al, 1995;Debnath et al, 2004) and combine the outputs of the binary classifiers to construct a multi-class classifier (Friedman, 1996;Hastie and Tibshirani, 1997;Hullermeier and Vanderlooy, 2010). Similarly, multiple variable selections for binary classification problems also can be substituted for variable selection for multi-class classification.…”