Comparing Combination Rules of Pairwise Neural Networks Classifiers

Abstract

Abstract. A decomposition approach to multiclass classification problems consists in decomposing a multiclass problem into a set of binary ones. Decomposition splits the complete multiclass problem into a set of smaller classification problems involving only two classes (binary classification: dichotomies). With a decomposition, one has to define a recombination which recomposes the outputs of the dichotomizers in order to solve the original multiclass problem. There are several approaches to the decomposition, the most famous ones being one-againt-all and one-against-one also called pairwise. In this paper, we focus on pairwise decomposition approach to multiclass classification with neural networks as the base learner for the dichotomies. We are primarily interested in the different possible ways to perform the so-called recombination (or decoding). We review standard methods used to decode the decomposition generated by a one-against-one approach. New decoding methods are proposed and compared to standard methods. A stacking decoding is also proposed which consists in replacing the whole decoding or a part of it by a trainable classifier to arbiter among the conflicting predictions of the pairwise classifiers. Proposed methods try to cope with the main problem while using pairwise decomposition: the use of irrelevant classifiers. Substantial gain is obtained on all datasets used in the experiments. Based on the above, we provide future research directions which consider the recombination problem as an ensemble method.

An ensemble approach of dual base learners for multi-class classification problems

“…One way to address this difficulty is to divide the original problem into several binary sub-problems [23,[27][28][29][30][31][32].…”

Optimal construction of one-against-one classifier based on meta-learning

“…Some researchers exploited stacking to combine binary base classifiers of class pairs that are based on the one-against-one approach. Savicky and Fürnkranz [38] used ripper, DT, and nearest neighbor as meta classifiers to combine binary rippers, Lézoray and Cardot [39] used DT to combine binary ANNs. Menahem et al [40] proposed a three-layer architecture based on LR.…”

A novel developmental genetic programming methodology for mathematical modeling and neuroevolution

“…A majority vote combination rule is used to determine the winning class. A multigenic implementation is used, where the number of genes in each chromosome (n) is determined by Equation 3.5 [21]. Each chromosome contains n = k(k-1)/2 genes, where k is the number of classes in the problem.…”