Combining Linear Classifiers Using Probability-Based Potential Functions

Abstract

The score function can be used as a measure for evaluating predicted probabilities of the classification models. In multiple classifiers systems, one of the problems is the diversity of the way of determining the scoring function of individual base classifiers. To alleviate this limitation, in this article, we propose a novel concept of calculating a scoring function defined by the probability-based potential function. The proposed potential functions take into account the distance of the recognized object from the decision boundary as well as a prior probability of the class labels. The proposed score function has the same nature for all linear base classifiers, which defined the multiple classifiers model. Additionally, the proposed method is compared with other ensemble algorithms based on homogeneous linear base classifiers. The experiments on seventy databases demonstrate the effectiveness of our method. To discuss the results of our experiments, we use multiple classification performance measures dedicated to standard and imbalanced datasets. The statistical analysis of the experiments is also performed. INDEX TERMSEnsemble of classifiers, linear classifier, score function, supervised learning.

Probability-driven scoring functions in combining linear classifiers

“…The values close to the limits still express relatively high class-specific support outside the competence region of the base classifier. Our previous research has shown that reducing the value of the discriminant function outside the competence region of the classifier may significantly improve the classification quality achieved by the ensemble [48,38]. Our first attempt was to provide a simple non-monotonic parametric function [48]:…”

“…To eliminate these drawbacks, we proposed an approach that models the data spread along the plane vector using kernel probability estimators [38]. The conducted experimental evaluation showed that the previously proposed method offers some improvement over the formerly proposed and reference methods.…”

“…• ψ KE -The ensemble in which base classifiers are combined according to approach proposed in [38]. See also equation (17).…”

“…In this paper, we extended the probability-based scoring function proposed in [38]. The scoring function combines measurement-level fusion with geometric space fusion.…”

“…Namely, they utilize information about point distribution along the decision plane. The function proposed in [38] uses only information about the point spread along the normal vector of the decision plane.…”