Full Border Identification for Reduction of Training Sets

2014

Pattern Recognition

This paper submits a comprehensive report of the use of Order Statistics (OS) for parametric Pattern Recognition (PR) for various distributions within the exponential family. Although the field of parametric PR has been thoroughly studied for over five decades, the use of the OS of the distributions to achieve this has not been reported. The pioneering work on using OS for classification was presented earlier for the Uniform distribution and for some members of the exponential family, where it was shown that optimal PR can be achieved in a counter-intuitive manner, diametrically opposed to the Bayesian paradigm, i.e., by comparing the testing sample to a few samples distant from the mean. Apart from the results for the Gaussian and doubleexponential which are merely cited here, our new results include the Rayleigh, Gamma and certain Beta distributions. The new scheme, referred to as Classification by Moments of Order Statistics (CMOS), has an accuracy that attains the Bayes' bound for symmetric distributions, and is, otherwise, very close to the optimal Bayes' bound, as has been shown both theoretically and by rigorous experimental testing. The results here also give a theoretical foundation for the families of Border Identification (BI) algorithms reported in the literature.

Section: Border Identification Algorithmsmentioning

confidence: 99%

mentioning

confidence: 99%

“Anti-Bayesian” parametric pattern classification using order statistics criteria for some members of the exponential family

2014

Pattern Recognition

“…The training is then performed on this reduced set, which is also called the "Reference" set. More recent advances have involved the use of Border Identification (BI) algorithms [5][6][7][8] to choose these prototypes from the so-called "border" points of the various classes.…”

Section: Introductionmentioning

confidence: 99%

Ultimate Order Statistics-Based Prototype Reduction Schemes

Lecture Notes in Computer Science

2013

Abstract. The objective of Prototype Reduction Schemes (PRSs) and Border Identification (BI) algorithms is to reduce the number of training vectors, while simultaneously attempting to guarantee that the classifier built on the reduced design set performs as well, or nearly as well, as the classifier built on the original design set. In this paper, we shall push the limit on the field of PRSs to see if we can obtain a classification accuracy comparable to the optimal, by condensing the information in the data set into a single training point. We, indeed, demonstrate that such PRSs exist and are attainable, and show that the design and implementation of such schemes work with the recently-introduced paradigm of Order Statistics (OS)-based classifiers. These classifiers, referred to as Classification by Moments of Order Statistics (CMOS) is essentially anti-Bayesian in its modus operandus. In this paper, we demonstrate the power and potential of CMOS to yield single-element PRSs which are either "selective" or "creative", where in each case we resort to a non-parametric or a parametric paradigm respectively. We also report a single-feature single-element creative PRS. All of these solutions have been used to achieve classification for real-life data sets from the UCI Machine Learning Repository, where we have followed an approach that is similar to the Naïve-Bayes' (NB) strategy although it is essentially of an anti-Naïve-Bayes' paradigm. The amazing facet of this approach is that the training set can be reduced to a single pattern from each of the classes which is, in turn, determined by the CMOS features. It is even more fascinating to see that the scheme can be rendered operational by using the information in a single feature of such a single data point. In each of these cases, the accuracy of the proposed PRS-based approach is very close to the optimal Bayes' bound and is almost comparable to that of the SVM.

“…Specializing this criterion, the current-day BI algorithms, designed by Duch [1], Foody [2,3], and Li et al [4], attempt to select a Reference set which contains border patterns derived, in turn, from the set of training patterns. Observe that, in effect, these algorithms also yield reduced training sets.…”

Section: Introductionmentioning

confidence: 99%

A Novel Border Identification Algorithm Based on an “Anti-Bayesian” Paradigm

Computer Analysis of Images and Patterns

2013

Abstract. Border Identification (BI) algorithms, a subset of Prototype Reduction Schemes (PRS) aim to reduce the number of training vectors so that the reduced set (the border set) contains only those patterns which lie near the border of the classes, and have sufficient information to perform a meaningful classification. However, one can see that the true border patterns ("near" border) are not able to perform the task independently as they are not able to always distinguish the testing samples. Thus, researchers have worked on this issue so as to find a way to strengthen the "border" set. A recent development in this field tries to add more border patterns, i.e., the "far" borders, to the border set, and this process continues until it reaches a stage at which the classification accuracy no longer increases. In this case, the cardinality of the border set is relatively high. In this paper, we aim to design a novel BI algorithm based on a new definition for the term "border". We opt to select the patterns which lie at the border of the alternate class as the border patterns. Thus, those patterns which are neither on the true discriminant nor too close to the central position of the distributions, are added to the "border" set. The border patterns, which are very small in number (for example, five from both classes), selected in this manner, have the potential to perform a classification which is comparable to that obtained by well-known traditional classifiers like the SVM, and very close to the optimal Bayes' bound.