Order statistics-based parametric classification for multi-dimensional distributions

Computer Analysis of Images and Patterns

2013

Abstract. Border Identification (BI) algorithms, a subset of Prototype Reduction Schemes (PRS) aim to reduce the number of training vectors so that the reduced set (the border set) contains only those patterns which lie near the border of the classes, and have sufficient information to perform a meaningful classification. However, one can see that the true border patterns ("near" border) are not able to perform the task independently as they are not able to always distinguish the testing samples. Thus, researchers have worked on this issue so as to find a way to strengthen the "border" set. A recent development in this field tries to add more border patterns, i.e., the "far" borders, to the border set, and this process continues until it reaches a stage at which the classification accuracy no longer increases. In this case, the cardinality of the border set is relatively high. In this paper, we aim to design a novel BI algorithm based on a new definition for the term "border". We opt to select the patterns which lie at the border of the alternate class as the border patterns. Thus, those patterns which are neither on the true discriminant nor too close to the central position of the distributions, are added to the "border" set. The border patterns, which are very small in number (for example, five from both classes), selected in this manner, have the potential to perform a classification which is comparable to that obtained by well-known traditional classifiers like the SVM, and very close to the optimal Bayes' bound.

Section: Discussionmentioning

confidence: 99%

“…The intriguing feature of these few points is that they lie close to the boundary and not to the mean, implying an "anti-Bayesian" philosophy [5,6,7].…”

Section: A Novel Two-class "Anti-bayesian" Bi Schemementioning

confidence: 98%

A Novel Border Identification Algorithm Based on an “Anti-Bayesian” Paradigm

Thomas

Computer Analysis of Images and Patterns

2013

“…We first summarize the AB classification rules designed and proven in [8][9][10] for uni-dimensional features. To do this, we use the notation that for the j th dimension of the feature vector of class ω i , q i,j p is the quantiloid for the value p, i.e., the position where the feature's CDF has a value of p. In the case when both the classes are characterized by only a single feature X, q i p is ω i 's quantiloid for the value p, i.e., more formally q i p = P r(X < p|X ∈ ω i ).…”

Section: "Anti-bayesian" Classification Rulesmentioning

confidence: 99%

“…In this case, the comparison is based on the distant quantiloids and so the classification border is: The reader will observe that the latter case (Case 2) is the one that uses the so-called "Dual" scenario (please see [8][9][10]), and where the extreme quantiloids are used for the classification as opposed to the quantiloids that are close to the discriminant. In the symmetric cases analyzed in [8][9][10], it is easy to see that the assignments in the so-called "Dual" scenario reduce to those involving comparisons to the quantiloids that are close to the discriminant, but where the assignment is to the class that is the more distant one.…”

Section: "Anti-bayesian" Classification Rulesmentioning

confidence: 99%

“…Indeed, rather than follow a Bayesian classification methodology, it traverses one of the AB-based PR CMQSbased schemes reported earlier. In fact, unlike the k-means clustering strategies that rely on centroid-based criteria, we resort to quantiles positions distant from the cluster means [8][9][10], which is a strategy just as counter-intuitive and nonobvious as the CMQS schemes themselves. Central to the development of such CMQS clustering algorithms is the concept of a "Quantiloid" We will elaborate on the phenomenon of Quantiloids in the next section.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

“Anti-Bayesian” flat and hierarchical clustering using symmetric quantiloids

Hammer

Yazidi

2017

Information Sciences

Abstract.A myriad of works has been published for achieving data clustering based on the Bayesian paradigm, where the clustering sometimes resorts to Naïve-Bayes decisions. Within the domain of clustering, the Bayesian principle corresponds to assigning the unlabelled samples to the cluster whose mean (or centroid) is the closest. Recently, Oommen and his co-authors have proposed a novel, counter-intuitive and pioneering PR scheme that is radically opposed to the Bayesian principle. The rational for this paradigm, referred to as the "Anti-Bayesian" (AB) paradigm, involves classification based on the non-central quantiles of the distributions. The first-reported work to achieve clustering using the AB paradigm was in [1], where we proposed a flat clustering method which assigned unlabelled points to clusters based on the AB paradigm, and where the distances to the respective learned clusters was based on their quantiles rather than the clusters' centroids for uni-dimensional and two-dimensional data. This paper, extends the results of [1] in many directions. Firstly, we generalize our previous AB clustering [1], initially proposed for handling uni-dimensional and two-dimensional spaces, to arbitrary d-dimensional spaces using their so-called "quantiloids". Secondly, we extend the AB paradigm to consider how the clustering can be achieved in hierarchical ways, where we analyze both the TopDown and the Bottom-Up clustering options. Extensive experimentation demonstrates that our clustering achieves results competitive to the state-of-the-art flat, Top-Down and Bottom-Up clustering approaches, demonstrating the power of the AB paradigm.

Ultimate Order Statistics-Based Prototype Reduction Schemes

Thomas

Lecture Notes in Computer Science

2013

Abstract. The objective of Prototype Reduction Schemes (PRSs) and Border Identification (BI) algorithms is to reduce the number of training vectors, while simultaneously attempting to guarantee that the classifier built on the reduced design set performs as well, or nearly as well, as the classifier built on the original design set. In this paper, we shall push the limit on the field of PRSs to see if we can obtain a classification accuracy comparable to the optimal, by condensing the information in the data set into a single training point. We, indeed, demonstrate that such PRSs exist and are attainable, and show that the design and implementation of such schemes work with the recently-introduced paradigm of Order Statistics (OS)-based classifiers. These classifiers, referred to as Classification by Moments of Order Statistics (CMOS) is essentially anti-Bayesian in its modus operandus. In this paper, we demonstrate the power and potential of CMOS to yield single-element PRSs which are either "selective" or "creative", where in each case we resort to a non-parametric or a parametric paradigm respectively. We also report a single-feature single-element creative PRS. All of these solutions have been used to achieve classification for real-life data sets from the UCI Machine Learning Repository, where we have followed an approach that is similar to the Naïve-Bayes' (NB) strategy although it is essentially of an anti-Naïve-Bayes' paradigm. The amazing facet of this approach is that the training set can be reduced to a single pattern from each of the classes which is, in turn, determined by the CMOS features. It is even more fascinating to see that the scheme can be rendered operational by using the information in a single feature of such a single data point. In each of these cases, the accuracy of the proposed PRS-based approach is very close to the optimal Bayes' bound and is almost comparable to that of the SVM.