On the classification of dynamical data streams using novel “Anti-Bayesian” techniques

Abstract

The classication of dynamical data streams is among the most complex problems encountered in classication. This is, rstly, because the distribution of the data streams is nonstationary, and it changes without any prior warning. Secondly, the manner in which it changes is also unknown. Thirdly, and more interestingly, the model operates with the assumption that the correct classes of previously-classied patterns become available at a juncture after their appearance. This paper pioneers the use of unreported novel schemes that can classify such dynamical data streams by invoking the recently-introduced Anti-Bayesian (AB) techniques.Contrary to the Bayesian paradigm, that compare the testing sample with the distribution's central points, AB techniques are based on the information in the distant-from-the-mean samples.Most Bayesian approaches can be naturally extended to dynamical systems by dynamically tracking the mean of each class using, for example, the exponential moving average based estimator, or a sliding window estimator. The AB schemes introduced by Oommen et al., on the other hand, work with a radically dierent approach and with the non-central quantiles of the distributions. Surprisingly and counter-intuitively, the reported AB methods work equally or close-to-equally well to an optimal supervised Bayesian scheme on a host of accepted Pattern Recognition problems. This thus begs its natural extension to the unexplored arena of classication for dynamical data streams. Naturally, for such an AB classication approach, we need to track the non-stationarity of the quantiles of the classes. To achieve this, in this paper, we develop an AB approach for the online classication of data streams by applying the ecient and robust quantile estimators developed by Yazidi and Hammer [12,37].

An Integrated Online Mechanism for Feature Selection and Classification in Streaming Data

“…Approximately all most-used traditional classification approaches depend, either explicitly or implicitly, on the Bayesian theory principle, which yields optimal classification rules [19]. In the theory of probability and statistics, Bayesian's theory identifies the probability of an event based on historical information that may be relevant to that event.…”

“…Various methods have been proposed for these tasks, where a comprehensive review of the literature can be found in research done by Nguyen et al[22]. Traditional classification techniques are proposed for dynamic data streams, usually either directly or implicitly, depending on the Bayesian principle of optimal classification[19].The Gaussian density function is one of the most common probability density functions. Computational flexibility and its ability to model a large volume of features make it popular.…”

An Integrated Online Mechanism for Feature Selection and Classification in Streaming Data

“…Within a Bayesian paradigm, if one wants to classify a new input sample just by utilizing one feature from each class, the Bayesian strategy would be to achieve this based on the "distance" (for example, Euclidean) from the corresponding means in the respective distributions. Therefore, to deal with the challenges of data streams, a large body of research has focused on the idea of summarizing the characteristics of a data stream by instead tracking the properties of the stream [19].…”

Microcluster-Based Incremental Ensemble Learning for Noisy, Nonstationary Data Streams

“…A combined online ensemble method was used to simultaneously consider concept drift and the high-dimension problem [9]. Additionally, in the light of various classification scenarios, many supervised learning approaches recently have been widely explored [10][11][12][13][14][15][16][17], and some have been applied in data stream classification, such as support vector machine (SVM) and Bayesian technique.…”