Variational inference based bayes online classifiers with concept drift adaptation

Abstract

We present VIGO, a novel online Bayesian classifier for both binary and multiclass problems. In our model, variational inference for multivariate distribution technique is exploited to approximate the class conditional probability density functions of data in an online manner. To handle concept drift that could arise in streaming data, we develop 2 new adaptive methods based on VIGO, which we called VIGOw and VIGOd. While VIGOw naturally adapts to any kind of changing environments, VIGOd maximizes the benefit of a static environment as long as it does not detect any change. Extensive experiments on big/medium real-world/synthetic datasets demonstrate the superior performance of our algorithms over many state-of-the-art methods in the literature.

An Online Variational Inference and Ensemble Based Multi-label Classifier for Data Streams

“…Another concern is that in MLC problems, we have a lot of different measures, and it is not trivial to decide whether the predictive models need to be updated when a multi-label classifier makes wrong or right prediction, unlike in VIGO [2]. To overcome this, a "batch" ‹ ,x will be used to update the sub-model w ,x if it collects information from enough |‹| instances and is cleared totally after that to make space for new incoming instances.…”

“…It naturally emerged from the multiple meanings of many realworld objects and can be treated as the generalization of multiclass classification (also known as single-label classification), where an instance may have a set of relevant labels instead of only one label. In this paper, we adapt one of our recently published online multiclass classifier (named Online Variational Inference for multivariate Gaussians (VIGO)) [2] to multi-label classification due to its demonstrated superior performance over several well-known methods in the literature. Although many batch MLC algorithms have been proposed in the literature, there is relatively little work being done on online (incremental) MLC [3].…”

“…Our proposed VIGO [2] employs variational inference (VI) to approximate the distribution of data in each class by a multivariate Gaussian. It is a second-order generative method, which not only gives a point estimate of parameters, but also the distribution of possible solutions.…”

“…Calculate C, D as in (3), (4) Calculate S, R as in (5) Based on VIG, we built the online classifier VIGO [2]. When a small batch of size A of new observations ∈ class # comes, they can be used as dataset <.…”

“…This is to weigh the number of past instances in (3). Furthermore, the sufficient statistics I, W, X can be calculated in a sequential manner [2], hence there is no need of storing data in the VIGO framework. When a new instance l arrives, it can be used for updating I, W, X of its related class, after that it is discarded permanently.…”