Prototype-Based Classifiers in the Presence of Concept Drift: A Modelling Framework

Abstract

We present a modelling framework for the investigation of prototype-based classifiers in non-stationary environments. Specifically, we study Learning Vector Quantization (LVQ) systems trained from a stream of high-dimensional, clustered data. We consider standard winnertakes-all updates known as LVQ1. Statistical properties of the input data change on the time scale defined by the training process. We apply analytical methods borrowed from statistical physics which have been used earlier for the exact description of learning in stationary environments. The suggested framework facilitates the computation of learning curves in the presence of virtual and real concept drift. Here we focus on timedependent class bias in the training data. First results demonstrate that, while basic LVQ algorithms are suitable for the training in non-stationary environments, weight decay as an explicit mechanism of forgetting does not improve the performance under the considered drift processes.

Supervised Learning in the Presence of Concept Drift: A modelling framework

“…The approach is based on the idea of representing classes by more or less typical representatives of the training instances. This suggests that LVQ algorithms should also be capable of tracking changes in the density of samples, a hypothesis that has been studied for instance in [14,24], recently.…”

“…This paper presents extensions of our contribution to the Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Visualization (WSOM 2019) [46]. Consequently, parts of the text resemble or have been taken over literally from [14] without explictit notice. This concerns, for instance, parts of the introduction and the description of models and methodology in Sec.…”

“…2. Similarly, some of the results have also been presented in [14], which focused on the study of explicitly time-dependent densities in a stream of clustered data for LVQ training.…”

“…We complement our conference contribution [14] significantly by studying also the influence of drift on the training of regression type layered neural networks. First results concerning such systems with sigmoidal hidden unit activation function under concept drift have been published in [45], recently.…”

Reactive Soft Prototype Computing for Concept Drift Streams

“…Both equations are controlled by the decay-factor γ controlling the relevance of previous updates and the current ones. As pointed out in [22], a momentumbased gradient descent is a reliable strategy to handle concept drift with LVQ classifiers. In a streaming setting at every time step, every prototype will be updated by Eq.…”