Mining Recurrent Concepts in Data Streams Using the Discrete Fourier Transform

Abstract

In this research we address the problem of capturing recurring concepts in a data stream environment. Recurrence capture enables the re-use of previously learned classifiers without the need for re-learning while providing for better accuracy during the concept recurrence interval. We capture concepts by applying the Discrete Fourier Transform (DFT) to Decision Tree classifiers to obtain highly compressed versions of the trees at concept drift points in the stream and store such trees in a repository for future use. Our empirical results on real world and synthetic data exhibiting varying degrees of recurrence show that the Fourier compressed trees are more robust to noise and are able to capture recurring concepts with higher precision than a meta learning approach that chooses to re-use classifiers in their originally occurring form. The DFT, apart from its use in meta learning, has a number of other desirable properties that make it attractive for mining high speed data streams. This includes the ability to classify directly from the spectra generated, thus eliminating the need for expensive traversal of a tree structure.Our experimental results in section 5 clearly show the accuracy, processing speed and memory advantages of applying the DFT as opposed to the meta learning approach proposed by Gama and Kosina in [4].The rest of the paper is as follows. In section 2 we review work done in the area of capturing recurrences. We describe the basics of applying the DFT to decision trees in section 3. In section 4 we discuss a novel approach to optimizing the computation of the Fourier spectrum from a Decision Tree. Our experimental results are presented in section 5 and we conclude the paper in section 6 where we draw conclusions on the research and discuss some directions for future research.

Capturing recurring concepts using discrete Fourier transform

“…The research presented in the paper is an extension to the work in in the following aspects: A new metric called Robustness to Concept Change is introduced to compare the exten of reuse in recurring concept capturing context. A real‐world case study with flight simulator dataset is used to study the effectiveness of our method on an environment where similar concepts recur. A comparison is made to reveal the impact of two drift detectors with contrasting characteristics in recurring concept context. …”

The incremental Fourier classifier: Leveraging the discrete Fourier transform for classifying high speed data streams

“…The first approach taken by Sripirakas & Pears (2014), Sakthithasan et al (2015), Kithulgoda & Pears (2016) is to select the best performing tree from a decision tree forest at each concept drift point and apply the DFT to produce a spectrum. This spectrum is then aggregated with the most similar spectrum already resident in the repository through the use of a Euclidean distance measure.…”

“…It has several attractive properties for capturing patterns that sets it apart from conventional mechanisms such as decision trees and other types of classifiers. Firstly, it has been shown rigorously that spectra generated from hierarchical classifiers such as decision trees can be represented in compact form thus speeding up the classification process Sripirakas & Pears (2014). Secondly, Fourier spectra have the ability to embed several different patterns (concepts) into one entity unlike conventional ensemble classifier systems which maintain multiple models.…”

Online Ensemble Learning of Data Streams with Gradually Evolved Classes

“…Class reoccurrence in class evolution is relevant to recurrent concept drift, which represents the case where a past concept reoccurs again in the data stream [24] [25] [26]. However, the two cases are substantially different.…”