Enabling Fast Lazy Learning for Data Streams

Zhang, Peng; Gao, Byron J.; Zhu, Xingquan; Guo, Li

doi:10.1109/icdm.2011.63

Cited by 47 publications

(20 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We couple the proposed architecture with the kNN classifier as one of the most popular nonparametric models. KNN has already been applied in the streaming scenario, mainly with the goal to provide an efficient search [21] or a compressed representation [22]. It was also considered for drifting data streams [13], [23].…”

Section: Related Workmentioning

confidence: 99%

KNN Classifier with Self Adjusting Memory for Heterogeneous Concept Drift

Losing

Hammer

Wersing

2016

2016 IEEE 16th International Conference on Data Mining (ICDM)

179

148

View full text Add to dashboard Cite

Abstract-Data Mining in non-stationary data streams is gaining more attention recently, especially in the context of Internet of Things and Big Data. It is a highly challenging task, since the fundamentally different types of possibly occurring drift undermine classical assumptions such as data independence or stationary distributions. Available algorithms are either struggling with certain forms of drift or require a priori knowledge in terms of a task specific setting. We propose the Self Adjusting Memory (SAM) model for the k Nearest Neighbor (kNN) algorithm since kNN constitutes a proven classifier within the streaming setting. SAM-kNN can deal with heterogeneous concept drift, i.e different drift types and rates, using biologically inspired memory models and their coordination. It can be easily applied in practice since an optimization of the meta parameters is not necessary. The basic idea is to construct dedicated models for the current and former concepts and apply them according to the demands of the given situation. An extensive evaluation on various benchmarks, consisting of artificial streams with known drift characteristics as well as real world datasets is conducted. Thereby, we explicitly add new benchmarks enabling a precise performance evaluation on multiple types of drift. The highly competitive results throughout all experiments underline the robustness of SAM-kNN as well as its capability to handle heterogeneous concept drift.

show abstract

Section: Related Workmentioning

confidence: 99%

KNN Classifier with Self Adjusting Memory for Heterogeneous Concept Drift

Losing

Hammer

Wersing

2016

2016 IEEE 16th International Conference on Data Mining (ICDM)

179

148

View full text Add to dashboard Cite

show abstract

“…Examples are k Nearest Neighbour (Beringer and Hüllermeier 2007;Zhang et al 2011), Stochastic Gradient Descent (Bottou 2004) and SPegasos (Stochastic Primal Estimated sub-GrAdient SOlver for SVMs) (Shalev-Shwartz et al 2011). Both Stochastic Gradient Descent and SPegasos are gradient descent methods, capable of learning a variety of linear models, such as Support Vector Machines and Logistic Regression, depending on the chosen loss function.…”

Section: Related Workmentioning

confidence: 99%

The online performance estimation framework: heterogeneous ensemble learning for data streams

et al. 2017

View full text Add to dashboard Cite

Ensembles of classifiers are among the best performing classifiers available in many data mining applications, including the mining of data streams. Rather than training one classifier, multiple classifiers are trained, and their predictions are combined according to a given voting schedule. An important prerequisite for ensembles to be successful is that the individual models are diverse. One way to vastly increase the diversity among the models is to build an heterogeneous ensemble, comprised of fundamentally different model types. However, most ensembles developed specifically for the dynamic data stream setting rely on only one type of base-level classifier, most often Hoeffding Trees. We study the use of heterogeneous ensembles for data streams. We introduce the Online Performance Estimation framework, which dynamically weights the votes of individual classifiers in an ensemble. Using an internal evaluation on recent training data, it measures how well ensemble members performed on this and dynamically updates their weights. Experiments over a wide range of data streams show performance that is competitive with state of the art ensemble techniques, including Online Bagging and Leveraging Bagging, while being significantly faster. All experimental results from this work are easily reproducible and publicly available online.

show abstract

“…On the other hand, the K-d trees and M-trees can be used in the instance-based learning models and SVM models. For example, our recent work [38] proposes a new L-tree structure that extends M-trees to index instance-based learning on data streams.…”

Section: Related Workmentioning

confidence: 99%

E-Tree: An Efficient Indexing Structure for Ensemble Models on Data Streams

Zhang

Zhou

Wang

et al. 2015

IEEE Trans. Knowl. Data Eng.

Self Cite

View full text Add to dashboard Cite

Ensemble learning is a common tool for data stream classification, mainly because of its inherent advantages of handling large volumes of stream data and concept drifting. Previous studies, to date, have been primarily focused on building accurate ensemble models from stream data. However, a linear scan of a large number of base classifiers in the ensemble during prediction incurs significant costs in response time, preventing ensemble learning from being practical for many real-world time-critical data stream applications, such as Web traffic stream monitoring, spam detection, and intrusion detection. In these applications, data streams usually arrive at a speed of GB/second, and it is necessary to classify each stream record in a timely manner. To address this problem, we propose a novel Ensemble-tree (E-tree for short) indexing structure to organize all base classifiers in an ensemble for fast prediction. On one hand, E-trees treat ensembles as spatial databases and employ an R-tree like height-balanced structure to reduce the expected prediction time from linear to sub-linear complexity. On the other hand, E-trees can be automatically updated by continuously integrating new classifiers and discarding outdated ones, well adapting to new trends and patterns underneath data streams. Theoretical analysis and empirical studies on both synthetic and real-world data streams demonstrate the performance of our approach.

show abstract

Enabling Fast Lazy Learning for Data Streams

Cited by 47 publications

References 13 publications

KNN Classifier with Self Adjusting Memory for Heterogeneous Concept Drift

KNN Classifier with Self Adjusting Memory for Heterogeneous Concept Drift

The online performance estimation framework: heterogeneous ensemble learning for data streams

E-Tree: An Efficient Indexing Structure for Ensemble Models on Data Streams

Contact Info

Product

Resources

About