Mining high-speed data streams

Domingos, Pedro; Hulten, Geoff

doi:10.1145/347090.347107

Cited by 1,758 publications

(1,362 citation statements)

References 10 publications

Supporting

Mentioning

1,337

Contrasting

Unclassified

Order By: Relevance

“…The prominent representative of the former is the very fast decision tree (VFDT), originating from the Hoeffding tree algorithm (Domingos and Hulten, 2000), which induces a decision tree from a data stream incrementally, without the need for storing examples after they have been used to update the tree. It is based on exploiting the Hoeffding bound to select an attribute good enough for a split test in the tree nodes, which is done without viewing all the examples but guarantees the split to be correct at a user-specified probability.…”

Section: Classification Of Streamsmentioning

confidence: 99%

Exploring complex and big data

Stefanowski

Krawiec

Wrembel

2017

International Journal of Applied Mathematics and Computer Science

View full text Add to dashboard Cite

This paper shows how big data analysis opens a range of research and technological problems and calls for new approaches. We start with defining the essential properties of big data and discussing the main types of data involved. We then survey the dedicated solutions for storing and processing big data, including a data lake, virtual integration, and a polystore architecture. Difficulties in managing data quality and provenance are also highlighted. The characteristics of big data imply also specific requirements and challenges for data mining algorithms, which we address as well. The links with related areas, including data streams and deep learning, are discussed. The common theme that naturally emerges from this characterization is complexity. All in all, we consider it to be the truly defining feature of big data (posing particular research and technological challenges), which ultimately seems to be of greater importance than the sheer data volume.

show abstract

Section: Classification Of Streamsmentioning

confidence: 99%

Exploring complex and big data

Stefanowski

Krawiec

Wrembel

2017

International Journal of Applied Mathematics and Computer Science

View full text Add to dashboard Cite

show abstract

“…In addition, characteristics of data may change over time (concept drift). Here, supervised and unsupervised learning need to be adaptive to cope with changes [27,28,29,30]. There are two ways adaptive learning can be developed.…”

Section: Related Workmentioning

confidence: 99%

“…There are two ways adaptive learning can be developed. One is incremental learning [30] and the other one is ensemble-based learning [27,28,29]. Here is an example for incremental learning in user action prediction.…”

Section: Related Workmentioning

confidence: 99%

Supervised Learning for Insider Threat Detection Using Stream Mining

Parveen¹,

Weger²,

Thuraisingham³

et al. 2011

2011 IEEE 23rd International Conference on Tools With Artificial Intelligence

View full text Add to dashboard Cite

Abstract-Insider threat detection requires the identification of rare anomalies in contexts where evolving behaviors tend to mask such anomalies. This paper proposes and tests an ensemble-based stream mining algorithm based on supervised learning that addresses this challenge by maintaining an evolving collection of multiple models to classify dynamic data streams of unbounded length. The result is a classifier that exhibits substantially increased classification accuracy for real insider threat streams relative to traditional supervised learning (traditional SVM and one-class SVM) and other single-model approaches.

show abstract

“…AODEs perform well in most areas though classification time is quadratic in the number of attributes and it lacks the ability of incorporating prior information. An algorithm that performs well in terms of the first four requirements is the Hoeffding tree algorithm [2]. However, because it grows trees incrementally, it can require large amounts of memory.…”

Section: Bayesian Network Classifiers For Data Streamsmentioning

confidence: 99%

Voting Massive Collections of Bayesian Network Classifiers for Data Streams

Bouckaert

2006

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. We present a new method for voting exponential (in the number of attributes) size sets of Bayesian classifiers in polynomial time with polynomial memory requirements. Training is linear in the number of instances in the dataset and can be performed incrementally. This allows the collection to learn from massive data streams. The method allows for flexibility in balancing computational complexity, memory requirements and classification performance. Unlike many other incremental Bayesian methods, all statistics kept in memory are directly used in classification. Experimental results show that the classifiers perform well on both small and very large data sets, and that classification performance can be weighed against computational and memory costs.

show abstract

Mining high-speed data streams

Cited by 1,758 publications

References 10 publications

Exploring complex and big data

Exploring complex and big data

Supervised Learning for Insider Threat Detection Using Stream Mining

Voting Massive Collections of Bayesian Network Classifiers for Data Streams

Contact Info

Product

Resources

About