Knowledge discovery from data streams

Gama, João; Ganguly, Auroop R.; Omitaomu, Olufemi A.; Vatsavai, Ranga Raju; Gaber, Mohamed Medhat

doi:10.3233/ida-2009-0372

Cited by 132 publications

(263 citation statements)

References 0 publications

Supporting

Mentioning

212

Contrasting

Unclassified

Order By: Relevance

“…Alternatively, [50,51] suggest decaying older samples in favor of more recent samples. This approach, however, does have a drawback: a small time window, while ensuring fast adaptability in periods with changes, can decrease performance in more stable periods, whereas a large time window, while producing good results in periods of stability, cannot react quickly to changes [52].…”

Section: Related Worksupporting

confidence: 55%

Fast and accurate detection of changes in data streams

Badarna

Wolff

2014

Statistical Analysis

View full text Add to dashboard Cite

Change detection is one of the most important tasks in time series analysis. When the series is very long, or when it is rapidly updated, it has to be treated as a stream. This means that the change detection algorithm must process each sample in O (1) time and memory. A good algorithm must be generic in terms of the type of changes it can detect. Beyond all, a good algorithm must present a favorable and controlled ratio of the number of samples needed to detect a change to the rate of false positives. We present a change‐point detection algorithm called ProTO which dynamically manages a set of candidate change‐points whose expected size is a controllable constant. In terms of sample processing, ProTO is comparable with the fastest known algorithm—the Page‐Hinkley Test (PHT). Yet, because PHT is limited to just one candidate, ProTO outperforms it in terms of the ratio of the delay to the false positive rate, as well as in terms of robustness. We provide variants of ProTO for detecting changes in the mean or the variance of the stream, and experiment with two realistic applications, as well as with synthetic data. On real problems, ProTO compares favorably with state‐of‐the‐art algorithms implemented in the R‐package, which require more than O (1) time per sample.

show abstract

Section: Related Worksupporting

confidence: 55%

Fast and accurate detection of changes in data streams

Badarna

Wolff

2014

Statistical Analysis

View full text Add to dashboard Cite

show abstract

“…Therefore, there will be a great demand for tools allowing to gain value from this data (Rodríguez-Mazahua et al, 2016). Especially the analysis of data streams comes with new challenges such as non-stationary characteristics of the data and dynamic environments including new tools such as MOA (Bifet et al, 2018;Gama, 2010). A survey on data stream analysis with a focus on ensembles can be found in the article of Krawczyk, Minku, Gama, Stefanowski, & Woźniak (2017).…”

Section: Historical Development and State-of-the-artmentioning

confidence: 99%

Data mining tools

Bartschat

Reischl

Mikut

2019

WIREs Data Min & Knowl

View full text Add to dashboard Cite

The development and application of data mining algorithms requires the use of powerful software tools. With challenges such as big data encountered in economy or gene sequencing for life science, data mining is important for daily problems as well as specialized fields. However, the large variety of requirements and user groups lead to a huge number and diversity of software tools. We give an overview by discussing the historical development and presenting a range of existing state‐of‐the‐art data mining and related tools. This paper is an update of our previous article from 2011 following the encyclopedic aspect of Wiley Interdisciplinary Reviews to include new findings or references and changing outdated information. However, since the paper should be able to stand alone, it includes many still valid elements of the previous article. Following the original paper, we propose criteria for the tool categorization based on different user groups, data structures, data mining tasks and methods, visualization and interaction styles, import and export options for data and models, platforms, and license policies. These criteria are then used to classify data mining tools into nine different categories. The typical characteristics of these types are explained and a selection of the most important tools is categorized. This article is categorized under: Application Areas > Data Mining Software Tools

show abstract

“…Although incremental learning (IL) of data streams is well-established in the literature [1], one of the major limitations in the existing approaches lies in their fully supervised nature in which the true class label has to become available immediately after receiving a data point. Some delay is expected in associating the target class to the incoming sample [2].…”

Section: Introductionmentioning

confidence: 99%

Weakly Supervised Deep Learning Approach in Streaming Environments

Pratama

Ashfahani

Hady

2019

2019 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

The feasibility of existing data stream algorithms is often hindered by the weakly supervised condition of data streams. A self-evolving deep neural network, namely Parsimonious Network (ParsNet), is proposed as a solution to various weakly-supervised data stream problems. A self-labelling strategy with hedge (SLASH) is proposed in which its auto-correction mechanism copes with the accumulation of mistakes significantly affecting the model's generalization. ParsNet is developed from a closed-loop configuration of the self-evolving generative and discriminative training processes exploiting shared parameters in which its structure flexibly grows and shrinks to overcome the issue of concept drift with/without labels. The numerical evaluation has been performed under two challenging problems, namely sporadic access to ground truth and infinitely delayed access to the ground truth. Our numerical study shows the advantage of ParsNet with a substantial margin from its counterparts in the high-dimensional data streams and infinite delay simulation protocol. To support the reproducible research initiative, the source code of ParsNet along with supplementary materials are made available at https://bit.ly/2qNW7p4.

show abstract

Knowledge discovery from data streams

Cited by 132 publications

References 0 publications

Fast and accurate detection of changes in data streams

Fast and accurate detection of changes in data streams

Data mining tools

Weakly Supervised Deep Learning Approach in Streaming Environments

Contact Info

Product

Resources

About