Fading histograms in detecting distribution and concept changes

Sebastião, Raquel; Gama, João; Mendonça, Teresa

doi:10.1007/s41060-017-0043-4

Cited by 11 publications

(8 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In many of these situations, accuracy may not be the most important metric, and, because of the unavailability of the real labels or by the urgency of taking action, it is more interesting to detect a drift as soon as possible. For example, in cases of detection of diseases or epidemics, actions must be taken after the occurrence of the change, and it is essential to detect a drift as soon as possible, ideally immediately after it occurs [31]. In these problems, proactive approaches are possibly the best choice since they can detect some early drifts and avoid serious problems.…”

Section: Results Of the Experimentsmentioning

confidence: 99%

DetectA: abrupt concept drift detection in non-stationary environments

Escovedo

Koshiyama

Cruz³

et al. 2018

Applied Soft Computing

View full text Add to dashboard Cite

Almost all drift detection mechanisms designed for classification problems work reactively: after receiving the complete data set (input patterns and class labels) they apply a sequence of procedures to identify some change in the class-conditional distributiona concept drift. However, detecting changes after its occurrence can be in some situations harmful to the process under analysis. This paper proposes a proactive approach for abrupt drift detection, called DetectA (Detect Abrupt Drift). Briefly, this method is composed of three steps: (i) label the patterns from the test set (an unlabelled data block), using an unsupervised method; (ii) compute some statistics from the train and test sets, conditioned to the given class labels for train set; and (iii) compare the training and testing statistics using a multivariate hypothesis test. Based on the results of the hypothesis tests, we attempt to detect the drift on the test set, before the real labels are obtained. A procedure for creating datasets with abrupt drift has been proposed to perform a sensitivity analysis of the DetectA model. The result of the sensitivity analysis suggests that the detector is efficient and suitable for datasets of high-dimensionality, blocks with any proportion of drifts, and datasets with class imbalance. The performance of the DetectA method, with different configurations, was also evaluated on real and artificial datasets, using an MLP as a classifier. The best results were obtained using one of the detection methods, being the proactive manner a top contender regarding improving the underlying base classifier accuracy.

show abstract

Section: Results Of the Experimentsmentioning

confidence: 99%

DetectA: abrupt concept drift detection in non-stationary environments

Escovedo

Koshiyama

Cruz³

et al. 2018

Applied Soft Computing

View full text Add to dashboard Cite

show abstract

“…The current study discusses developments in the synopsis structure. In particular, researchers have proposed the adaptive cumulative windows model (ACWM) algorithm, which summarizes data streams using histograms [96].…”

Section: B Summarizationmentioning

confidence: 99%

A Systematic Review of Density Grid-Based Clustering for Data Streams

et al. 2022

View full text Add to dashboard Cite

Various applications, such as electronic business, satellite remote sensing, intrusion discovery, and network traffic monitoring, generate large unbounded data stream sequences at a rapid pace. The clustering of data streams has attracted considerable interest due to the increasing usage of evolving data streams. In particular, evolving data streams affect clustering because they introduce numerous challenges, such as time and memory limits and one-pass clustering. Furthermore, researchers need to be able to determine arbitrarily shaped clusters present in evolving data streams from applications. Due to these characteristics, conventional density grid-based clustering techniques cannot be used. Moreover, the existing density gridbased clustering algorithms have low cluster quality for clustering evolving data streams. This study conducted a systematic literature review (SLR) and noted numerous research-related issues encountered in solving the aforementioned problems. We summarized numerous grid-based clustering algorithms that have been used and determined their distinctive and limited features. We also observed how these algorithms address the challenges affecting the clustering of evolving data streams and studied their advantages and disadvantages. SLR was based on 104 articles published between 2010 and 2021. Numerous challenges remain for grid-based clustering algorithms, particularly in terms of time-limited and high-dimensional data handling. Last, our findings indicated a variety of active studies on density grid-based clustering.INDEX TERMS Clustering, data stream, grid-based clustering, data stream clustering, density-based clustering.

show abstract

“…Only then, the reactive detector applies a sequence of procedures to identify some change in the conditional class distribution -a concept drift. Examples of reactive detectors can be found in [14,5,36,4,42,3,31,23,13,46].…”

Section: Concept Driftmentioning

confidence: 99%

Neuroevolutionary learning in nonstationary environments

Escovedo

Koshiyama

Cruz³

et al. 2020

Appl Intell

View full text Add to dashboard Cite

This work presents a new neuro-evolutionary model, called NEVE (Neuroevolutionary Ensemble), based on an ensemble of Multi-Layer Perceptron (MLP) neural networks for learning in nonstationary environments. NEVE makes use of quantuminspired evolutionary models to automatically configure the ensemble members and combine their output. The quantuminspired evolutionary models identify the most appropriate topology for each MLP network, select the most relevant input variables, determine the neural network weights and calculate the voting weight of each ensemble member. Four different approaches of NEVE are developed, varying the mechanism for detecting and treating concepts drifts, including proactive drift detection approaches. The proposed models were evaluated in real and artificial datasets, comparing the results obtained with other consolidated models in the literature. The results show that the accuracy of NEVE is higher in most cases and the best configurations are obtained using some mechanism for drift detection. These results reinforce that the neuroevolutionary ensemble approach is a robust choice for situations in which the datasets are subject to sudden changes in behaviour.

show abstract

Fading histograms in detecting distribution and concept changes

Cited by 11 publications

References 30 publications

DetectA: abrupt concept drift detection in non-stationary environments

DetectA: abrupt concept drift detection in non-stationary environments

A Systematic Review of Density Grid-Based Clustering for Data Streams

Neuroevolutionary learning in nonstationary environments

Contact Info

Product

Resources

About