Concept drift detection via competence models

Ning, Lu; Zhang, Guangquan; Lu, Jie

doi:10.1016/j.artint.2014.01.001

Cited by 145 publications

(99 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Competence-based drift detection [6] compares two unknown, multi-dimensional, non-parametric data distribution in the competence space instead of the original data feature space. By modeling data into competence model, multidimensional data can be maintained in one-dimensional space.…”

Section: B Competence-based Drift Detection Methodsmentioning

confidence: 99%

“…The principle behind this is that the data feature space is partitioned using kdq-tree and apply Kulldorff's spatial scan statistic on nodes of kdq-tree. However, Lu, Zhang & Lu [6] argued that the partition made by kdq-tree does not guarantee that the regions of greatest change will coincide with the true interesting concepts, and the partition may not be easily explained and understood. Their competence-based drift detection method also quantitatively describes when, how and where data change takes place, and demonstrates good performance in different scenarios [6].…”

Section: A Concept Drift Detectionmentioning

confidence: 99%

“…However, Lu, Zhang & Lu [6] argued that the partition made by kdq-tree does not guarantee that the regions of greatest change will coincide with the true interesting concepts, and the partition may not be easily explained and understood. Their competence-based drift detection method also quantitatively describes when, how and where data change takes place, and demonstrates good performance in different scenarios [6]. The method they used to highlight drift-affected regions is based on competence models, called Top-P-Competence Areas (TPCA) [10].…”

Section: A Concept Drift Detectionmentioning

confidence: 99%

“…Little research endeavor has been made to explain how and where the data changes. Lu et al [6], [10] and Dasu et al [5] mention that their methods can locate the regions in which the data changes; however one issue should be addressed is that, the drift regions they identify are based on how their data model partitions data feature space. If the partitions are too coarse, the result will be imprecise.…”

Section: Introductionmentioning

confidence: 99%

“…Drift detection methods are techniques that explicitly identify the occurrence of concept drift. Many drift detection methods have been developed over the past decades, and they are typically implemented by monitoring 1) the underlying distribution of data [4]- [6]; 2) the output (error-rate) of the learner [3], [7]- [9]. Most existing drift detection methods focus on triggering a signal when drift is detected.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Concept drift region identification via competence-based discrepancy distribution estimation

Fan

et al. 2017

2017 12th International Conference on Intelligent Systems and Knowledge Engineering (ISKE)

Self Cite

View full text Add to dashboard Cite

Abstract-Real-world data analytics often involves cumulative data. While such data contains valuable information, the pattern or concept underlying these data may change over time and is known as concept drift. When learning under concept drift, it is essential to know when, how and where the context has evolved. Most existing drift detection methods focus only on triggering a signal when drift is detected, and little research has endeavored to explain how and where the data changes. To address this issue, we introduce kernel density estimation into competencebased drift detection method, and invent competence-based discrepancy distribution estimation to identify specific regions in the data feature space where drift has occurred. Two experiments demonstrate that our proposed approach, competence-based discrepancy density estimation, can quantitatively highlight drift regions through data feature space, and produce results that are very close to preset drift regions.

show abstract

Section: B Competence-based Drift Detection Methodsmentioning

confidence: 99%

Section: A Concept Drift Detectionmentioning

confidence: 99%

Section: A Concept Drift Detectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Concept drift region identification via competence-based discrepancy distribution estimation

Fan

et al. 2017

2017 12th International Conference on Intelligent Systems and Knowledge Engineering (ISKE)

Self Cite

View full text Add to dashboard Cite

show abstract

Adaptive flow abnormity identification based on information entropy

Liu

Fan

Wang

2018

Concurrency and Computation

View full text Add to dashboard Cite

SummaryAlong with the normal network flow, abnormal flow follows, threatening the security and normal use of the computer. Under the impact of the massive network flow, researchers were prompted to study identification methods based on machine learning. Currently, the existing identification methods based on machine learning still have deficiencies, ie, feature redundancy and deficiency in classifier. In order to deal with these problems, this paper proposes an adaptive approach (AEDs) to identify abnormal network flow. The AEDs utilizes the feature extraction algorithm proposed in this paper, which is based on information entropy to filter features and reduce the size of features. Then, a weight construction method based on rough set theory is implemented to construct the weights of given‐case‐based ensemble classifiers, and only the sub‐classifier with weight that is higher than the given threshold will be reserved. On this basis, in order to keep the effectiveness of the classifier, we utilize new flow data to update weights. Moreover, an adaptive detection algorithm is proposed to adaptively monitor the change of entropy of abnormal flow and enable the update of classifier. Finally, experiments have been conducted to illustrate the effectiveness and the feasibility of the approach.

show abstract

Exploration vs. Exploitation in Case-Base Maintenance: Leveraging Competence-Based Deletion with Ghost Cases

Leake

Schack

2018

Case-Based Reasoning Research and Development

View full text Add to dashboard Cite

Concept drift detection via competence models

Cited by 145 publications

References 25 publications

Concept drift region identification via competence-based discrepancy distribution estimation

Concept drift region identification via competence-based discrepancy distribution estimation

Adaptive flow abnormity identification based on information entropy

Exploration vs. Exploitation in Case-Base Maintenance: Leveraging Competence-Based Deletion with Ghost Cases

Contact Info

Product

Resources

About