Adaptive boosting in ensembles for outlier detection: Base learner selection and fusion via local domain competence

Abstract

Unusual data patterns or outliers can be generated because of human errors, incorrect measurements, or malicious activities. Detecting outliers is a difficult task that requires complex ensembles. An ideal outlier detection ensemble should consider the strengths of individual base detectors while carefully combining their outputs to create a strong overall ensemble and achieve unbiased accuracy with minimal variance. Selecting and combining the outputs of dissimilar base learners is a challenging task. This paper proposes a model that utilizes heterogeneous base learners. It adaptively boosts the outcomes of preceding learners in the first phase by assigning weights and identifying high‐performing learners based on their local domains, and then carefully fuses their outcomes in the second phase to improve overall accuracy. Experimental results from 10 benchmark datasets are used to train and test the proposed model. To investigate its accuracy in terms of separating outliers from inliers, the proposed model is tested and evaluated using accuracy metrics. The analyzed data are presented as crosstabs and percentages, followed by a descriptive method for synthesis and interpretation.

OAAE: Optimized adaptive anomaly detection ensemble—Base model boosting by parameter optimization

“…The method in Reference 53, however, does not include adaptive base model training. OAAE's closely related work, ADAHO, 9 includes adaptive base model training but does not take into account anomaly scores margin maximization, which is the focus of OAAE. In Reference 9, the focus is on base models' local neighborhood testing, while OAAE's focus is strongly on score margin maximization, so that a clear contrast between the anomalies and normal data is generated.…”

“…OAAE's closely related work, ADAHO, 9 includes adaptive base model training but does not take into account anomaly scores margin maximization, which is the focus of OAAE. In Reference 9, the focus is on base models' local neighborhood testing, while OAAE's focus is strongly on score margin maximization, so that a clear contrast between the anomalies and normal data is generated. Both algorithms, however, take into account the fact that anomalies can be found in a far‐off or local region depending on the nearest neighbor distance or density.…”

“…Hence, both methods use base models that consider the distance and the density of a test point. Furthermore, both algorithms use different kinds of base models, that is, heterogeneous, but while 9 use four base models, OAAE trains two to detect. In Reference 9, the accuracy of a base model is tested not on the whole testing set but strictly only on the local regions where that model is good at classifying.…”

“…The problem of identifying anomalous events, often known as anomalies, has received a lot of attention in recent years 1–5,8–10,13,14,20 . Anomaly detection is the process of recognizing these unusual events in a dataset (based on a specific definition of an abnormal event) 21 .…”

“…Furthermore, there have been reports of some unsupervised approaches generating too many false positives or negatives, especially when using a single unsupervised model, which is dangerous 8 . Therefore, to increase the accuracy of these approaches, multiple models rather than single model formations have been used 9–13 . In addition, the models may be heterogeneous, which means they are made up of different base algorithms, tweaked parameters, or varying data sub‐samples 14 .…”