CDDM: Concept Drift Detection Model for Data Stream

Abstract

<p>Data stream is the huge amount of data generated in various fields, including financial processes, social media activities, Internet of Things applications, and many others. Such data cannot be processed through traditional data mining algorithms due to several constraints, including limited memory, data speed, and dynamic environment. Concept Drift is known as the main constraint of data stream mining, mainly in the classification task. It refers to the change in the data stream underlining distribution over time. Thus, it results in accuracy deterioration of classification models and wrong predictions. Spam emails, consumer behavior changes, and adversary activates, are examples of Concept Drift. In this paper, a Concept Drift detection model is introduced, Concept Drift Detection Model (CDDM). It monitors the accuracy of the classification model over a sliding window, assuming the decline in accuracy indicates a drift occurrence. A modification over CDDM is a weighted version of the CDDM as W-CDDM.</p><p>Both models have evaluated against two real datasets and four artificial datasets. The experimental results of abrupt drift show that CDDM, W-CDDM outperforms the other models in the dataset of 100K and 1M instances, respectively. Regarding gradual drift, the W-CDDM overtook the rest in terms of accuracy, run time, and detection delays in the dataset of 100 K instances. While in the dataset of 1M instances, CDDM has got the highest accuracy using the NB classifier. Moreover, W-CDDM achieves the highest accuracy on real datasets.</p>

Analytical Engineering for Data Stream

“…Data stream mining uses specific methods and tools to perform mining activities that cannot be processed by traditional Data Mining tools due to some constraints, such as limited memory, data speed and dynamic environment [27].…”

“…Concept drift is a specific constraint for data stream mining that considers the changes observed in the stream, altering the distribution over time and impairing results and accuracy, especially for classification models [27] [41] [44] [45].…”

“…For AP, some special features must be provided by these tools. According to [27] [51] [52] concept drift is a relevant practice linked to data stream solutions and systems.…”

Public Sender Score System (S3) by ESPs for Email Spam Mitigation with Score Management in Mobile Application

“…a) DMARC: There are various ways in which the sender is notified about their reputation or email deliverability. Domain-based Message Authentication, Reporting, and Conformance (DMARC) method, which was discussed before, also allows the sender to get reports of their email delivery through an email which is specified in the DMARC DNS TXT record [3]. b) Feedback Loops: There are Feedback Loops (FBL) [7] [8] [9] provided by some Mailbox Providers which will be useful for the marketing Email Service Providers.…”

“…There are clickbait detection mechanisms like [2] that identifies spammers too. Some content differs from the subject and the spammers insert spam content in between the email content which can be identified by mechanisms like [3]. There are a lot of other mechanisms in place by the email providers in order to identify spam in the email content.…”