An Operational Framework for Alert Correlation using a Novel Clustering Approach

Abstract

Intrusion Detection System (IDS) is a well known security feature and widely implemented among practitioners. However, since the creation of IDS the enormous number of alerts generated by the detection sensors has always been a setback in the implementation environment. Moreover due to this obtrusive predicament, two other problems have emerged which are the difficulty in processing the alerts accurately and also the decrease in performance rate in terms of time and memory capacity while processing these alerts. Thus, based on the specified problems, the purpose of our overall research is to construct a holistic solution that is able to reduce the number of alerts to be processed and at the same time to produce a high quality attack scenarios that are meaningful to the administrators in a timely manner. In this paper we will present our proposed framework together with the result of our novel clustering method, architectured solely with the intention of reducing the amount of alerts generated by IDS. The clustering method was tested against two dataset; a globally used dataset, DARPA and a live dataset from a cyber attack monitoring unit that uses SNORT engine to capture the alerts. The result obtained from the experiment is very promising; the clustering algorithm was able to reduce about 86.9% of the alerts used in the experiment. From the result we are able to highlight the contribution to practitioners in an actual working environment.

A taxonomy on intrusion alert aggregation techniques

“…According to Mohamed et al [40] they claimed that the data mining approaches that applied for the purpose of cluster are not suitable for two reasons:…”

“…Mohamed et al [40] and others extracted three attributes from alerts (destination IP, signature type or id and timestamp) and applied these attributes to MD5 hash function. The MD5 generates a unique hash value that is used for the initial clustering process.…”

“…Hashing function compress a string of arbitrary length to a string of fixed length and are used to allocate as uniformly as possible storage for the records of a file [40]. It is a powerful and simple technique for accessing information: Each key is mathematically converted into a number, which is then used as an index into a table.…”

False alarm minimization techniques in signature-based intrusion detection systems: A survey

“…Alarm Clustering: The idea is to group same alarms together [102] possibly generated from different sensors. A time window is normally considered within which alarms are merged.…”

An innovative framework for collaborative intrusion alert correlation

“…Their proposed framework and the novel clustering method, architectured solely with the intention of reducing the amount of alerts generated by IDS. The clustering method was tested against two datasets; a globally used dataset, DARPA and a live dataset from a cyber attack monitoring unit that uses Snort engine to capture the alerts [34].…”