Anomaly Detection Using Data Mining Methods in IT Systems: A Decision Support Application

Abstract

Although there are various studies on anomaly detection, effective and simple anomaly detection approaches are necessary as the inadequacy of appropriate ways for substantial network environments. In the existing analysis methods, it is seen that the methods of preliminary analysis are generally used, the extrapolations and probabilities are not taken into account and the unsupervised neural network (NN) methods are not used enough. As an alternative, the use of the Self-Organizing Maps has been preferred in the study. In other studies, analysis of data obtained from network traffic is analyzed, here, analysis of other information systems data and suggestions for alternative solutions are given, too. In addition, in-memory database systems have been used in practice in order to enable faster processing in analysis studies, due to the large size of data to be analyzed in large-scale network environments. An analysis of the application log data obtained from the management tools in the information systems was carried out. After anomaly detection results obtained and the verification test results are compared, it is found out that anomaly detection process is successful by 96%. The advantage offered for the company and users at IT and security monitoring processes is to eliminate the need for pre-qualification and to reduce the heavy workload. By this way, it is thought that a significant cost item is eliminated. It is also contemplated that the security vulnerabilities and problems associated with unpredictable issues will be detected through practice and thus many attacks and problems will be prevented in advance.

Integrating Traffics with Network Device Logs for Anomaly Detection

“…To validate the performance of integrating traffics with logs for anomaly detection, we conduct comparison experiments through only leveraging the traffic data (or the log data). As displayed in Tables 5-8 and Figures 3-6, it is clear that neither traffics nor logs can independently achieve desirable results in detecting cyberattacks (both the False Negative (FN) and False Positive (FP) values decrease significantly), which is consistent with [16]. On the contrast, when we integrate the traffic flows with network device logs, the detection performance can be significantly improved.…”

“…However, due to the complexity of the behaviors from different networks and applications, it is difficult to accurately identify the normal behaviors. The existing anomaly detection methods are usually based on device logs or traffic flows alone [16]. In general, their methods are too simple to achieve satisfactory results [17].…”

Online Anomaly Detection with Sparse Gaussian Processes

“…Online anomaly detection is significant because abnormal data can often convey essential and critical information in an extensive range of applications [4,5,6,7,8,9,10,11,12,13,14]. For example, when abnormal traffic occurs in a computer network, a hacker may be using the attacked computer to send sensitive data to the target computer [15].…”