Combining Log Files and Monitoring Data to Detect Anomaly Patterns in a Data Center

Abstract

Context—Anomaly detection in a data center is a challenging task, having to consider different services on various resources. Current literature shows the application of artificial intelligence and machine learning techniques to either log files or monitoring data: the former created by services at run time, while the latter produced by specific sensors directly on the physical or virtual machine. Objectives—We propose a model that exploits information both in log files and monitoring data to identify patterns and detect anomalies over time both at the service level and at the machine level. Methods—The key idea is to construct a specific dictionary for each log file which helps to extract anomalous n-grams in the feature matrix. Several techniques of Natural Language Processing, such as wordclouds and Topic modeling, have been used to enrich such dictionary. A clustering algorithm was then applied to the feature matrix to identify and group the various types of anomalies. On the other side, time series anomaly detection technique has been applied to sensors data in order to combine problems found in the log files with problems stored in the monitoring data. Several services (i.e., log files) running on the same machine have been grouped together with the monitoring metrics. Results—We have tested our approach on a real data center equipped with log files and monitoring data that can characterize the behaviour of physical and virtual resources in production. The data have been provided by the National Institute for Nuclear Physics in Italy. We have observed a correspondence between anomalies in log files and monitoring data, e.g., a decrease in memory usage or an increase in machine load. The results are extremely promising. Conclusions—Important outcomes have emerged thanks to the integration between these two types of data. Our model requires to integrate site administrators’ expertise in order to consider all critical scenarios in the data center and understand results properly.

Anomaly Detection in Data Center IT & Physical Infrastructure

“…In previous work, we have focused on the detection of anomaly patterns by considering service log files and data coming from IT monitoring measurements, and leveraging natural language processing (NLP) solutions juxtaposed with multivariate time series anomaly detection techniques [1]. This study has revealed thousands of anomalies that have been verified by a comparison with the same log messages derived from the different services considered for the analysis.…”

“…This contribution brings work [1] further, exploring the statistical Z-score [2] and percentile [3] approaches, and Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise (DBSCAN) [4] in the anomaly detection field for time series numerical metrics related to IT and physical infrastructure sensors. The paper describes models defined by considering critical scenarios and a wide range and type of monitoring data.…”

Anomaly Detection in Data Center IT Infrastructure using Natural language Processing and Time Series Solutions

“…Starting from our initial study on log files aimed at building an anomaly dictionary and getting knowledge from files with the application of natural language processing (NLP) solutions and other ML techniques [1], we have continued combining a subset of log files and monitoring data information to detect anomaly patterns involving heterogeneous unstructured data [2]. NLP solutions have been applied to log files to identify anomalies from words and sequences of terms.…”