Large scale log anomaly detection via spatial pooling

Hirakawa, Rin; Uchida, Hironori; Nakano, Asato; Tominaga, Keitaro; Nakatoh, Yoshihisa

doi:10.1016/j.cogr.2021.10.001

Cited by 7 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this experiment, we used a total of five models, consisting of two supervised learning models, namely Convolutional Neural Network (CNN) [6] and Long Short-Term Memory (LSTM) [7], and two unsupervised learning models, namely Auto Encoder (AE) [8] and Transformer [9], in addition to our proposed SPClassifier [10]. An overview of each model and the Toolkit parameters used in this study is provided in Section 2.2.1-2.2.5.…”

Section: Evaluated Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Verification of Generalizability in Software Log Anomaly Detection Models

Uchida¹,

Tominaga²,

Itai³

et al. 2024

Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

With recent rapid technological advances, the automatic analysis of software logs has received particular attention. Currently, there is much research on the use of Deep Learning in the field of software log anomaly detection, and they have reported high accuracy of more than 0.9 in the f1-score. On the other hand, there are reports that it has not been used in the field of software development. We conducted a generalized evaluation against representative models for log anomaly detection to elucidate the cause of this problem. Five models were used in the subject: four representative models (two supervised and two unsupervised) and our proposed Neocortical Algorithm (supervised). We used the commonly used Blue Gene/L supercomputer log(BGL) dataset. The learning curves and cross-validation showed a tendency toward overfitting in all models. In addition, a survey of the frequency of log patterns confirmed the need for a more diverse dataset, as many of the patterns are a series of specific logs.

show abstract

Section: Evaluated Modelsmentioning

confidence: 99%

“…SPClassifier is a model with sparse features and internal representations suitable for training in CPU environments [10]. The proposed method consists of one spatial pooling layer [11,12] and one Feedforward Neural Network for classification, which identifies anomalous patterns from log data transformed into 2D features.…”

Section: Spclassifier (Supervised Model)mentioning

confidence: 99%

Verification of Generalizability in Software Log Anomaly Detection Models

Uchida¹,

Tominaga²,

Itai³

et al. 2024

Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

show abstract

“…Araştırmacılar tarafından 2021 yılında gerçekleştirilen geniş ölçekli anomali tespiti için mekânsal havuzlama yöntemi kullanan bir çalışmada sonuçların test edilmesi için Blue Gene/L veri setinden elde edilen ve 4,747,963 log kaydı içeren BGL veri seti kullanılmıştır [12]. 2020 yılında gerçekleştirilen bir çalışmada araştırmacılar tarafından loglar üzerinde denetimsiz anomali tespiti modeli Thunderbird veri seti üzerinde gerçekleştirilmiştir [13]. 2017 yılında gerçekleştirilmiş bir araştırmada RedStorm veri setinin makine öğrenimi kullanılarak HPC (High Performance Computer) sistemleri uygulamalarındaki performans değişikliklerinin tanılanması konusunda kullanıldığı görülmektedir [14].…”

Section: İlgili çAlışmalarunclassified

Big Data Visualization for Cyber Security: BETH Dataset

DOĞANAY

Orman

Dener

2022

ECJSE

View full text Add to dashboard Cite

In this study, the literature on big data visualization for cyber security purposes was scanned and a purposeful data visualization study was carried out on a sample data set. When the visualization study carried out is compared with its counterpart in the literature, it reveals that if the visualization with the criteria suggested in this study is applied, the user (human) can read the graphics much more easily and it will be a facilitating way for attack detection. The criteria in the study are based on the use of current data sets such as BETH and the use of methods such as Principle Component Analysis (PCA).

show abstract

Log message anomaly detection with fuzzy C-means and MLP

Farzad

Gulliver

2022

Appl Intell

View full text Add to dashboard Cite

Large scale log anomaly detection via spatial pooling

Cited by 7 publications

References 7 publications

Verification of Generalizability in Software Log Anomaly Detection Models

Verification of Generalizability in Software Log Anomaly Detection Models

Big Data Visualization for Cyber Security: BETH Dataset

Log message anomaly detection with fuzzy C-means and MLP

Contact Info

Product

Resources

About