IT Infrastructure Anomaly Detection and Failure Handling: A Systematic Literature Review Focusing on Datasets, Log Preprocessing, Machine &amp; Deep Learning Approaches and Automated Tool

Bhanage, Deepali Arun; Pawar, Ambika; Kotecha, Ketan

doi:10.1109/access.2021.3128283

Cited by 18 publications

(14 citation statements)

References 114 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This chapter outlines the research methods of the study. Further details on the implementation can be found via the replication package 1 .…”

Section: Methodsmentioning

confidence: 99%

“…A recent systematic literature review [1] analyses the qualitative and performance metrics of datasets, technical approaches and automated tools with an emphasis on failure detection and prediction. The review presents results from several studies that have utilized the same DL models and same public datasets used in this study.…”

Section: Related Workmentioning

confidence: 99%

“…Software system logs are considered as one of the primary sources for determining the cause of failure [1]. Log event prediction is one of the strategies of anomaly detection providing means for root cause detection as well as failure identification, tolerance, and recovery [1]. There is existing work that has focused on event prediction for anomaly detection e.g., [2]- [4].…”

Section: Introductionmentioning

confidence: 99%

“…A systematic literature review [8] shows that majority of the recent studies are based on computationally heavy approaches such as deep learning or utilize supervised models. Another literature review [9] notes the lack of real world use-cases and reports Loglizer 1 as the only available toolkit for log anomaly analysis.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

How to Configure Masked Event Anomaly Detection on Software Logs?

Nyyssala

Mäntylä

Varela

2022

2022 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

Software Log anomaly event detection with masked event prediction has various technical approaches with countless configurations and parameters. Our objective is to provide a baseline of settings for similar studies in the future. The models we use are the N-Gram model, which is a classic approach in the field of natural language processing (NLP), and two deep learning (DL) models long short-term memory (LSTM) and convolutional neural network (CNN). For datasets we used four datasets Profilence, BlueGene/L (BGL), Hadoop Distributed File System (HDFS) and Hadoop. Other settings are the size of the sliding window which determines how many surrounding events we are using to predict a given event, mask position (the position within the window we are predicting), the usage of only unique sequences, and the portion of data that is used for training. The results show clear indications of settings that can be generalized across datasets. The performance of the DL models does not deteriorate as the window size increases while the N-Gram model shows worse performance with large window sizes on the BGL and Profilence datasets. Despite the popularity of Next Event Prediction, the results show that in this context it is better not to predict events at the edges of the subsequence, i.e., first or last event, with the best result coming from predicting the fourth event when the window size is five. Regarding the amount of data used for training, the results show differences across datasets and models. For example, the N-Gram model appears to be more sensitive toward the lack of data than the DL models. Overall, for similar experimental setups we suggest the following general baseline: Window size 10, mask position second to last, do not filter out non-unique sequences, and use a half of the total data for training.

show abstract

“…This chapter outlines the research methods of the study. Further details on the implementation can be found via the replication package 1 .…”

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

How to Configure Masked Event Anomaly Detection on Software Logs?

Nyyssala

Mäntylä

Varela

2022

2022 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

show abstract

“…Logs são registros textuais de operac ¸ões ou erros realizados por um sistema em tempo de execuc ¸ão [Bhanage et al 2021]. De maneira similar a métricas, logs são coletados diretamente de processos em execuc ¸ão.…”

Section: Fundamentos Sobre Observabilidadeunclassified

Uma Investigação Empírica sobre Observabilidade em Sistemas 5G Nativos de Nuvem

Rodrigues

Bruno²,

Cardoso³

et al. 2022

Anais Do XL Simpósio Brasileiro De Redes De Computadores E Sistemas Distribuídos (SBRC 2022)

View full text Add to dashboard Cite

Ao migrar sofisticados recursos de hardware para componentes de software executados em sistemas de nuvem, as redes 5G proveem grande flexibilidade. No entanto, a softwarização das redes 5G impõe grandes desafios à gerência da rede, especialmente em relação à corretude do sistema em tempo de execução. Neste contexto, se torna essencial que o comportamento do sistema seja facilmente observável. Este trabalho apresenta uma investigação empírica sobre observabilidade em um sistema 5G formado pelas redes de núcleo e de acesso. Resultados mostram que a análise combinada de métricas e logs melhora a observabilidade do sistema e facilita a antecipação de ocorrências de falhas. Nossa investigação demonstra também que, apesar de úteis, as mensagens de logs geradas pelo sistema 5G implantado são pouco estruturadas e pouco inteligíveis.

show abstract