UniLog: Deploy One Model and Specialize it for All Log Analysis Tasks

Abstract

Log analysis is vitally important for network reliability, and many log-based tasks are derived to analysis logs, such as anomaly detection, failure prediction, log compression, and log summarization. It is desired to have a unified log analysis framework to simultaneously run all these log analysis tasks on one model to achieve deployment convenience, superior task performance, and low maintenance cost. However, due to severe challenges about log data heterogeneity across devices, pioneer works design specialized algorithms for each task. In this work, we formulate the log analysis as a multi-task learning approach and propose to train a single model that can perform various log analysis tasks. We name this unified log analysis approach as UniLog. To effectively build an UniLog model, we propose a log data pretrained transformer to utilize the enormous unlabeled log data, and a corresponding multi-log-tasking finetune strategy for various log analysis tasks. Extensive experiments across seven datasets on four log analysis tasks demonstrate that UniLog achieves remarkable performance.

Interactive Log Parsing via Light-weight User Feedbacks

“…Nulog [22] casts the parsing task as a masked language modeling (MLM) problem and uses a self-supervised learning model to address it. UniLog 2021 [36] casts the log analysis problem as a multi-task learning problem and proposes a log data pretrained transformer to parse logs. LogDTL [23] is a semi-supervised method.…”