SPINE: a scalable log parser with feedback guidance

Wang, Xuheng; Zhang, Xu; Li, Liqun; He, Shilin; Zhang, Hongyu; Liu, Yudong; Zheng, Lei; Kang, Yu; Lin, Qingwei; Dang, Yingnong; Rajmohan, Saravanakumar; Zhang, Dongmei

doi:10.1145/3540250.3549176

Cited by 26 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2) Frequent pattern mining: SLCT [25], LFA [16], and Logram [15] leverage frequent patterns of token position or n-gram information to extract log templates that appear constantly across log messages. 3) Heuristicsbased searching: Drain [10], Spell [9], SwissLog [17], and SPINE [26] utilize a tree structure to parse logs into multiple templates. 4) Deep learning based parsing: UniParser [20] formulate log parsing as a token classification problem and LogPPT [18] leverages language models to perform log parsing in few-shot scenarios.…”

Section: B Log Parsingmentioning

confidence: 99%

“…We compare our proposed method with five state-of-theart log parsers, including AEL [34], Spell [9], Drain [10], Logram [15], and SPINE [26]. We choose these five parsers in our evaluation since their source code is publicly available; and a prior study [11], [32] finds that these parsers have high accuracy and efficiency among the evaluated log parsing methods.…”

Section: Baselinesmentioning

confidence: 99%

See 1 more Smart Citation

Log-based Anomaly Detection Without Log Parsing

Zhang

2021

2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)

126

View full text Add to dashboard Cite

Logs generated by large-scale software systems provide crucial information for engineers to understand the system status and diagnose problems of the systems. Log parsing, which converts raw log messages into structured data, is the first step to enabling automated log analytics. Existing log parsers extract the common part as log templates using statistical features. However, these log parsers often fail to identify the correct templates and parameters because: 1) they often overlook the semantic meaning of log messages, and 2) they require domain-specific knowledge for different log datasets. To address the limitations of existing methods, in this paper, we propose LogPPT to capture the patterns of templates using prompt-based few-shot learning. LogPPT utilises a novel prompt tuning method to recognise keywords and parameters based on a few labelled log data. In addition, an adaptive random sampling algorithm is designed to select a small yet diverse training set. We have conducted extensive experiments on 16 public log datasets. The experimental results show that LogPPT is effective and efficient for log parsing.

show abstract

Section: B Log Parsingmentioning

confidence: 99%

Section: Baselinesmentioning

confidence: 99%

Log-based Anomaly Detection Without Log Parsing

Zhang

2021

2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)

126

View full text Add to dashboard Cite

show abstract

“…While frequent pattern mining-based log parsing methods demonstrate competitive parsing efciency, they struggle to match rare logs with low frequency to any log template, resulting in suboptimal parsing results [10][11][12]. Clustering-based log parsing methods often sufer from low parsing accuracy due to their simplistic parsing patterns (e.g., dividing log groups based on word frequency or diferent word types) [2,[13][14][15]. In comparison to static code or frequent pattern miningbased methods, clustering methods have slower parsing speeds and require numerous iterations.…”

Section: Introductionmentioning

confidence: 99%

DLLog: An Online Log Parsing Approach for Large-Scale System

Cheng,

Ying,

Duan

et al. 2024

International Journal of Intelligent Systems

View full text Add to dashboard Cite

Syslog is a critical data source for analyzing system problems. Converting unstructured log entries into structured log data is necessary for effective log analysis. However, existing log parsing methods demonstrate promising accuracy on limited datasets, but their generalizability and precision are uncertain when applied to diverse log data. Enhancements in these areas are necessary. This paper proposes an online log parsing method called DLLog, which is based on deep learning and has the longest common subsequence. DLLog utilizes the GRU neural network to mine template words and applies the longest common subsequence to parse log entries in real-time. In the offline stage, DLLog combines multiple log features to accurately extract the template words, creating a log template set to assist online log parsing. In the online stage, DLLog parses log entries by calculating the matching degree between the real-time log entry and the log template in the log template set. This method also supports the incremental update of the log template set to handle new log entries generated by systems. We summarized the previous works and validated DLLog using real log data collected from 16 systems. The results demonstrate that DLLog achieves high parsing accuracy, universality, and adaptability.

show abstract

A Survey on the Usage of Centrality Measures in Error Prediction

Marjai,

Kiss

2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

SPINE: a scalable log parser with feedback guidance

Cited by 26 publications

References 29 publications

Log-based Anomaly Detection Without Log Parsing

Log-based Anomaly Detection Without Log Parsing

DLLog: An Online Log Parsing Approach for Large-Scale System

A Survey on the Usage of Centrality Measures in Error Prediction

Contact Info

Product

Resources

About