SMARTLOG: Place error log statement by deep understanding of log intention

Jia, Zhouyang; Li, Shanshan; Liu, Xiaodong; Liao, Xiangke; Liu, Yunhuai

doi:10.1109/saner.2018.8330197

Cited by 32 publications

(20 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RQ1 (model performance in a large-scale system): We use the Adyen dataset (see Table II) and apply the process and learning algorithms described in Section III-C. Unfortunately, the related work either lacks publicly available implementation [11], [12] or requires a full re-implementation compatible with our Java code base [10], [16]. For this reason, we use two probabilistic baselines: random guess with p = 50.0% and biased guess with p = 7.7% for the positive class.…”

Section: Discussionmentioning

confidence: 99%

“…The research community has been proposing techniques to support developers in deciding what parts of the system to log. For instance, Jia et al [16] proposed an approach based on association rule mining to place error logs on if statements; Li et al [12] studied the use of topic modeling for log placement at method-level, and; Li et al [11] proposed a deep learning-based approach to indicate the need for logging at block-level. Those techniques rely mostly on code vocabulary to learn placement patterns in the source code, and experiments with open-source data show their relevance on the placement problem.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Exploratory Study of Log Placement Recommendation in an Enterprise System

Cândido

Haesen²,

Aniche

et al. 2021

2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)

View full text Add to dashboard Cite

Logging is a development practice that plays an important role in the operations and monitoring of complex systems. Developers place log statements in the source code and use log data to understand how the system behaves in production. Unfortunately, anticipating where to log during development is challenging. Previous studies show the feasibility of leveraging machine learning to recommend log placement despite the data imbalance since logging is a fraction of the overall code base. However, it remains unknown how those techniques apply to an industry setting, and little is known about the effect of imbalanced data and sampling techniques. In this paper, we study the log placement problem in the code base of Adyen, a large-scale payment company. We analyze 34,526 Java files and 309,527 methods that sum up +2M SLOC. We systematically measure the effectiveness of five models based on code metrics, explore the effect of sampling techniques, understand which features models consider to be relevant for the prediction, and evaluate whether we can exploit 388,086 methods from 29 Apache projects to learn where to log in an industry setting. Our best performing model achieves 79% of balanced accuracy, 81% of precision, 60% of recall. While sampling techniques improve recall, they penalize precision at a prohibitive cost. Experiments with opensource data yield under-performing models over Adyen's test set; nevertheless, they are useful due to their low rate of false positives. Our supporting scripts and tools are available to the community.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Exploratory Study of Log Placement Recommendation in an Enterprise System

Cândido

Haesen²,

Aniche

et al. 2021

2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)

View full text Add to dashboard Cite

show abstract

“…The data of all the Stack Exchange websites are publically available under cc-by-sa 4.0 license. 8 We downloaded the dataset of all the six websites and preprocessed it. We first had to identify all the logging questions from these websites.…”

Section: Dataset Extractionmentioning

confidence: 99%

An exploratory semantic analysis of logging questions

Gujral

Lal

2021

J Software Evolu Process

View full text Add to dashboard Cite

Logging is an integral part of software development. Software practitioners often face issues in software logging, and they post these issues on Q&A websites to take suggestions from the experts. In this study, we perform a three-level empirical analysis of logging questions posted on six popular technical Q&A websites, namely, Stack Overflow (SO), Serverfault (SF), Superuser (SU), Database Administrators (DB), Software Engineering (SE), and Android Enthusiasts (AE). The findings show that logging issues are prevalent across various domains, for example, database, networks, and mobile computing, and software practitioners from different domains face different logging issues. The semantic analysis of logging questions using Latent Dirichlet Allocation (LDA) reveals trends of several existing and new logging topics, such as logging conversion pattern, Android device logging, and database logging. In addition, we observe specific logging topics for each website: DB (log shipping and log file growing/ shrinking), SU (event log and syslog configuration), SF (log analysis and syslog configuration), AE (app install and usage tracking), SE (client server logging and exception logging), and SO (log file creation/deletion, Android emulator logging, and logger class of Log4j).We obtain an increasing trend of logging topics on the SO, SU, and DB websites whereas a decreasing trend of logging topics on the SF website.

show abstract

“…Bao et al [26] proposed a universal method to detect anomalies for large-scale software systems by analyzing their console logs based on probabilistic suffix trees. Jia et al [31] designed SmartLog to place error log statement properly by exploring the intentions of logs and mining log mechanisms with identical intentions, which can provide high-efficiency guidance for diagnosis and troubleshooting. Maikel et al [32] proposed Statechart Workbench, a novel tool that supports users to conveniently obtain an accurate and reliable understanding of the observed software system at multiple levels of granularity.…”

Section: B Related Workmentioning

confidence: 99%

“…Xu et al [6], [20] proposed a log-based methodology to detect problems without any prior knowledge from operators as input or any system instrumentation that is incapable of understanding the meaning of logs generated from the nuclear power industry. Sherlog [4], Elas-ticSearch [28], SmartLog [31], Statechart Workbench [32] and other methods [6], [20], [26] have been designed for fault diagnosis and troubleshooting; however, the above tools and approaches make a common assumption that the source code of the system is available. For intellectual property and commercial reasons, nuclear power system vendors do not provide the source code; therefore, our NTH methodology requires no source code.…”

Section: Introductionmentioning

confidence: 99%

Nowhere to Hide Methodology: Application of Clustering Fault Diagnosis in the Nuclear Power Industry

et al. 2019

View full text Add to dashboard Cite

When a system crashes, fast and accurate log-based fault diagnosis can remarkably reduce the recovery time of the system and avoid further economic losses. Especially for the nuclear power industry, recovery time will lead not only to economic losses but also to international repercussions. Nevertheless, the massive quantity of obscure log information and the existence of hidden nodes pose major challenges to fault diagnosis and root cause determination. To overcome these obstacles, we propose the nowhere to hide (NTH) methodology, an efficient method to diagnose faults and locate root causes. We implement log-node and node-log mapping to avoid vital data loss in collecting fault logs and hidden nodes; furthermore, we utilize the logic of the nuclear power unit process system to reveal the crucial information in fault logs and hidden nodes and their causality to determine the root cause. We evaluate the methodology in a real nuclear industrial environment. The results show that system administrators can efficiently determine the root cause with the proposed methodology. Finally, we discuss the enhancements that are underway to improve the methodology.INDEX TERMS Clustering fault diagnosis, key log, distributed control system (DCS), hidden target nodes (HTNs), hidden nontarget nodes (HNNs), detectable target nodes (DTNs), detectable nontarget nodes (DNNs), topological graph, mapping, fault log correlation analysis, and logic correlation analysis.

show abstract

SMARTLOG: Place error log statement by deep understanding of log intention

Cited by 32 publications

References 21 publications

An Exploratory Study of Log Placement Recommendation in an Enterprise System

An Exploratory Study of Log Placement Recommendation in an Enterprise System

An exploratory semantic analysis of logging questions

Nowhere to Hide Methodology: Application of Clustering Fault Diagnosis in the Nuclear Power Industry

Contact Info

Product

Resources

About