An empirical assessment of machine learning approaches for triaging reports of static analysis tools

Mordahl, Austin; Koç, Uğur; Wei, Shiyi; Foster, Jeffrey S.; Carpuat, Marine; Porter, Adam

doi:10.1007/s10664-022-10253-z

Cited by 3 publications

(2 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the support vector machine (SVM) proposed by Yang et al [35] for the experiments, the recall rate exceeded 87% with AUC scores of over 97% when predicting actionable warnings. While Yang et al would not recommend DL models, Yerramreddy et al [36] demonstrated that neural networks that learn via source code outperform traditional models. This sparked our interest in attempting the application of DL models for processing static analysis alarms.…”

Section: Rq2: Effectivenessmentioning

confidence: 99%

“…They also mentioned that their attempts failed, even after experimenting with various architectures including feedforward networks, CNNs, and CodeBERT. Moreover, Yerramreddy et al [36] presented a comparative empirical study of three machine learning techniques for classifying correct and incorrect results generated by ASATs. Their observations suggest that neural networks, such as RNNs, trained on source code outperform traditional models.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Method for Processing Static Analysis Alarms Based on Deep Learning

Tan,

Tian

2024

Applied Sciences

View full text Add to dashboard Cite

Automatic static analysis tools (ASATs), also known as static analyzers, have demonstrated their significance and practicability in detecting software defects. ASATs assist developers to identify potential vulnerabilities, errors, and security hazards in source code without executing the software. As software systems grow in scale and complexity, ASATs are replacing manual security audits and becoming crucial for detecting issues in code. However, ASATs often generate numerous warnings with high false positive rates, while developers typically only take measures on a small portion of actionable alarms. To cope with this problem, we propose an innovative method that combines the pre-trained CodeBERT model and neural networks to reduce false positives detected by ASATs. Our approach was evaluated on the Defects4J dataset, which comprises 835 real-world software defects extracted from 17 open-source Java projects. The experimental results explicitly manifest the effectiveness in processing static analysis alarms. By employing a bidirectional recurrent neural network for context embeddings, our approach achieved an accuracy of 95.77% and an AUC score of 98.3%. This research enables developers to minimize false positive alarms and ensure a reasonable number of actionable warnings while guaranteeing software quality and security.

show abstract

Section: Rq2: Effectivenessmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

A Method for Processing Static Analysis Alarms Based on Deep Learning

Tan,

Tian

2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

Automatic Testing and Benchmarking for Configurable Static Analysis Tools

Mordahl¹

2023

Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Machine Learning for Actionable Warning Identification: A Comprehensive Survey

Ge,

Fang,

et al. 2024

ACM Comput. Surv.

View full text Add to dashboard Cite

Actionable Warning Identification (AWI) plays a crucial role in improving the usability of static code analyzers. With recent advances in Machine Learning (ML), various approaches have been proposed to incorporate ML techniques into AWI. These ML-based AWI approaches, benefiting from ML’s strong ability to learn subtle and previously unseen patterns from historical data, have demonstrated superior performance. However, a comprehensive overview of these approaches is missing, which could hinder researchers and practitioners from understanding the current process and discovering potential for future improvement in the ML-based AWI community. In this paper, we systematically review the state-of-the-art ML-based AWI approaches. First, we employ a meticulous survey methodology and gather 51 primary studies from 2000/01/01 to 2023/09/01. Then, we outline a typical ML-based AWI workflow, including warning dataset preparation, preprocessing, AWI model construction, and evaluation stages. In such a workflow, we categorize ML-based AWI approaches based on the warning output format. Besides, we analyze the key techniques used in each stage, along with their strengths, weaknesses, and distribution. Finally, we provide practical research directions for future ML-based AWI approaches, focusing on aspects like data improvement (e.g., enhancing the warning labeling strategy) and model exploration (e.g., exploring large language models for AWI).

show abstract

An empirical assessment of machine learning approaches for triaging reports of static analysis tools

Cited by 3 publications

References 58 publications

A Method for Processing Static Analysis Alarms Based on Deep Learning

A Method for Processing Static Analysis Alarms Based on Deep Learning

Automatic Testing and Benchmarking for Configurable Static Analysis Tools

Machine Learning for Actionable Warning Identification: A Comprehensive Survey

Contact Info

Product

Resources

About