A Critical Comparison on Six Static Analysis Tools: Detection Agreement and Precision

Lenarduzzi, Valentina; Lujan, Savanna; Saarimäki, Nyyti; Palomba, Fabio

doi:10.2139/ssrn.4044439

Cited by 7 publications

(3 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the past, several empirical static analysis comparison studies have been conducted, but they tended to overlook Java [30]- [33] or included a wide range of programming language tools, leaving limited attention to Java-specific tools [18], [21], [34]- [37]. Despite this, some studies that specifically targeted Java had either a restricted scope by evaluating only a few security vulnerability categories [19], [20], [26], [38]- [46] or concentrated solely on Android static analysis tools [47]. Now, let's take a closer look at these studies, categorizing them into two subsets: those utilizing only the Juliet Test Suite for evaluation and those that employed real-world programs, with or without using Juliet.…”

Section: Related Workmentioning

confidence: 99%

“…Table 5 presents the summary of the related works in this category. Now, we explain the studies in the second subset [18], [19], [21], [31]- [33], [35], [38], [42]- [44], [46], [47]. Katerina Goseva et al [18] investigated the ability of three -unknowncommercial static tools to find Java and C/C++ security flaws.…”

Section: Related Workmentioning

confidence: 99%

“…Six free static analysis tools were evaluated by Valentina et al [38] to examine 47 Java projects from the Qualitas Cor-pus dataset: PMD, Better Code Hub, CheckStyle, Findbugs, Coverity Scan, and Sonar. First, the researchers measured the precision of the tools; second, they compared the warnings detected by the tools and developed a taxonomy that can assist researchers and tool vendors in identifying critical warnings for refactoring; and third, they analyzed the agreement among the tools, which can provide information to tool vendors about the limitations of the current state-of-the-art tools.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Comprehensive Evaluation of Static Analysis Tools for Their Performance in Finding Vulnerabilities in Java Code

Alqaradaghi,

Kozsik

2024

IEEE Access

View full text Add to dashboard Cite

Various static code analysis tools have been designed with the aim of detecting software faults and security vulnerabilities automatically. This paper aims to 1) Conduct an empirical evaluation to assess the performance of five free and state-of-the-art static analysis tools in detecting Java security vulnerabilities using a well-defined and repeatable approach. 2) Report on the vulnerabilities that are best and worst detected by static Java analyzers. We used the Juliet benchmark test suite in a controlled experiment to assess the effectiveness of five widely used Java static analysis tools. The vulnerabilities were successfully detected by one, two, or three tools. Only one vulnerability has been detected by four tools. The tools missed 13% of the Java vulnerability categories appearing in our experiment. More critically, none of the five tools could identify all the vulnerabilities in our experiment. We conclude that, despite recent improvements in their methodologies, current state-of-the-art static analysis tools are still ineffective for identifying the security vulnerabilities occurring in a small-scale, artificial test suite.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Comprehensive Evaluation of Static Analysis Tools for Their Performance in Finding Vulnerabilities in Java Code

Alqaradaghi,

Kozsik

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

A machine and deep learning analysis among SonarQube rules, product, and process metrics for fault prediction

2022

View full text Add to dashboard Cite

Background Developers spend more time fixing bugs refactoring the code to increase the maintainability than developing new features. Researchers investigated the code quality impact on fault-proneness, focusing on code smells and code metrics. Objective We aim at advancing fault-inducing commit prediction using different variables, such as SonarQube rules, product, process metrics, and adopting different techniques. Method We designed and conducted an empirical study among 29 Java projects analyzed with SonarQube and SZZ algorithm to identify fault-inducing and fault-fixing commits, computing different product and process metrics. Moreover, we investigated fault-proneness using different Machine and Deep Learning models. Results We analyzed 58,125 commits containing 33,865 faults and infected by more than 174 SonarQube rules violated 1.8M times, on which 48 software product and process metrics were calculated. Results clearly identified a set of features that provided a highly accurate fault prediction (more than 95% AUC). Regarding the performance of the classifiers, Deep Learning provided a higher accuracy compared with Machine Learning models. Conclusion Future works might investigate whether other static analysis tools, such as FindBugs or Checkstyle, can provide similar or different results. Moreover, researchers might consider the adoption of time series analysis and anomaly detection techniques.

show abstract

On the adoption and effects of source code reuse on defect proneness and maintenance effort

Giordano,

Festa,

Catolino

et al. 2023

Empir Software Eng

View full text Add to dashboard Cite

Software reusability mechanisms, like inheritance and delegation in Object-Oriented programming, are widely recognized as key instruments of software design that reduce the risks of source code being affected by defects, other than to reduce the effort required to maintain and evolve source code. Previous work has traditionally employed source code reuse metrics for prediction purposes, e.g., in the context of defect prediction. However, our research identifies two noticeable limitations of the current literature. First, still little is known about the extent to which developers actually employ code reuse mechanisms over time. Second, it is still unclear how these mechanisms may contribute to explaining defect-proneness and mainten0ance effort during software evolution. We aim at bridging this gap of knowledge, as an improved understanding of these aspects might provide insights into the actual support provided by these mechanisms, e.g., by suggesting whether and how to use them for prediction purposes. We propose an exploratory study, conducted on 12 Java projects–over 44,900 commits–of the Defects4J dataset, aiming at (1) assessing how developers use inheritance and delegation during software evolution; and (2) statistically analyzing the impact of inheritance and delegation on fault proneness and maintenance effort. Our results let emerge various usage patterns that describe the way inheritance and delegation vary over time. In addition, we find out that inheritance and delegation are statistically significant factors that influence both source code defect-proneness and maintenance effort.

show abstract

A Critical Comparison on Six Static Analysis Tools: Detection Agreement and Precision

Cited by 7 publications

References 41 publications

Comprehensive Evaluation of Static Analysis Tools for Their Performance in Finding Vulnerabilities in Java Code

Comprehensive Evaluation of Static Analysis Tools for Their Performance in Finding Vulnerabilities in Java Code

A machine and deep learning analysis among SonarQube rules, product, and process metrics for fault prediction

On the adoption and effects of source code reuse on defect proneness and maintenance effort

Contact Info

Product

Resources

About