A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in Apache open source projects

Trautsch, Alexander; Herbold, Steffen; Grabowski, Jens

doi:10.1007/s10664-020-09880-1

Cited by 29 publications

(21 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SORALD is extensible and is not bound to only use SONARJAVA. Extending SORALD to use other rule-based static analyzers like SpotBugs [26] and PMD [27] is a straightforward endeavour. The only requirement from SO-RALD is that the static analyzer in question provides an accurate source code position for the violation.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Sorald: Automatic Patch Suggestions for SonarQube Static Analysis Violations

Etemadi,

Harrand,

Larsen

et al. 2021

Preprint

View full text Add to dashboard Cite

Previous work has shown that early resolution of issues detected by static code analyzers can prevent major cost later on. However, developers often ignore such issues for two main reasons. First, many issues should be interpreted to determine if they correspond to actual flaws in the program. Second, static analyzers often do not present the issues in a way that makes it apparent how to fix them. To address these problems, we present SORALD: a novel system that adopts a set of predefined metaprogramming templates to transform the abstract syntax trees of programs to suggest fixes for static issues. Thus, the burden on the developer is reduced from both interpreting and fixing static issues, to inspecting and approving solutions for them. SORALD fixes violations of 10 rules from SONARQUBE, one of the most widely used static analyzers for Java. We also implement an effective mechanism to integrate SORALD into development workflows based on pull requests. We evaluate SORALD on a dataset of 161 popular repositories on GITHUB. Our analysis shows the effectiveness of SORALD as it fixes 94% (1,153/1,223) of the violations that it attempts to fix. Overall, our experiments show it is possible to automatically fix violations of static analysis rules produced by the state-of-the-art static analyzer SONARQUBE.

show abstract

Section: Methodsmentioning

confidence: 99%

“…There are many common static code analyzers that perform static analysis on programs to identify potential bugs, vulnerabilities, or code smells [30], such as SONARQUBE, SPOTBUGS [26], FINDBUGS [31], and PMD [27].…”

Section: Related Workmentioning

confidence: 99%

Sorald: Automatic Patch Suggestions for SonarQube Static Analysis Violations

Etemadi,

Harrand,

Larsen

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…One of the results of Herbold et al (2021) is that non-production code is often tangled with bug fixing changes. Therefore we only add files that are production files to our final data analogous to Trautsch et al (2020a). This also helps us to provide a clearer picture of warning density based features as production code may have a different evolution of warning density than, e.g., test or example code.…”

Section: Methodsmentioning

confidence: 99%

“…In prior work (Trautsch et al, 2020a) we investigated trends of ASAT warnings and investigated whether usage of an ASAT has an influence on software quality measured via defect density (Fenton and Bieman, 2014). However, our analysis with regards to bugs via defect density was coarse grained.…”

Section: Introductionmentioning

confidence: 99%

“…In our previous work, we found that static analysis warnings are evolving over time and that we can not just use the density or the sum of warnings in our case study (Trautsch et al, 2020a). Therefore, we use an approach that is able to produce a current snapshot view of the files we are interested in by measuring the file that induces a bug and, at the same time, all other files of the project.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Are automated static analysis tools worth it? An investigation into relative warning density and external software quality

Trautsch¹,

Herbold²,

Grabowski³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Automated Static Analysis Tools (ASATs) are part of software development best practices. ASATs are able to warn developers about potential problems in the code. On the one hand, ASATs are based on best practices so there should be a noticeable effect on software quality. On the other hand, ASATs suffer from false positive warnings, which developers have to inspect and then ignore or mark as invalid. In this article, we ask the question if ASATs have a measurable impact on external software quality, using the example of PMD for Java. We investigate the relationship between ASAT warnings emitted by PMD on defects per change and per file. Our case study includes data for the history of each file as well as the differences between changed files and the project in which they are contained. We investigate whether files that induce a defect have more static analysis warnings than the rest of the project. Moreover, we investigate the impact of two different sets of ASAT rules. We find that, bug inducing files contain less static analysis warnings than other files of the project at that point in time. However, this can be explained by the overall decreasing warning density. When compared with all other changes, we find a statistically significant difference in one metric for all rules and two metrics for a subset of rules. However, the effect size is negligible in all cases, showing that the actual difference in warning density between bug inducing changes and other changes is small at best.

show abstract