A Comparison of Bug Finding Tools for Java

Abstract

Bugs in software are costly and difficult to find and fix. In recent years, many tools and techniques have been developed for automatically finding bugs by analyzing source code or intermediate code statically (at compile time). Different tools and techniques have different tradeoffs, but the practical impact of these tradeoffs is not well understood. In this paper, we apply five bug finding tools, specifically Bandera, ESC/Java 2, FindBugs, JLint, and PMD, to a variety of Java programs. By using a variety of tools, we are able to cross-check their bug reports and warnings. Our experimental results show that none of the tools strictly subsumes another, and indeed the tools often find non-overlapping bugs. We discuss the techniques each of the tools is based on, and we suggest how particular techniques affect the output of the tools. Finally, we propose a meta-tool that combines the output of the tools together, looking for particular lines of code, methods, and classes that many tools warn about.

Evaluating the relation between coding standard violations and faultswithin and across software versions

“…Rutar et al studied the correlation and overlap between warnings generated by the ESC/Java, FindBugs, JLint, and PMD static analysis tools [19]. They did not evaluate individual warnings nor did they try to relate them to actual faults.…”

Prioritizing Warning Categories by Analyzing Software History

“…Ruter et al [10] compared bug finding tools for Java including PMD, JLint, and FindBugs. They analyze overlapping warning categories and category correlation.…”

Comparative Study of the Quality Assessment Tools Based on a Model: Sonar, Squale, EvalMetrics

“…The study of Rutar et al (2004) compare five bug detection tools (Bandera, ESC/Java 2, FindBugs, JLint and PMD) using static analysis of Java code. The result of this study is that although there are some overlaps among the types of errors detected, most of them are different.…”