Learning to rank relevant files for bug reports using domain knowledge

Abstract

When a new bug report is received, developers usually need to reproduce the bug and perform code reviews to find the cause, a process that can be tedious and time consuming. A tool for ranking all the source files of a project with respect to how likely they are to contain the cause of the bug would enable developers to narrow down their search and potentially could lead to a substantial increase in productivity. This paper introduces an adaptive ranking approach that leverages domain knowledge through functional decompositions of source code files into methods, API descriptions of library components used in the code, the bug-fixing history, and the code change history. Given a bug report, the ranking score of each source file is computed as a weighted combination of an array of features encoding domain knowledge, where the weights are trained automatically on previously solved bug reports using a learning-to-rank technique. We evaluated our system on six large scale open source Java projects, using the before-fix version of the project for every bug report. The experimental results show that the newly introduced learning-to-rank approach significantly outperforms two recent state-of-the-art methods in recommending relevant files for bug reports. In particular, our method makes correct recommendations within the top 10 ranked source files for over 70% of the bug reports in the Eclipse Platform and Tomcat projects.

Locating bugs without looking back

“…Our simpler approach only uses the first of Ye et al (2014) six features, lexical similarity, and yet provides better results on Tomcat, as we'll show. Youm et al (2015) introduced an approach where the scoring methods utilised in previous studies (Zhou et al 2012;Saha et al 2013;Wong et al 2014;) are first calculated individually and then combined together by varying alpha and beta parameter values.…”

“…Current state-of-the-art approaches for Java programs [BugLocator (Zhou et al 2012), BRTracer (Wong et al 2014), BLUiR (Saha et al 2013), AmaLgam , LearnToRank (Ye et al 2014), BLIA (Youm et al 2015) and Rahman et al (2015)] rely on project history to improve the suggestion of relevant source files. In particular they use similar BRs and recently modified files.…”

“…To address RQ1 we propose a novel approach and then evaluate it against existing approaches (Zhou et al 2012;Saha et al 2013;Moreno et al 2014;Wong et al 2014;Ye et al 2014) on the same datasets and with the same performance metrics. Like other approaches, ours scores each file against a given BR and then ranks the files in descending order of score, aiming for at least one of the files affected by the BR to be among the top-ranked ones, so that it can serve as an entry point to navigate the code and find the other affected files.…”

“…It has been argued that existing approaches treat source code files as one whole unit and bug report (BR) as one plain text (Poshyvanyk et al 2007;Saha et al 2013;Ye et al 2014;Kevic and Fritz 2014;Abebe et al 2011). However, BRs may contain useful information like stack traces, code fragments, patches and recreation steps (Bettenburg et al 2008).…”

“…Subsequently, BLUiR incorporated BugLocator's SimiScore, which did further improve its performance. Ye et al (2014) claim that if a source file is recently changed than it may still contain defects and if a file is changed frequently than it may be more likely to have additional errors. They argued that there is a significant inherent mismatch between the descriptive natural language vocabulary of a BR and the vocabulary of a programming language found in the source code files.…”