Towards an automation of the traceability of bugs from development logs

Abstract

Context: Information and tracking of defects can be severely incomplete in almost every Open Source project, resulting in a reduced traceability of defects into the development logs (i.e., version control commit logs). In particular, defect data often appears not in sync when considering what developers logged as their actions. Synchronizing or completing the missing data of the bug repositories, with the logs detailing the actions of developers, would benefit various branches of empirical software engineering research: prediction of software faults, software reliability, traceability, software quality, effort and cost estimation, bug prediction and bug fixing.Objective: To design a framework that automates the process of synchronizing and filling the gaps of the development logs and bug issue data for open source software projects.Method: We instantiate the framework with a sample of OSS projects from GitHub, and by parsing, linking and filling the gaps found in their bug issue data, and development logs. UML diagrams show the relevant modules that will be used to merge, link and connect the bug issue data with the development data.Results: Analysing a sample of over 300 OSS projects we observed that around 1/2 of bug-related data is present in either development logs or issue tracker logs: the rest of the data is missing from one or the other source. We designed an automated approach that fills the gaps of either source by making use of the available data, and we successfully mapped all the missing data of the analysed projects, when using one heuristics of annotating bugs. Other heuristics need to be investigated and implemented.

Locating Bug IDs and Development Logs in Open Source Software (OSS) projects: An Experience Report

“…Based on the precision and recall, we then computed the F-measure as detailed below: 100 1 3 96 0 2 Leaflet 22 0 0 22 0 3 Reddit 74 1 1 72 0 4 CocoaPods 100 0 3 97 0 5 Puma 81 0 6 75 0 6 AutoMapper 68 0 1 67 0 7 MonoDevelop 100 29 8 63 0 8 CodeHub 42 0 8 34 0 9 Manos 100 0 2 98 0 10 puppet 100 0 18 82 0 The '#' symbol gained an F-measure of 0.726, the 'Fix' keyword 0.582 and the 'Bug' keyword only 0.019. Since the F-measure is often used, in the context of information retrieval, to assess the performance of searches, this further test confirms the earlier findings reported by [17], [16]. Analysing the unstructured data of the development logs of the Brackets project as a pilot study, we conclude that the most precise proxy of bug IDs is the '#' identifier, when considering the free-text descriptions of changes written by developers as an addendum to their commits to the VC systems.…”

“…After manual inspection, we found that 40 of the sampled OSS projects were empty. Hence giving an overall number of 'alive' projects of 344 in which we analysed their development logs and bug data automatically using our approach and the proposed tool-chain developed for this research [16]. The data sets (i.e., 344 OSS projects repositories) can be found on Figshare.…”

“…Instead, the information refers to a request to merge some changes in the distributed VC system. We marked these occurrences as 'false positives' and excluded them from the pilot study as well as the extended study [17], [16].…”

A Systematic Literature Review of Issue-Based Requirement Traceability

“…[37], [38], [44]. Researchers have discussed the lack of integration between revision control systems and issue tracking systems, which affects the recovery of trace links, as well as the prediction of software faults [16], [51], [52]. Researchers addressed that issues form a complex network by themselves, but there were no approaches to predict link types [40], [43].…”

“…A total of 18 studies linked issue reports to commits. [16], [25], [27], [29], [34]- [38], [42], [44], [48], [51], [54]- [56], [58], [62]. Among them, eleven studies focused on recovering the missing trace links between issue reports and commits [16], [27], [29], [36]- [38], [44], [51], [56], [58], [62].…”