A Longitudinal Analysis of Bug Handling Across Eclipse Releases

Abstract

Large open source software projects, like Eclipse, follow a continuous software development process, with a regular release cycle. During each release, new bugs are reported, triaged and resolved. Previous studies have focused on various aspects of bug fixing, such as bug triaging, bug prediction, and bug process analysis. Most studies, however, do not distinguish between what happens before and after each scheduled release. We are also unaware of studies that compare bug fixing activities across different project releases. This paper presents an empirical analysis of the bug handling process of Eclipse over a 15-year period, considering 138K bug reports from Bugzilla, including 16 annual Eclipse releases and two quarterly releases in 2018. We compare the bug resolution rate, the fixing rate, the bug triaging time and the fixing time before and after each release date, and we study the possible impact of "release pressure". Among others, our results reveal that Eclipse bug handling activity is improving over time, with an important decrease in the number of reported bugs before releases, an increase in the bug fixing rate and an increasingly balanced bug handling workload before and after releases. The recent transition from an annual to a quarterly release cycle continued to improve the bug handling process.

Collaboration and Innovation Dynamics in Software Ecosystems: A Technology Management Research Perspective

“…But, researchers may be however overwhelmed by the magnitude of data (and may suppose they have the whole picture, which is in general wrong as many interactions may not be recorded, and risk to perform superficial or erroneous analyses [17]. In this context, case studies, normally aimed at tracking a specific attribute or establishing relationships between different attributes [18], offer a good approach, as already shown in [16], [19]- [21].…”

“…This, in turn, affects to boost innovation as long as release dates are planned. 3) Feature Freeze: In several SECOs, it has been reported that all creative development must be completed by the feature freeze milestone date specified in the release proposal [21]. This allows development and innovation to work on deadlines and identified boundaries.…”

Does the duration of rapid release cycles affect the bug handling activity?

“…This is important for defining the fixing duration. Moreover, a bug is triaged when a developer gets assigned to the issue representing the bug [6]. Therefore, we define the following auxiliary functions:…”

“…However, the adoption of rapid releases may result in less time for the community to address bugs [5]. While switching to rapid release cycles has shown to be beneficial for mature projects with large developer communities such as Eclipse [5], [6] and Firefox [4], [7], it remains unknown whether and how such a switch impacts projects with different characteristics. The optimal release cycle duration for different projects may differ depending on the size of the developer community, project size, development processes used and level of automation.…”

Studying the Impact of Policy Changes on Bug Handling Performance

“…In our prior work [15], we studied the research questions through an empirical analysis of the bug handling process of four core Eclipse products over a 15-year period, considering 138K bug reports from Bugzilla, including 16 annual Eclipse releases and two quarterly releases. We relied on four metrics to quantify bug handling: bug triaging and fixing time, and bug resolution and fixing rate.…”