An Empirical Study of Adoption of Software Testing in Open Source Projects

Kochhar, Pavneet Singh; Bissyandé, Tegawendé F.; Lo, David; Jiang, Lingxiao

doi:10.1109/qsic.2013.57

Cited by 51 publications

(28 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When investigating the presence or absence of tests, Kochar et al mined 20,000 open-source projects and found that 62% contain unit tests [36]. LaToza et al [37] surveyed 344 software engineers, testers and architects at Microsoft, with 79% of the respondents indicating to use unit tests.…”

Section: Related Workmentioning

confidence: 99%

“…The work presented in this paper differs from the aforementioned works in that the data that we use is not obtained (1) from a software repository [3,4,27,36,41] or (2) purely by means of a survey or interview [37][38][39]42]. Instead, our data is automatically gathered inside the IDE, which makes it (1) more fine-grained than commitlevel activities and (2) more objective than surveys.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

When, how, and why developers (do not) test in their IDEs

Beller

Gousios

Panichella

et al. 2015

Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering

142

115

View full text Add to dashboard Cite

The research community in Software Engineering and Software Testing in particular builds many of its contributions on a set of mutually shared expectations. Despite the fact that they form the basis of many publications as well as open-source and commercial testing applications, these common expectations and beliefs are rarely ever questioned. For example, Frederic Brooks' statement that testing takes half of the development time seems to have manifested itself within the community since he first made it in the "Mythical Man Month" in 1975. With this paper, we report on the surprising results of a large-scale field study with 416 software engineers whose development activity we closely monitored over the course of five months, resulting in over 13 years of recorded work time in their integrated development environments (IDEs). Our findings question several commonly shared assumptions and beliefs about testing and might be contributing factors to the observed bug proneness of software in practice: the majority of developers in our study does not test; developers rarely run their tests in the IDE; Test-Driven Development (TDD) is not widely practiced; and, last but not least, software developers only spend a quarter of their work time engineering tests, whereas they think they test half of their time.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

When, how, and why developers (do not) test in their IDEs

Beller

Gousios

Panichella

et al. 2015

Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering

142

115

View full text Add to dashboard Cite

show abstract

“…Thus the effectiveness of a dynamic approach is often dependent on the quality of a test suite. Unfortunately, Kocchar et al have shown that the adoption of software testing in many projects is often poor [16]. Spectrum-based fault localization, e.g., [2,14,18,29] and model-based fault localization, e.g., [10,22], are some of the well known dynamic approaches.…”

Section: Bug Localizationmentioning

confidence: 99%

Version history, similar report, and structure: putting them together for improved bug localization

Wang

2014

Proceedings of the 22nd International Conference on Program Comprehension

Self Cite

179

158

View full text Add to dashboard Cite

During the evolution of a software system, a large number of bug reports are submitted. Locating the source code files that need to be fixed to resolve the bugs is a challenging problem. Thus, there is a need for a technique that can automatically figure out these buggy files. A number of bug localization solutions that take in a bug report and output a ranked list of files sorted based on their likelihood to be buggy have been proposed in the literature. However, the accuracy of these tools still need to be improved.In this paper, to address this need, we propose AmaLgam, a new method for locating relevant buggy files that puts together version history, similar reports, and structure. To do this, AmaLgam integrates a bug prediction technique used in Google which analyzes version history, with a bug localization technique named BugLocator which analyzes similar reports from bug report system, and the state-ofthe-art bug localization technique BLUiR which considers structure. We perform a large-scale experiment on four open source projects, namely AspectJ, Eclipse, SWT and ZXing to localize more than 3,000 bugs. Compared with a historyaware bug localization solution of Sisman and Kak, our approach achieves a 46.1% improvement in terms of mean average precision (MAP). Compared with BugLocator, our approach achieves a 24.4% improvement in terms of MAP. Compared with BLUiR, our approach achieves a 16.4% improvement in terms of MAP.

show abstract

“…They find that only 14% of the active projects use pull-requests and 60% of the pull-requests are processed in a day. Kochhar et al investigate 50,000 projects to study the correlation between the presence of test cases and various project development characteristics, including the lines of code and the size of development teams [26], [27].…”

Section: Other Large Scale Studies On Githubmentioning

confidence: 99%

A Large Scale Study of Multiple Programming Languages and Code Quality

Kochhar

Wijedasa

2016

2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER)

Self Cite

View full text Add to dashboard Cite

Nowadays, most software use multiple programming languages to implement certain functionalities based on the strengths and weaknesses of different languages. Researchers in the past have studied the impact of independent programming languages on software quality, however, there has been little or no research on the impact of multiple languages on the quality of software. Does the use of multiple languages cause more bugs? Are certain languages when used with other languages make software more bug prone? What are the relationships between multi-language usage and various bug categories?In this study, we perform a large scale empirical investigation to provide some answers to these questions. We gather a large dataset consisting of popular projects from GitHub (628 projects, 85 million SLOC, 134 thousand authors, 3 million commits, in 17 languages) to understand the impact of using multiple languages on software quality. We build multiple regression models to study the effects of using different languages on the number of bug fixing commits while controlling for factors such as project age, project size, team size, and the number of commits. Our results show that in general implementing a project with more languages has a significant effect on project quality, as it increases defect proneness. Moreover, we find specific languages that are statistically significantly more defect prone when they are used in a multi-language setting. These include popular languages like C++, Objective-C, and Java. Furthermore, we note that the use of more languages significantly increases bug proneness across all bug categories. The effect is strongest for memory, concurrency, and algorithm bugs.

show abstract

An Empirical Study of Adoption of Software Testing in Open Source Projects

Cited by 51 publications

References 36 publications

When, how, and why developers (do not) test in their IDEs

When, how, and why developers (do not) test in their IDEs

Version history, similar report, and structure: putting them together for improved bug localization

A Large Scale Study of Multiple Programming Languages and Code Quality

Contact Info

Product

Resources

About