Quality of Testing in Test Driven Development

Abstract

Test-driven development is an essential part of eXtreme Programming approach with the preference of being followed in other Agile methods as well. For several years, researchers are performing empirical investigations to evaluate quality improvements in the resulting code when test-driven development is being used. However, very little had been reported into investigating the quality of the testing performed in conjunction with test-driven development.In this paper we present results from an experiment specifically designed to evaluate the quality of test cases created by developers who used the test-first and the traditional test-last approaches. On an average, the quality of testing in test-driven development was almost the same as the quality of testing using test-last approach. However, detailed analysis of test cases, created by test-driven development group, revealed that 29% of test cases were "negative" test cases (based on non-specified requirements) but contributing as much as 65% to the overall tests quality score of test-first developers.We are currently investigating the possibility of extending testdriven development to facilitate non-specified requirements to a higher extent and thus minimise the impact of a potentially inherent effect of positive test bias.

On the effectiveness of unit tests in test-driven development

“…We then present the experiment details and conclusions from three prior academic experiments [11,12,26] that investigated the effects of TDD on unit test cases.…”

“…Kaczanowski [25] discusses three characteristics to measure test quality: Test smells, code coverage and mutation score indicator. Causevic et al [11,12] in their experiments also refer these as test quality attributes. A recent mapping study summarizes different ways to assess the quality of test-code among 28 primary studies [43].…”

“…Therefore, the immediate impact of TDD compared to a traditional test-last approach is expected to be on unit test cases. Considering the mentioned relation, we found few empirical studies that investigate the effects of TDD on unit test case effectiveness in terms of catching code defects [11,12,26]. These three studies conducted in academic settings report that there is minor or no difference between the effectiveness of unit tests implemented in TDD and test-last development.…”

“…We analyse the effects of TDD on unit test case effectiveness with respect to mutation score indicator. We also measured the code coverage with respect to unit test cases, as all the prior academic experiments [11,12,26] also assessed the test effectiveness in terms of coverage. Our study further contributes to the existing literature by assessing the same hypotheses with professionals in an industrial setting.…”

Quality Metrics of Test Suites in Testdriven Designed Applications

“…Finally, Cauevic and Punnekkat [34] also measured code coverage and mutation score indicators. However, those two metrics were utilized for an analysis to external attribute, a defect detecting ability attribute.…”