Test case quality in test driven development: a study design and a pilot experiment

Čaušević, Adnan; Sundmark, Daniel; Punnekkat, Sasikumar

doi:10.1049/ic.2012.0029

Cited by 21 publications

(34 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We then present the experiment details and conclusions from three prior academic experiments [11,12,26] that investigated the effects of TDD on unit test cases.…”

Section: Related Workmentioning

confidence: 99%

“…Kaczanowski [25] discusses three characteristics to measure test quality: Test smells, code coverage and mutation score indicator. Causevic et al [11,12] in their experiments also refer these as test quality attributes. A recent mapping study summarizes different ways to assess the quality of test-code among 28 primary studies [43].…”

Section: Unit Testingmentioning

confidence: 99%

“…Their experiments did not reveal any significant differences in terms of branch coverage and mutation score of unit test cases between the two treatments. Causevic et al [11,12] later extended the experiment of Madeyski [26] in academia with master students. They conducted one experiment with 14 students, and published results of this experiment in two studies [11,12].…”

Section: Unit Test Effectiveness In Tddmentioning

confidence: 99%

“…Therefore, the immediate impact of TDD compared to a traditional test-last approach is expected to be on unit test cases. Considering the mentioned relation, we found few empirical studies that investigate the effects of TDD on unit test case effectiveness in terms of catching code defects [11,12,26]. These three studies conducted in academic settings report that there is minor or no difference between the effectiveness of unit tests implemented in TDD and test-last development.…”

Section: Introductionmentioning

confidence: 99%

“…We analyse the effects of TDD on unit test case effectiveness with respect to mutation score indicator. We also measured the code coverage with respect to unit test cases, as all the prior academic experiments [11,12,26] also assessed the test effectiveness in terms of coverage. Our study further contributes to the existing literature by assessing the same hypotheses with professionals in an industrial setting.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

On the effectiveness of unit tests in test-driven development

Tosun

Ahmed

Turhan

et al. 2018

Proceedings of the 2018 International Conference on Software and System Process

View full text Add to dashboard Cite

Background: Writing unit tests is one of the primary activities in test-driven development. Yet, the existing reviews report few evidence supporting or refuting the effect of this development approach on test case quality. Lack of ability and skills of developers to produce sufficiently good test cases are also reported as limitations of applying test-driven development in industrial practice.Objective: We investigate the impact of test-driven development on the effectiveness of unit test cases compared to an incremental test last development in an industrial context. Method: We conducted an experiment in an industrial setting with 24 professionals. Professionals followed the two development approaches to implement the tasks. We measure unit test effectiveness in terms of mutation score. We also measure branch and method coverage of test suites to compare our results with the literature.Results: In terms of mutation score, we have found that the test cases written for a test-driven development task have a higher defect detection ability than test cases written for an incremental test-last development task. Subjects wrote test cases that cover more branches on a test-driven development task compared to the other task. However, test cases written for an incremental test-last development task cover more methods than those written for the second task. Conclusion: Our findings are different from previous studies conducted at academic settings. Professionals were able to perform more effective unit testing with test-driven development. Furthermore, we observe that the coverage measure preferred in academic studies reveal different aspects of a development approach. Our results need to be validated in larger industrial contexts.

show abstract

“…We then present the experiment details and conclusions from three prior academic experiments [11,12,26] that investigated the effects of TDD on unit test cases.…”

Section: Related Workmentioning

confidence: 99%

Section: Unit Testingmentioning

confidence: 99%

Section: Unit Test Effectiveness In Tddmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

On the effectiveness of unit tests in test-driven development

Tosun

Ahmed

Turhan

et al. 2018

Proceedings of the 2018 International Conference on Software and System Process

View full text Add to dashboard Cite

show abstract

An empirical evaluation of a three‐tier conduit framework for multifaceted test case classification and selection using fuzzy‐ant colony optimisation approach

Kumar

Sharma²,

Kumar

2014

Softw Pract Exp

View full text Add to dashboard Cite

Summary The test case optimisation is an NP‐complete, knowledge‐driven, data‐driven, and multidimensional search space partitioning and dimension reduction problem. In the multifaceted test case classification, partitioning and reducing the multidimensional test case fitness search space is the critical problem. The vague nature of fitness parameters, conflicting nature objectives, and ambiguity in the test case fitness evaluation have created and increased the uncertainty, the imprecision, and the incompleteness in the test case classification and selection. Because of the increasing ambiguity, the complexity, and the cost of software testing, automated test case classification and selection has emerged as an appropriate tool to classify test cases into predefined categories using the multifaceted concept. Most of the test cases affecting the performance of the classifier are irrelevant and redundant. A strong need therefore exists to devise an intelligent technique to identify and remove test cases affecting the performance of the classifier. For increasing the performance of classifier, multifaceted test case selection is used to reduce fitness search space to be searched. In this paper, a three‐tier sequential framework is proposed for a multifaceted test case classification and selection. The first stage of the proposed framework is the fuzzy synthesis‐based filtration approach for multifaceted test case fitness evaluation and classification. The second stage of the proposed framework is the fuzzy entropy‐based filtration technique with a backward search strategy, used for estimating and reducing the ambiguity in test case fitness evaluation, classification, and selection. The third stage of the proposed framework is the ant colony optimisation‐based wrapper technique with a forward search strategy, employed to select test cases from the output (reduced) test suite by the second stage. The proposed framework is tested on artefacts of benchmark applications. The results of the empirical study clearly show that the third stage of our proposed method outperforms the second and first stages, and the performance of the algorithms used in all three stages increases on average as the stages are escalating. The classification accuracy is enhanced by reducing the ambiguity in fitness and the classification of test cases, increasing the number of test cases accurately classified, and reducing the number in the test case pool to be exercised. Copyright © 2014 John Wiley & Sons, Ltd.

show abstract