An Empirical Assessment on Affective Reactions of Novice Developers When Applying Test-Driven Development

Abstract

We study whether and in which phase Test-Driven Development (TDD) influences affective states of novice developers in terms of pleasure, arousal, dominance, and liking. We performed a controlled experiment with 29 novice developers. Developers in the treatment group performed a development task using TDD, whereas those in the control group used a non-TDD development approach. We compared the affective reactions to the development approaches, as well as to the implementation and testing phases, exploiting a lightweight, powerful, and widely used tool, i.e., Self-Assessment Manikin. We observed that there is a difference between the two development approaches in terms of affective reactions. Therefore, it seems that affective reactions play an important role when applying TDD and their investigation could help researchers to better understand such a development approach.

Why Research on Test-Driven Development is Inconclusive?

“…Dogša and Batic [10] The effectiveness of test-driven development : an industrial case study Fucci and Turhan [16] A Replicated Experiment on the Effectiveness of Test-first Development Thomson et al [44] What Makes Testing Work: Nine Case Studies of Software Development Teams Romano et al [35] Findings from a multi-method study on test-driven development Buchan et al [7] Causal Factors, Benefits and Challenges of Test-Driven Development: Practitioner Perceptions Scanniello et al [38] Students' and Professionals' Perceptions of Test-driven Development: A Focus Group Study Beller et al [4] Developer Testing in The IDE: Patterns, Beliefs, And Behavior Bannerman and Martin [1] A multiple comparative study of test-with development product changes and their effects on team speed and product quality Table 3: The primary studies collected in the third step Authors Title Karac et al [23] A Controlled Experiment with Novice Developers on the Impact of Task Description Granularity on Software Quality in Test-Driven Development Tosun et al [46] Investigating the Impact of Development Task on External Quality in Test-Driven Development: An Industry Experiment Borle et al [6] Analyzing the effects of test driven development in GitHub Fucci et al [15] A longitudinal cohort study on the retainment of test-driven development Kazerouni et al [24] Assessing Incremental Testing Practices and Their Impact on Project Outcomes Santos et al [37] Improving Development Practices through Experimentation : an Industrial TDD Case Tosun et al [45] On the Effectiveness of Unit Tests in Test-driven Development Santos et al [36] Does the Performance of TDD Hold Across Software Companies and Premises? A Group of Industrial Experiments on TDD Romano et al [34] An Empirical Assessment on Affective Reactions of Novice Developers When Applying Test-Driven Development Sundelin et al [43] Test-Driving FinTech Product Development: An Experience Report In terms of test quality, research has explored the quality of tests by measuring mutation scores (i.e., the bug detection ability of the tests) and code coverage (i.e., the degree to which the source code of a program is executed when a test suite runs). For example, Tosun et al [45] conducted an experiment with 24 professionals and found that unit-test cases developed in TDD have a higher mutation score and branch coverage, but less method coverage than those developed in ITL.…”

“…Latorre [26] found that in unit test-driven development, junior developers are not able to discover the best design, and this translates into a performance penalty since they need to revise their design choices more frequently than skilled developers. Romano et al [34] investigated the affective reactions of novice developers to the development approach and reported that novices seem to like a non-TDD development approach more than TDD, and that the testing phase makes developers using TDD less happy. Suleman et al [42] conducted an early pilot study with students who experienced TDD in an introductory programming course.…”

“…There may be more factors at play when comparing two techniques. For instance, a recent work has shown that testing phase makes novice developers using TDD less happy [34]. In the same vein, students perceive TDD more of an obstacle than a help [42].…”

The empirical study: encouraging students’ interest in software development using test-driven development

“…These results are similar to those in previous works. For example, previous studies [16], [18], [20], [32] suggested that TDD should be used in a class to provide students with software development skills because it can help improve the quality of software development. In addition, the use of TDD shows that students make fewer programming errors [19].…”

The effectiveness of test-driven development approach on software projects: A multi-case study

“…In the test-first development process, a test is first generated prior to the creation of adequate code, and then refactoring is done to convert the new code to accepted standards [9]. TDD includes 3 phases of test writing and test failing, writing the simplest code to pass the test, and the use of refactoring to eliminate iterations and achieve code clear [10]. The critical point is that TDD is not a technique to test, but it is a development process that generates more coherent codes that are less dependent on the traditional approach [11].…”