Positive test bias in software testing among professionals: A review

Leventhal, Laura Marie; Teasley, Barbee; Rohlman, Diane S.; Instone, Keith

doi:10.1007/3-540-57433-6_50

Cited by 23 publications

(10 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results are also similar to the findings of Mäntylä et al (2014), Nan and Harter (2009) and Topi et al (2005), as time pressure did not affect the dependent variable. The results of our study also challenge the results of Leventhal et al (1993) and Leventhal et al (1994), as they interpreted the results with respect to valid (positive) and invalid (negative) equivalence classes. The results of these studies might differ if the test cases are reassessed according to (in)consistent terminology.…”

Section: Discussionsupporting

confidence: 60%

A controlled experiment on time pressure and confirmation bias in functional software testing

2018

View full text Add to dashboard Cite

Context Confirmation bias is a person's tendency to look for evidence that strengthens his/her prior beliefs rather than refutes them. Manifestation of confirmation bias in software testing may have adverse effects on software quality. Psychology research suggests that time pressure could trigger confirmation bias. Objective In the software industry, this phenomenon may deteriorate software quality. In this study, we investigate whether testers manifest confirmation bias and how it is affected by time pressure in functional software testing. Method We performed a controlled experiment with 42 graduate students to assess manifestation of confirmation bias in terms of the conformity of their designed test cases to the provided requirements specification. We employed a one factor with two treatments between-subjects experimental design. Results We observed, overall, participants designed significantly more confirmatory test cases as compared to disconfirmatory ones, which is in line with previous research. However, we did not observe time pressure as an antecedent to an increased rate of confirmatory testing behaviour. Conclusion People tend to design confirmatory test cases regardless of time pressure. For practice, we find it necessary that testers develop self-awareness of confirmation bias and counter its potential adverse effects with a disconfirmatory attitude. We recommend further replications to investigate the effect of time pressure as a potential contributor to the manifestation of confirmation bias.

show abstract

Section: Discussionsupporting

confidence: 60%

A controlled experiment on time pressure and confirmation bias in functional software testing

2018

View full text Add to dashboard Cite

show abstract

“…Positive test case, on the other hand, is exercising a program in a way that was specified in the requirements. In literature, phenomenon of more positive approach to testing is known as a "positive test bias" [19], [20].…”

Section: Quality Of Test Cases ( )mentioning

confidence: 99%

Quality of Testing in Test Driven Development

Cauevic

Punnekkat

Sundmark

2012

2012 Eighth International Conference on the Quality of Information and Communications Technology

View full text Add to dashboard Cite

Test-driven development is an essential part of eXtreme Programming approach with the preference of being followed in other Agile methods as well. For several years, researchers are performing empirical investigations to evaluate quality improvements in the resulting code when test-driven development is being used. However, very little had been reported into investigating the quality of the testing performed in conjunction with test-driven development.In this paper we present results from an experiment specifically designed to evaluate the quality of test cases created by developers who used the test-first and the traditional test-last approaches. On an average, the quality of testing in test-driven development was almost the same as the quality of testing using test-last approach. However, detailed analysis of test cases, created by test-driven development group, revealed that 29% of test cases were "negative" test cases (based on non-specified requirements) but contributing as much as 65% to the overall tests quality score of test-first developers.We are currently investigating the possibility of extending testdriven development to facilitate non-specified requirements to a higher extent and thus minimise the impact of a potentially inherent effect of positive test bias.

show abstract

“…This leads to a situation where only a small portion of the vast set of possible scenarios that the system could be subjected to are thoroughly tested, while the others are left entirely unexplored. In addition, test cases are typically based on requirements defining how the system should behave during normal operation [14]. While this provides valuable confirmation with respect to the system's fitness for use in the normal case, there are results indicating that focusing on normal requirement-based cases might not be the best strategy when trying to maximize fault-detection (see e.g., [15] and [16]).…”

Section: A Automotive System Integration Testingmentioning

confidence: 99%

Offline Analysis of Independent Guarded Assertions in Automotive Integration Testing

Rodríguez-Navas

Kobetski

Sundmark

et al. 2015

2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium

View full text Add to dashboard Cite

The size and complexity of software in automotive systems have increased steadily over the last decades. Modern vehicles typically contain numerous electrical control units (ECUs), and more and more features require real-time interaction between several dedicated ECUs (e.g., gearbox, brake and engine control units) in order to perform their tasks. Since system safety and reliability must not be adversely affected by this increase in complexity, proper quality assurance is a must. Such quality assurance is often performed by testing the system in different levels of integration throughout the development process. However, the growth of complexity of the system under test aslo affects the testing, making it laborious, difficult and costly. This paper presents a novel method for efficient offline analysis of traces, which has been especially tailored for integration testing of automotive systems. The method exploits the recently defined concept of independent guarded assertion in order to formally describe the events that are relevant for the analysis as well as the expected behavior in those events. The offline analysis is implemented using a standard commercial model checker and has shown good performance in the conducted experiments.

show abstract

Positive test bias in software testing among professionals: A review

Cited by 23 publications

References 16 publications

A controlled experiment on time pressure and confirmation bias in functional software testing

A controlled experiment on time pressure and confirmation bias in functional software testing

Quality of Testing in Test Driven Development

Offline Analysis of Independent Guarded Assertions in Automotive Integration Testing

Contact Info

Product

Resources

About