Why software testing is sometimes ineffective: Two applied studies of positive test strategy.

Teasley, Barbee; Leventhal, Laura Marie; Mynatt, Clifford R.; Rohlman, Diane S.

doi:10.1037/0021-9010.79.1.142

Cited by 29 publications

(46 citation statements)

References 28 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An accumulating body of research on clinical judgment also report existence of confirmatory behaviour [16], [17]. In their laboratory and field studies, Teasley et al also observed that experienced programmers/testers exhibit confirmatory behaviour as much as novices [18], [19]. Our previous findings within the context of software engineering were inline with all these above mentioned studies.…”

Section: Characteristics Of the Software Engineerssupporting

confidence: 80%

Empirical analysis of factors affecting confirmation bias levels of software engineers

Çalıklı

Bener

2014

Software Qual J

View full text Add to dashboard Cite

Confirmation bias is defined as the tendency of people to seek evidence that verifies a hypothesis rather than seeking evidence to falsify it. Due to the confirmation bias, defects may be introduced in a software product during requirements analysis, design, implementation and/or testing phases. For instance, testers may exhibit confirmatory behaviour in the form of a tendency to make the code run rather than employing a strategic approach to make it fail. As a result, most of the defects that have been introduced in the earlier phases of software development may be overlooked leading to an increase in software defect density. In this paper, we quantify confirmation bias levels in terms of a single derived metric. However, the main focus of this paper is the analysis of factors affecting confirmation bias levels of software engineers. Identification of these factors can guide project managers to circumvent negative effects of confirmation bias; as well as providing guidance for the recruitment and effective allocation of software engineers. In this empirical study, we observed low confirmation bias levels among participants with logical reasoning and hypothesis testing skills.

show abstract

Section: Characteristics Of the Software Engineerssupporting

confidence: 80%

Empirical analysis of factors affecting confirmation bias levels of software engineers

Çalıklı

Bener

2014

Software Qual J

View full text Add to dashboard Cite

show abstract

“…We have also used verbal protocol techniques: The goal of the verbal protocol studies is to identify how positive test bias occurs in naturalistic software testing. In the following sections, we will review our previous work involving naturalistic testing tasks [see Teasley, Leventhal, Mynatt and Rohlman, 1993;Mynatt, Mynatt, Rohlman and Leventhal, 1992]. In our review below, we will refer to Studies 1, 2, 3 and 4.…”

Section: Review Of Our Workmentioning

confidence: 98%

Positive test bias in software testing among professionals: A review

Leventhal

Teasley

Rohlman

et al. 1993

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Fundamental but virtually unexplored issues in human-computer interaction involve the roles of biases in software engineering tasks. In studies of naturalistic testing tasks, as well as ones which follow common laboratory models in this area, we have found ample evidence that testers have positive test bias. This bias is manifest as a tendency to execute about four times as many positive tests, designed to show that "the program works," as tests which challenge the program. In our prior work, we have found that the expertise of the subjects, the completeness of the software specifications, and the presence/absence of program errors may reduce positive test bias. Skilled computer scientists invent specifications to test in the absence of actual specifications, but still exhibit positive test bias.

show abstract

“…test cases) that are compatible with their hypotheses. Our participants are developers, and as we mentioned previously, there is a similarity between software testing and Wason's Rule Discovery Task [64]. Hence, developers with Ind elim/enum values lower than 1 are more likely to be inclined to select positive test cases to verify their code.…”

Section: Interactive Question Metricsmentioning

confidence: 99%

“…al. in [64], there is a similarity between Wason's Rule Discovery Task and functional testing. Since the interactive question is Wason's Rule Discovery Task itself, the hypotheses testing behavior of the developer gives us clues about the strategies employed by the developer to test his/her own code.…”

Section: Confirmation Bias Metricsmentioning

confidence: 99%

See 1 more Smart Citation

Influence of confirmation biases of developers on software quality: an empirical study

Çalıklı

Bener

2012

Software Qual J

View full text Add to dashboard Cite

The thought processes of people have a significant impact on software quality, as software is designed, developed and tested by people. Cognitive biases, which are defined as deviations of human mind from the laws of logic and mathematics, are likely to cause software defects. However, there is little empirical evidence to date to substantiate this assertion. In this research, we focus on a specific cognitive bias type called confirmation bias, which is defined as the tendency of people to seek for evidence to verify hypotheses rather than seeking for evidence to falsify them. Due to confirmation bias, developers might perform unit tests to make their program work rather than to break. Hence, confirmation bias is believed to be one of the factors that lead to increased software defect density. In this research, we present a metric scheme to explore the impact of developers' confirmation bias on software defect density. In order to estimate effectiveness of our metric scheme in quantification of confirmation bias within the context of software development, we performed an empirical study that addressed the prediction of the defective parts of software. In our empirical study, we used confirmation bias metrics on five datasets obtained from two companies. Our results provide empirical evidence that human thought processes and cognitive aspects deserve further investigation to improve decision making in software development for effective process management and resource allocation.

show abstract

Why software testing is sometimes ineffective: Two applied studies of positive test strategy.

Cited by 29 publications

References 28 publications

Empirical analysis of factors affecting confirmation bias levels of software engineers

Empirical analysis of factors affecting confirmation bias levels of software engineers

Positive test bias in software testing among professionals: A review

Influence of confirmation biases of developers on software quality: an empirical study

Contact Info

Product

Resources

About