Examining Software Engineering Beliefs about System Testing Defects

Abstract

Research findings are often expressed as short startling sentences or software engineering (SE) beliefs such as "about 80 percent of the defects come from 20 percent of the modules" and "peer reviews catch 60 percent of the defects" [2]. Such SE beliefs are particularly important in industry, as they are attention-getting, easily understandable, and thus practically useful. In this paper we examine the power of such SE beliefs to justify process improvement through empirical validation of selected beliefs related to the increase or decrease of defects in system testing. We explore four basic SE beliefs in data from two midsize embedded software development organizations in Japan, and based on this information, identify possible process improvement actions for each organization. Based on our study, even small and medium-sized enterprises (SMEs) can use this approach to find possible directions to improve their process, which will result in better products.

A Cost-Effectiveness Metric for Association Rule Mining in Software Defect Prediction

“…This value is obtained by b 1), assuming unit testing and assigning 100 test cases per 1,000 lines of code (i.e., S ti = 10), which can detect 50% of all bugs (i.e., Ĥ(ti) a = 0.5). The reason for assuming that 100 test cases are assigned per 1,000 lines is based on the following fact; (1) a large japanese software development company uses "100 cases per 1,000 lines" as the company standard for unit testing [14], (2) a medium-sized japanese software company assigned 99.31 test cases per 1,000 lines on average [10]. Therefore, we believe "100 test cases per 1,000 lines of code" is not the unrealistic value.…”

“…Therefore, we believe "100 test cases per 1,000 lines of code" is not the unrealistic value. In addition, the assumption that 50% of all bugs can be found under this condition is based on the case that an average of 50% of bugs in the unit test cases is detected in some japanese software company [10]. Although b 0 can vary among different projects, this paper uses b 0 = 6.932 as one of the realistic values.…”