Using the Methods of Statistical Data Analysis to Improve the Trustworthiness of Software Reliability Modeling

Abstract

In the past, many Software Reliability Growth Models (SRGMs) have been proposed to predict various aspects of software reliability and NHPP (Non-Homogeneous Poisson Process) based SRGMs have drawn a lot of attention. But it should be noted that, in practice, blindly applying SRGMs might not lead to meaningful results when the characteristics of data (i.e., data type) don't fit the characteristics of selected SRGM(s). Also, there does not exist any guidelines that project managers or developers can follow to select the most appropriate SRGMs for a particular failure data set. On the other hand, waiting to collect a substantial amount of failure data before being able to fit SRGM(s) might not be feasible in many cases. In this paper, we propose to use the methods of statistical data analysis to investigate the characteristics of software failure data and to improve the accuracy of software reliability growth modeling. Here SRGMs will be broadly partitioned into two categories: the "NHPP models" and the "not belonging to the class of NHPP models". Similarly, software failure data can be classified into two categories, the "NHPP-type data" and the "not NHPP-type data". We will propose a two-phase method to verify the hypothesis that failure data follows an NHPP. Through the proposed two-phase test and the Laplace trend test, we will be able to infer whether a dataset is the NHPP-type and whether it is properly used by NHPP-based SRGMs. In this case, the final decision of choosing the most suitable NHPP model(s) to model the software failure process would be made, and the software reliability prediction result is trustworthy and helpful after we can correctly judge whether the dataset is said to be absolutely NHPP-type or not. Experiments based on 25 real software failure data are presented and analyzed. Using our proposed method, project managers and/or developers will be able to analyze and use collected failure data to help improve both planning accuracy and software quality.

Queueing-Based Simulation for Software Reliability Analysis

“…However, the foregoing discussion showed that most studies assume that software reliability growth behavior follows a non-homogeneous poisson process (NHPP), based on collected software failure data. It is worth noting that Hou [33] reported that before applying the failure data to the SRGMs, whether the dataset is of the NHPP type and whether it is used correctly by the NHPP-based SRGM should be explained. On the other hand, Kanoun and Lapire [4] also observed that the use of NHPP-based SRGMs during the early stages of development and validation is much less convincing.…”