Evaluating long-term predictive power of standard reliability growth models on automotive systems

Abstract

Software is today an integral part of providing improved functionality and innovative features in the automotive industry. Safety and reliability are important requirements for automotive software and software testing is still the main source of ensuring dependability of the software artifacts. Software Reliability Growth Models (SRGMs) have been long used to assess the reliability of software systems; they are also used for predicting the defect inflow in order to allocate maintenance resources. Although a number of models have been proposed and evaluated, much of the assessment of their predictive ability is studied for short term (e.g. last 10% of data). But in practice (in industry) the usefulness of SRGMs with respect to optimal resource allocation depends heavily on the long term predictive power of SRGMs i.e. much before the project is close to completion. The ability to reasonably predict the expected defect inflow provides important insight that can help project and quality managers to take necessary actions related to testing resource allocation on time to ensure high quality software at the release. In this paper we evaluate the long-term predictive power of commonly used SRGMs on four software projects from the automotive sector. The results indicate that Gompertz and Logistic model performs best among the tested models on all fit criterias as well as on predictive power, although these models are not reliable for long-term prediction with partial data.

Analysing defect inflow distribution of automotive software projects

“…Given that we are interested in exploring the defect distribution, defect data on four large software projects from the embedded software development is used for analysis. The defect data used in this study has been used previously in our earlier work [21], where the main objective was to evaluate long-term predictive power of commonly used software reliability growth models (SRGMs). While in this study, the objective is to explore the underlying distribution of defect inflow which can aid in the selection of appropriate reliability models and other statistical techniques for various defect data analysis and modelling.…”

“…A number of SRGMs has also been compared for their ability to fit data from telecom domain in study by Staron and Meding [26]. Seven SRGMs have been compared on their performance on predictive power using partial in-process data from real projects in Rana et al [21]. Contrary to earlier studies where different software reliability models have been compared and assessed on their ability to fit defect data, in this study we compare between standard distributions such as Weibull, beta, exponential etc.…”

Deep Cross-Project Software Reliability Growth Model Using Project Similarity-Based Clustering

“…Detecting and resolving failures or defects would enable software systems to be more stable and reliable. To understand the underlying condition of the system, such processes are often described using a mathematical expression, usually based on parameters such as the number of failures or failure density [12]. Studies report many ways to create models based on the model's assumption of failure occurrence patterns.…”

Deep Cross-Project Software Reliability Growth Model using Project Similarity Based Clustering

“…After development is done, if the detected failures or defects are resolved which enables system more stable and reliable. Therefore, to understand the underlying condition of the system, such processes are described using a mathematical expression, usually based on parameters such as number of failure or failure density, etc [17]. The literature reports many ways to create models based on the model's assumption of failure occurrence patterns.…”