Estimating Return on Investment for SAVI (a Model-Based Virtual Integration Process)

Reliability Validation and Improvement Framework

“…Such methods result in avoidance of certain rework and reduction of retest cycles, thus reducing error leakage and increasing our confidence in the qualification evidence. For this metric, we can draw on two pieces of work: the AVSI SAVI return-on-investment (ROI) study [Ward 2011] and the COnstructive QUALity MOdel (COQUALMO) work [Madachy 2010]. Both these efforts draw on a taxonomy and on results of a NASA study by Hayes [Hayes 2003].…”

“…The first phase, shown as the proof-of-concept loop, was completed in 2009. It included a proof-ofconcept demonstration (see Section 3.2.4) and an ROI study (see Section 4.3) [Ward 2011]. The objectives of the SAVI initiative coincide with the objectives of the reliability validation and improvement framework, with the latter placing more focus on the qualification of softwarereliant systems.…”

“…This expansion requires − evaluating existing SILs (as well as the proposed VSILs) by identifying problem categories they can discover early now and problem categories that currently leak to later phases but could be addressed earlier by improved capability of a SIL or VSIL − piloting the concept of a VSIL in an Army lab on an actual system to validate the ability of VSILs to discover certain problem categories − developing an error-leakage-reduction-driven ROI model that predicts cost savings achieved in qualification through rework and retest cost avoidance using VSILs and value-added SILs. As outlined in Section 4.3, this can be achieved by adapting the ROI model developed under SAVI Phase 1 [Ward 2011] and incorporating aspects of the COQUALMO [Madachy 2010], as well as calibrating it with Army-specific data. reflecting in the argumentation aspect of assurance cases, the risk of not providing sufficient or sufficiently qualified evidence.…”

“…The effects of acting on this early discovery are reduced error leakage rates to later development phases (e.g., residual defect prediction through the COQUALMO [Madachy 2010]) and major system cost savings through rework and retest avoidance (e.g., as demonstrated by the SAVI ROI study [Ward 2011]). We can leverage these cost models to guide the cost-effective application of appropriate validation methods.…”