Modeling Launch Vehicle Reliability Growth as Defect Elimination

Abstract

The historical success and failure record of launch vehicles clearly demonstrates the presence of reliability growth over successive launches. The reality of reliability growth is critical to decisions on ground and flight testing programs, and is a much greater driver of the expected number of failures over a campaign, than the best analysis of mature reliability can ever be. While mathematical models exist that match the reliability growth demonstrated by historical systems, the space industry is still lacking a practical method to develop forecasts of reliability growth for new systems, update those forecasts on the basis of early tests and flight results, and accurately estimate integrated campaign metrics over several launches. Modeling the failure probability as originating from potential "defects" in the system, each with a probability of trigger, a conditional probability of causing loss of mission, and a probability of detection and correction, provides a starting place to address this need. The method provides a model of reliability growth that is mathematically sound, matches historical results, is directly amenable to system engineering inputs, clearly identifies and quantifies the drivers of reliability growth, and provides a clear basis for uncertainty analysis and Bayesian updating. Nomenclature p min = irreducible (mature) probability of failure of the system D = number of defect types defined in the model d k = initial number of defects of type k present in the system λ k = given presence of a defect of type k, probability of defect trigger at any given flight τ k = given presence and trigger of defect of type k, conditional probability of loss of mission ω = given loss of mission, conditional probability of loss of crew ( = 1−ΑΕ) ν k = after defect of type k caused partial anomaly, conditional probability that it is observable ϕ k = after triggered defect of type k is observable, conditional probability that it is noticed and reported γ k = after a defect of type k is uncovered, conditional probability to eliminate the defect N = index of flight (or test) number p k = λ k τ k = probability of system failure from a defect of type k, if the defect is present in the system ρ k = ν k ϕ k γ k = after defect of type k caused partial anomaly, conditional probability to eliminate it η k = 1/ τ k = mean number of triggers of a defect of type k for one trigger to cause loss of mission α k = (η k -1)ρ k = measure of the potential to eliminate a defect of type k before it causes loss of mission ζ k = (η k -1) ν k ϕ k = measure of the potential to report a defect of type k before it causes loss of mission δ k (N) = probability that a given defect of type k is still present in the system at flight number N ε k = 'test effectiveness' for defects of type k = probability of defect elimination at each flight (or test) P k (N) = probability of loss of mission at flight number N from a single defect of type k R(N)= system reliability at flight number N Δ k (N) = probability to not have any loss of mission in the fi...

Developing probabilistic safety performance margins for unknown and underappreciated risks

“…For example, the Atlas launch vehicle had the first ten failures occurring within the first 19 flights, so we took a failure rate of 10/19 = 0.53 as being representative of the initial failure rate. Others have analyzed the same data using other approaches, including Bayesian [19,20] and frequentist [21,22]. Not surprisingly, the different methods have resulted in significantly different estimates of the initial failure rate, both higher and lower than the results we obtained using a running snapshot.…”

Launch vehicle reliability growth

“…A failure occurs when an anomalous condition is severe enough to cross the threshold failure criteria, or if the condition persists for a long enough time period to cause damage to the overarching system structure. The elimination of defects in early flights has been demonstrated historically to be the driving element in reliability growth [1].…”

“…Total failure frequency was a common method used to model reliability growth in the past; however, that approach was recognized to produce a conservative estimate of reliability. Some have attempted to compensate for that effect by using 'discount factors' that weigh older failures lower than recent ones [1]. Discounting for older failures is one attempt at acknowledging reliability growth of historical systems, but it does not help with forecasts for new launchers [7].…”

“…It often grows because defects are discovered and the design is improved over time and over flights or tests of the LV [1]. The Space Shuttle Main Engine (SSME) is a very good example of this concept.…”

“…The mature risk estimates produced by this methodology under-predicted the risk observed when examining the historical record of LVs during early flights [5]. To account for this discrepancy, the next iteration of the model development then sought to rectify this problem by including the modeling of LV reliability growth from a low initial reliability to maturity in terms of defect elimination [1].…”