Evaluation of Empirical Bayes Estimators of Correlated Event Rates and Implications for Predicting new Product Reliability

Abstract

A method for estimating the reliability of a new product based on a comparative analysis of observed data for similar existing products has been motivated by an industry problem; see Quigley and Walls (to appear). Such estimates are required as part of contractual discussions with customers, as well as to inform the reliability program management for the new design. An empirical Bayes inference method has been developed based on a multivariate Poisson-Gamma probability model. The model aims to capture both aleatory and epistemic uncertainties. The latter are those which have the potential to be bought down by gathering more information such as learning about the existence of a potential design weaknesses. The model is multivariate since it involves modelling elements of the new product design reliability in relation to a set of relevant elements from multiple similar products. See, Quigley et al ( 2013) for more details of probability modelling of correlated events which provides a theoretical framework, and Quigley and Walls (2017) for an approach to construct prior distributions using empirical data. The probability model contains a correlation parameter capturing the strength of the statistical dependency between the anticipated reliability of the new design element and the reliability of the corresponding element in each of the similar product. Practically, this correlation is assessed by relevant engineers using an approach grounded in the paired comparisons method used in structured expert judgement. Scientifically, we aim to evaluate the statistical performance of the probability inference. Our first objective is to investigate the accuracy of estimators for different parameter settings under different sample sizes for the comparator data sets of events for similar products. Our second objective is to explore the robustness of results and hence the implications of misspecification of the correlation parameter as might occur in expert elicitation. Our scientific study is based upon theoretical analysis and simulation experiments. We discuss the implications of the scientific results for the practical use of the probability modelling approach. We advise on situations where the inference can be used and when it should not be used based on the amount of relevant empirical data for comparative products. We show the robustness of the reliability estimation for different degrees of misspecification of the statistical dependency between the reliability of the elements of the new and existing products.