Dealing with missing data in MSPC: several methods, different interpretations, some examples

Abstract

This paper addresses the problem of using future multivariate observations with missing data to estimate latent variable scores from an existing principal component analysis (PCA) model. This is a critical issue in multivariate statistical process control (MSPC) schemes where the process is continuously interrogated based on an underlying PCA model. We present several methods for estimating the scores of new individuals with missing data: a so-called trimmed score method (TRI), a single-component projection method (SCP), a method of projection to the model plane (PMP), a method based on the iterative imputation of missing data, a method based on the minimization of the squared prediction error (SPE), a conditional mean replacement method (CMR) and various least squared-based methods: one based on a regression on known data (KDR) and the other based on a regression on trimmed scores (TSR). The basis for each method and the expressions for the score estimators, their covariance matrices and the estimation errors are developed. Some of the methods discussed have already been proposed in the literature (SCP, PMP and CMR), some are original (TRI and TSR) and others are shown to be equivalent to methods already developed by other authors: iterative imputation and SPE methods are equivalent to PMP; KDR is equivalent to CMR. These methods can be seen as different ways to impute values for the missing variables. The efficiency of the methods is studied through simulations based on an industrial data set. The KDR method is shown to be statistically superior to the other methods, except the TSR method in which the matrix to be inverted is of a much smaller size.

Cross-validation in PCA models with the element-wise k-fold (ekf) algorithm: Practical aspects

“…This was also the method to estimate the missing elements employed at model exploitation in the results shown in Figure 1. Direct estimation is well known to provide a sub-optimal estimation solution, as shown in [18] where it is referred as trimmed score imputation (TRI). Since other missing data estimation methods provide more accurate estimates than direct estimation, it is sensible to use these approaches instead of direct estimation in MD applications.…”

“…Therefore, if a different estimation method is used during model exploitation, the ekf needs to be modified accordingly. In this paper, the extension of ekf to three MD methods is considered: iterative imputation [28], Projection to Model Plane (PMP) [15] and Trimmed Score Regression (TSR) [18]. The last method is representative of regression-based imputation methods [19], some of which were also presented in [15].…”

“…Strictly speaking, the iterative estimation and PMP are different only when Ω hm in (A.9) has eigenvalues equal to 1 [18]. Numerically, differences can be observed when the eigenvalues approach 1.…”

“…For instance, when the objective is data compression or dimensionality reduction [11][12][13][14], probably the most frequent PCA application, the actual values are available and their estimates by PCA are employed as compressed data. On the other hand, when the data of the object is incomplete due to any problem during data collection, the actual values to estimate are not available and PCA is used to infer them [15][16][17][18][19]. In the following, this sort of applications will be termed Missing Data (MD) applications.…”

“…Forx (0) n,m = 0, it has been referred by [18] as trimmed score (TRI) imputation. The direct imputation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 can be extended to the more general case when the values of several variables are missing at the same time:…”