Aspects of pseudorank estimation methods based on an estimate of the size of the measurement error

Abstract

The estimation of the pseudorank of a matrix, i.e., the rank of a matrix in the absence of measurement error, is a major problem in multivariate data analysis. In the practice of analytical chemistry it is often even the only problem. An important example is the determination, of the purity of a chromatographic peak. In this paper we discuss three pseudorank estimation methods that make use of prior knowledge about the size of the measurement error. The first method (Method A) is based on the standard errors in the diagonal elements of the row-echelon form of the matrix, the second method (Method Bl is based on the eigenvalues of principal component analysis @CA) and the third method (a t-test) is based on the singular values. Methods A and B are modifications of methods that are well known in analytical chemistry. However, these methods cannot provide significance levels for the estimated pseudorank. This holds for the original methods as well as the present modifications. The main reason for introducing these modifications is that in this way relationships are established between the t-test and methods that are already known. The aspects that are covered in this paper include the sampling distribution of the test statistic, the number of degrees of freedom to be used in the test, the adequacy of theoretical predictions and the bias that results from random measurement noise. The object of this paper is to demonstrate that using prior knowledge about the size of the measurement error may yield powerful pseudorank estimation methods. This is illustrated by comparing the significance levels obtained by the r-test and Malinowski's F-test. The r-test yields sharper significance levels for experimental data obtained from the literature as well as simulated data. This can be satisfactorily explained by the larger number of degrees of freedom that is employed in this test. The viability of the new t-test is supported by a thorough evaluation of the test data by a large number of conventional methods. As a remarkable by-product of the present investigation we find that a plot of the singular values yields a promising graphical pseudorank estimation method. Graphical methods have proved their use in the past in the case that the size of the measurement error is unknown. This new graphical method therefore provides a natural complement to the t-test.

Modification of Malinowski'sF-test for abstract factor analysis applied to the Quail Roost II data sets

“…This leads to a pseudorank estimate of seven, which can also be inferred from the reduced eigenvalues in the next column ( λ 8 λ 20 ). It is emphasized that the reduced eigenvalues 8 -20 are not constant owing to the approximate character of equation (2). Rather the curve of the secondary reduced eigenvalues goes through a distinct minimum.…”

“…From equation (2) one may readily infer that the degrees of freedom associated with a secondary factor are given by…”

Enhancing the Limit of Detection for Comprehensive Two-Dimensional Gas Chromatography (GC×GC) using Bilinear Chemometric Analysis

“…The number of components was estimated by inspection of the singular values [36]. Estimation of the number of components in a bilinear matrix has been addressed in many articles [25, 36 -38].…”

Selecting significant factors by the noise addition method in principal component analysis

“…Statistical methods often assume that the errors in the data set are normally distributed throughout the matrix and are independent of the signal at any indices. In this paper, four statistical methods-residual standard deviation (RSD) [4], chi-squared ( 2 ) [4], cross-validation (CV) [4] and the factor indicator function (IND) [4], an empirical method-were compared with the NAM. These criteria were chosen because each can be automated and they work best under different conditions and have different assumptions (Table I).…”

“…They are divided into two sections: methods based on the imbedded error and methods based on external error information. Faber et al [4] describe advantages to having prior knowledge of the size of the measurement errors when estimating the number of factors in the data set.…”