Modeling multi‐way data with linearly dependent loadings

Abstract

A generalization/specialization of the PARAFAC model is developed that improves its properties when applied to multi-way problems involving linearly dependent factors. This model is called PARALIND (PARAllel profiles with LINear Dependences). Linear dependences can arise when the empirical sources of variation being modeled by factors are causally or logically linked during data generation, or circumstantially linked during data collection. For example, this can occur in a chemical context when end products are related to the precursor or in a psychological context when a single stimulus generates two incompatible feelings at once. For such cases, the most theoretically appropriate PARAFAC model has loading vectors that are linearly dependent in at least one mode, and when collinear, are nonunique in the others. However, standard PARAFAC analysis of fallible data will have neither of these features. Instead, latent linear dependences become high surface correlations and any latent nonuniqueness is replaced by a meaningless surface-level 'unique orientation' that optimally fits the particular random noise in that sample. To avoid these problems, any set of components that in theory should be rank deficient are re-expressed in PARALIND as a product of two matrices, one that explicitly represents their dependency relationships and another, with fewer columns, that captures their patterns of variation. To demonstrate the approach, we apply it first to fluorescence spectroscopy (excitation-emission matrices, EEM) data in which concentration values for two analytes covary exactly, and then to flow injection analysis (FIA) data in which subsets of columns are logically constrained to sum to a constant, but differently in each of two modes. In the PARAFAC solutions of the EEM data, all factors are 'unique' but this is only meaningful for two of the factors that are also unique at the latent level. In contrast, the PARALIND solutions directly display the extent and nature of partial nonuniqueness present at the latent level by exhibiting a corresponding partial uniqueness in their recovered loadings. For the FIA data, PARALIND constraints restore latent uniqueness to the concentration estimates. Comparison of the solutions shows that PARALIND more accurately recovers latent structure, presumably because it uses fewer parameters and hence fits less error.

From basis components to complex structural patterns

“…For example, motor imagery activities in a BCI system yield several rhythms which usually occur over the motor and sensory cortices of the brain. This linear dependence problem also commonly appears in fluorescence excitationemission data, or in flow injection analysis (FIA) [6].…”

“…al. proposed the structured CP decomposition (CPD) with additional dependence matrices Q n which often consist of zeros and ones [1,6], that is,…”

“…A partial uniqueness of FIA-type PARALIND with doubly linear dependence was investigated in [6]. In another direction, L. de Lathauwer [8] restricted interactions among components in sub tensors instead of completely spreading entries of the core tensor in the Tucker decomposition, which leads to the rank-(L r , L r , 1) block term decomposition (BTD) for order-3 tensors, and the generalized rank-…”

“…The first new CP-like model has factor matrices in the Kronecker form (product) which allows factorization of large-scale tensors with (very) high rank, say R > 1000. The second CP model is a particular case of the former one, but it can be considered as generalization of the PARALIND model for rank-overlap problems [6], or the BTD model [10]. The models have been verified on both synthetic and real data, using a real application for clustering.…”

“…Such behavior is related to the rank-overlap problem which often exists in real-world signals such as chemical data, flow injection analysis (FIA) data [1,6], or spectral tensors of EEG signals. The structured CPD in Lemma 2.2 is a particular case of the CP model with linearly dependent loadings (PARALIND) in (2) [6] in which the dependency matrices are fixed and given in Lemma 2.2. Discussions on uniqueness of the CPD with linearly dependent loadings can be found in [11,14].…”