An Iterative Penalized Least Squares Approach to Sparse Canonical Correlation Analysis

Abstract

It is increasingly interesting to model the relationship between two sets of highdimensional measurements with potentially high correlations. Canonical correlation analysis (CCA) is a classical tool that explores the dependency of two multivariate random variables and extracts canonical pairs of highly correlated linear combinations. Driven by applications in genomics, text mining, and imaging research, among others, many recent studies generalize CCA to high-dimensional settings. However, most of them either rely on strong assumptions on covariance matrices, or do not produce nested solutions. We propose a new sparse CCA (SCCA) method that recasts high-dimensional CCA as an iterative penalized least squares problem. Thanks to the new iterative penalized least squares formulation, our method directly estimates the sparse CCA directions with efficient algorithms. Therefore, in contrast to some existing methods, the new SCCA does not impose any sparsity assumptions on the covariance matrices. The proposed SCCA is also very flexible in the sense that it can be easily combined with properly chosen penalty functions to perform structured variable selection and incorporate prior information. Moreover, our proposal of SCCA produces nested solutions and thus provides great convenient in practice. Theoretical results show that SCCA can consistently estimate the true canonical pairs with an overwhelming probability in ultra-high dimensions. Numerical results also demonstrate the competitive performance of SCCA.

Multivariate association between brain function and eating disorders using sparse canonical correlation analysis

“…CCA is a useful method to identify the multivariate association between two types of highdimensional features (e.g., neuroimaging and questionnaire) [18]. Sparse CCA (SCCA) is an extension of CCA, which aims to identify a sparse set of loading vectors in two types of features using a regularization technique [19][20][21][22][23][24][25][26][27][28][29][30]. SCCA could be more suitable than the regular CCA since it helps to reduce the overfitting problem, which is becoming more problematic with increased dimensionality of modern neuroimaging data.…”

Multi-omics reveals microbiome, host gene expression, and immune landscape in gastric carcinogenesis

“…To identify significant associations between the intragastric microbiome and host gastric mucosa transcriptome, we performed CCA, which utilizes dimensionality reduction to identify correlated sets of variables (i.e., canonical variates) from two different data types ( Mai and Zhang, 2019 ; Witten et al., 2009 ). For the first few strongly correlated (microbial and host gene expression) canonical variates in the CCA model, bacterial ASVs having positive canonical coefficients can be generally regarded as being associated with genes having positive canonical coefficients.…”

Sparse Reduced-Rank Regression for Exploratory Visualisation of Paired Multivariate Data

“…(2009) maximises covariance (and so could in fact have been called ‘sparse PLS’ and not ‘sparse CCA’), which may provide too strong ℓ 2 regularisation. More recent sparse CCA methods (Mai & Zhang, 2019; Suo et al., 2017) have not been benchmarked here.…”

“…A statistical method that directly maximises correlation between Xw and Yv is called canonical correlation analysis (CCA). A number of different methods for sparse CCA have been suggested in the last decade (Chen et al., 2012b; Chu et al., 2013; Gao et al., 2017; Hardoon & Shawe‐Taylor, 2011; Lykou & Whittaker, 2010; Mai & Zhang, 2019; Parkhomenko et al., 2009; Suo et al., 2017; Waaijenborg et al., 2008; Wiesel et al., 2008; Wilms & Croux, 2015; Witten & Tibshirani, 2009; Witten et al., 2009), of which the sparse CCA of Witten et al. (2009) is arguably the most well‐known (judging by the number of citations in Google Scholar at the time of writing).…”

“…They can be understood as looking for projections with maximal correlation (CCA) or maximal covariance (PLS) between X and Y , whereas RRR looks for projections with maximal explained variance in Y . In recent years, multiple approaches to sparse CCA (Chen et al., 2012b; Chu et al., 2013; Gao et al., 2017; Hardoon & Shawe‐Taylor, 2011; Lykou & Whittaker, 2010; Mai & Zhang, 2019; Parkhomenko et al., 2009; Suo et al., 2017; Waaijenborg et al., 2008; Wiesel et al., 2008; Wilms & Croux, 2015; Witten & Tibshirani, 2009; Witten et al., 2009) and sparse PLS (Chun & Keleş, 2010; Lê Cao et al., 2008, 2011) have been suggested in the literature. Here, we chose sparse RRR at the core of our framework, because for the Patch‐seq data it seems more meaningful to predict electrophyiological properties from transcriptomic information instead of treating them symmetrically, as genes give rise to physiological function.…”