Large Covariance Estimation for Compositional Data Via Composition-Adjusted Thresholding

Abstract

High-dimensional compositional data arise naturally in many applications such as metagenomic data analysis. The observed data lie in a high-dimensional simplex, and conventional statistical methods often fail to produce sensible results due to the unit-sum constraint. In this article, we address the problem of covariance estimation for high-dimensional compositional data, and introduce a composition-adjusted thresholding (COAT) method under the assumption that the basis covariance matrix is sparse. Our method is based on a decomposition relating the compositional covariance to the basis covariance, which is approximately identifiable as the dimensionality tends to infinity. The resulting procedure can be viewed as thresholding the sample centered log-ratio covariance matrix and hence is scalable for large covariance matrices. We rigorously characterize the identifiability of the covariance parameters, derive rates of convergence under the spectral norm, and provide theoretical guarantees on support recovery.Simulation studies demonstrate that the COAT estimator outperforms some naive thresholding estimators that ignore the unique features of compositional data. We apply the proposed method to the analysis of a microbiome dataset in order to understand the dependence structure among bacterial taxa in the human gut.

Microbial networks in SPRING - Semi-parametric rank-based correlation and partial correlation estimation for quantitative microbiome data

“…One of the key challenges in working with compositional data is the presence of unit-sum constraint. For correlation estimation, a common approach (see, e.g., Aitchison (1983); Cao et al (2018); Kurtz et al (2015)) is to first apply the centered log-ratio transform (clr) to the compositional vector of each sample x i ∈ S p…”

“…Microbiome community-level analysis tasks, such as quantifying community composition shifts across conditions or associating high-dimensional species compositions and their taxonomic profiles to each other and to environmental or host-associated covariates, require statistical estimation procedures that can handle the restrictive nature of such sparse proportional (or compositional) microbiome datasets (Li, 2015). Important examples include differential abundance techniques (Mandal et al, 2015;McMurdie and Holmes, 2014), proportionality estimation (Quinn et al, 2017), regression models with compositional covariates (Holmes et al, 2012;Lin et al, 2014), composition-adjusted correlation estimation techniques (Cao et al, 2018;Friedman and Alm, 2012), and sparse graphical models for microbial association networks (Kurtz et al, 2015;Tipton et al, 2018).…”

“…For these reasons, the sample covariance matrix is not appropriate for capturing dependencies present in microbiome data. Several methods use techniques from compositional data analysis (Aitchison, 1983), including log-ratio transforms, to adjust the data prior to any estimation, and enforce different structural constraints on the correlation or inverse correlation matrix (Cao et al, 2018;Friedman and Alm, 2012;Kurtz et al, 2015). The problem of excess zeros is typically dealt with by adding an arbitrary pseudocount or, more recently, estimating the pseudocounts from multiple samples (Cao et al, 2017).…”

Disentangling microbial associations from hidden environmental and technical factors via latent graphical models

“…A popular linear measure of association is covariance or, when data are standardized, Pearson correlation. In the microbiome context, covariance estimators are available both for compositional data [20,8] and for absolute microbial abundance data [54]. To account for transitive correlations, estimators of the inverse covariance (or precision) matrix are also available [30,16].…”

“…We first investigated the ability of the latent graphical model (SE-slr) to identify direct associations that are obfuscated by purely compositional effects. We compared its performance to SE-glasso and the compositionadjusted thresholding (COAT) estimator [8] under two different scenarios: sparse networks generated (i) from synthetic absolute abundance data and (ii) measured quantitative microbiome profiling (QMP) data [51].…”