Inferring Correlation Networks from Genomic Survey Data

Abstract

High-throughput sequencing based techniques, such as 16S rRNA gene profiling, have the potential to elucidate the complex inner workings of natural microbial communities - be they from the world's oceans or the human gut. A key step in exploring such data is the identification of dependencies between members of these communities, which is commonly achieved by correlation analysis. However, it has been known since the days of Karl Pearson that the analysis of the type of data generated by such techniques (referred to as compositional data) can produce unreliable results since the observed data take the form of relative fractions of genes or species, rather than their absolute abundances. Using simulated and real data from the Human Microbiome Project, we show that such compositional effects can be widespread and severe: in some real data sets many of the correlations among taxa can be artifactual, and true correlations may even appear with opposite sign. Additionally, we show that community diversity is the key factor that modulates the acuteness of such compositional effects, and develop a new approach, called SparCC (available at https://bitbucket.org/yonatanf/sparcc), which is capable of estimating correlation values from compositional data. To illustrate a potential application of SparCC, we infer a rich ecological network connecting hundreds of interacting species across 18 sites on the human body. Using the SparCC network as a reference, we estimated that the standard approach yields 3 spurious species-species interactions for each true interaction and misses 60% of the true interactions in the human microbiome data, and, as predicted, most of the erroneous links are found in the samples with the lowest diversity.

Microbial communities reflect temporal changes in cyanobacterial composition in a shallow ephemeral freshwater lake

“…In contrast with other studies, examining soil and sediment-bound microbial communities (Bissett et al, 2013;Sun et al, 2013), which have observed a lack of correlations between absolute abiotic measurements and 16S rRNA OTU relative abundances, numerous correlations were observed between biotic and abiotic factors. Although consideration should be made that such approaches, wherein relative abundances are used to derive correlations between species, may overestimate the numbers of true correlations (Friedman, 2012;r14933), this analysis was effective at classifying factors characteristic of the three observed periods, namely summer, autumn and winter periods, also observed in the dbRDA analysis (Figure 2). In this context, with cyanobacterial cell counts as the driver, 16S rRNA gene OTUs, cell counts and abiotic data were considered to be associated with each of the distinct periods when the observed variables exhibited strong positive correlations to either the cell counts or to one another ( Figure 5).…”

Correlation detection strategies in microbial data sets vary widely in sensitivity and precision

“…Also, as microbes live in communities, there are likely three-feature interactions, four-feature interactions and more. An additional challenge is that microbial sequence data provide relative abundances based on a fixed total number of sequences rather than absolute abundances, which introduces the problem of compositions (Lovell et al, 2010;Friedman and Alm, 2012). Sparsity of the features and missing data owing to incomplete sampling further complicates statistical analysis (Reshef et al, 2011;Friedman and Alm, 2012).…”

“…An additional challenge is that microbial sequence data provide relative abundances based on a fixed total number of sequences rather than absolute abundances, which introduces the problem of compositions (Lovell et al, 2010;Friedman and Alm, 2012). Sparsity of the features and missing data owing to incomplete sampling further complicates statistical analysis (Reshef et al, 2011;Friedman and Alm, 2012). Finally, microbes may display diverse types of relationships, such as linear, exponential or periodic, and most tests are not general enough to detect them all; even those that do are unlikely to detect different functions with the same efficiency (Reshef et al, 2011).…”

An in vitro culture model to study the dynamics of colonic microbiota in Syrian golden hamsters and their susceptibility to infection with Clostridium difficile

“…Effect of hamster microbiota on C. difficile M Miezeiewski et al (Friedman and Alm, 2012). Interesting phyla generally considered as environmental, such as the recently described Armatimonadetes (Tamaki et al, 2011) were not only encountered in the fecal samples but demonstrated significant increase in relative abundance after antibiotic application.…”