Microbiome Preterm Birth DREAM Challenge: Crowdsourcing Machine Learning Approaches to Advance Preterm Birth Research

Golob, Jonathan L.; Oskotsky, Tomiko; Tang, Alice; Roldan, Alennie; Chung, Verena; Ha, Connie; Wong, Ronald J.; Flynn, Kaitlin J.; Parraga-Leo, Antonio; Wibrand, Camilla; Minot, Samuel S.; Andreoletti, Gaia; Kosti, Idit; Bletz, Julie; Nelson, Amber; Gao, Jian; Wei, Zhoujingpeng; Chen, Guanhua; Tang, Zheng-Zheng; Novielli, Pierfrancesco; Romano, Donato; Pantaleo, Ester; Amoroso, Nicola; Monaco, A.; Vacca, Mirco; Angelis, Maria De; Bellotti, Roberto; Tangaro, Sabina; Kuntzleman, Abigail; Bigcraft, Isaac; Techtmann, Stephen M.; Bae, Daehun; Kim, Eun Young; Jeon, Jongbum; Joe, Soobok; Theis, Kevin R.; Ng, Sherrianne; Li, Yun S. Lee; Bennett, Phillip R.; MacIntyre, David A.; Stolovitzky, Gustavo; Lynch, Susan V.; Albrecht, Jake; Gomez‐Lopez, Nardhy; Romero, Roberto; Stevenson, David K.; Aghaeepour, Nima; Tarca, Adi L.; Costello, James C.; Sirota, Marina

doi:10.1101/2023.03.07.23286920

Cited by 10 publications

(13 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, we reanalyzed data from a study of the vaginal microbiome and preterm birth, and examined the first time point from each of the 40 individuals included in the study 13 (Methods). As previously reported [22][23][24] , we observed an association between α diversity and subsequent preterm birth (Fig. 2a, Mann-Whitney U p = 0.037).…”

Section: Clr-transformed Sparse Features Can Be Associated With Clini...supporting

confidence: 88%

Compositional transformations can reasonably introduce phenotype-associated values into sparse features

Austin,

Korem

2024

Preprint

View full text Add to dashboard Cite

It was recently argued1that an analysis of tumor-associated microbiome data2is invalid because features that were originally very sparse (genera with mostly zero read counts) became associated with the phenotype following batch correction1. Here, we examine whether such an observation should necessarily indicate issues with processing or machine learning pipelines. We focus on the centered log ratio (CLR) transformation, which is often recommended for analysis of compositional microbiome data3. The CLR transformation has similarities to Voom-SNM4,5, the batch-correction method brought into question1,2, yet is a sample-wise operation that cannot, in itself, “leak” information or invalidate downstream analyses. We show that because the CLR transformation divides each value by the geometric mean of its sample, common imputation strategies for missing or zero values result in transformed features that are associated with the geometric mean. Through analyses of both synthetic and vaginal microbiome datasets we demonstrate that when the geometric mean is associated with a phenotype, sparse and CLR-transformed features will also become associated with it. We re-analyze features highlighted by Gihawi et al.1and demonstrate that the phenomena of sparse features becoming phenotype-associated can also be observed after a CLR transformation. While we do not intend to validate tumor-associated microbiome signatures2or evaluate other concerns regarding their detection and analysis1,6, we conclude that as phenotype-associated features that were initially sparse can be created by a sample-wise transformation that cannot artifactually inflate machine learning performance, their detection is not independently sufficient to demonstrate an analytic issue in machine learning pipelines. However, as was also previously noted by others, features transformed with sample-wise operations such as the CLR transformation should be interpreted with caution.

show abstract

Section: Clr-transformed Sparse Features Can Be Associated With Clini...supporting

confidence: 88%

Compositional transformations can reasonably introduce phenotype-associated values into sparse features

Austin,

Korem

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Stabl was also tested on a dataset where previous models did not perform as well (AUROC < 0.7). The Microbiome Preterm Birth DREAM challenge aimed to classify pre-term (PT) and term (T) labor pregnancies using nine publicly available vaginal microbiome (phylotypic and taxonomic) datasets 48,49 . The top 20 models submitted by 318 participating analysis teams achieved AUROC scores between 0.59 and 0.69 for the task of predicting PT delivery.…”

Section: Articlementioning

confidence: 99%

Discovery of sparse, reliable omic biomarkers with Stabl

Hédou,

Marić,

Bellan

et al. 2024

Nat Biotechnol

Self Cite

View full text Add to dashboard Cite

Adoption of high-content omic technologies in clinical studies, coupled with computational methods, has yielded an abundance of candidate biomarkers. However, translating such findings into bona fide clinical biomarkers remains challenging. To facilitate this process, we introduce Stabl, a general machine learning method that identifies a sparse, reliable set of biomarkers by integrating noise injection and a data-driven signal-to-noise threshold into multivariable predictive modeling. Evaluation of Stabl on synthetic datasets and five independent clinical studies demonstrates improved biomarker sparsity and reliability compared to commonly used sparsity-promoting regularization methods while maintaining predictive performance; it distills datasets containing 1,400–35,000 features down to 4–34 candidate biomarkers. Stabl extends to multi-omic integration tasks, enabling biological interpretation of complex predictive models, as it hones in on a shortlist of proteomic, metabolomic and cytometric events predicting labor onset, microbial biomarkers of pre-term birth and a pre-operative immune signature of post-surgical infections. Stabl is available at https://github.com/gregbellan/Stabl.

show abstract

“…The results are presented in figure 6. Albeit not on the same validation dataset, our approach describes better results than the best submission in the DREAM challenge for term vs preterm prediction (ROC-AUC = 0.68, accuracy = 67%, sensitivity = 0.48, specificity = 0.79) [26], despite using microbial abundances only up to the end of the 2 nd trimester, i.e., the 24 th week of gestation, as opposed to the 32 nd week of gestation in the DREAM challenge.…”

Section: Resultsmentioning

confidence: 99%

“…One of the sub-problems for the DREAM challenge consisted of predicting term births (>= 37 weeks of gestation) and preterm births (< 37 weeks of gestation) using vaginal microbiomes. The dataset for this challenge was derived from 9 different studies, and amounted to 3578 samples collected from 1268 individuals [26]. The dataset used in this study was also part of the DREAM challenge.…”

Section: Discussionmentioning

confidence: 99%

“…Most of the top submissions in the challenge used tree-based classifiers. On our test dataset, neural CDEs show better predictivity (mean test set ROC-AUC = 0.82, accuracy = 75%, sensitivity = 0.71, specificity = 0.85) than the best submission in the DREAM challenge on their validation dataset (ROC-AUC = 0.69, accuracy = 67%, sensitivity = 0.48, specificity = 0.79) [26], despite using microbial abundances only up to the end of the 2 nd trimester, i.e., the 24 th week of gestation. The DREAM challenge for PTB prediction used taxonomic abundance data upto the 32 nd week of gestation.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Neural differential equations enable early-stage prediction of preterm birth using vaginal microbiota

Karambelkar,

Baranwal

2023

Preprint

View full text Add to dashboard Cite

Preterm births (PTBs), i.e., births before 37 weeks of gestation are completed, are one of the leading issues concerning infant health, and is a problem that plagues all parts of the world. Millions of infants are born preterm globally each year, resulting in developmental disorders in infants and increase in neonatal mortality. Although there are known risk factors for PTB, the current procedures used to assess PTB risk are effective only at the later stages of pregnancy, which reduces the impact of currently possible interventions administered to prevent PTB or mitigate its ill-effects. Vaginal microbial communities have recently garnered attention in the context of PTB, with the notion that a highly diverse microbiome is detrimental as far as PTB is concerned. Increased abundance or scarcity of certain microbial species belonging to specific genera has also been linked to PTB risk. Consequently, attempts have been made towards establishing a correlation between alpha-diversity indices associated with vaginal microbial communities, and PTB. However, the vaginal microbiome varies greatly from individual to individual, and this variation is more pronounced in racially, ethnically and geographically diverse populations, which diversity indices may not be able to overcome. Machine learning (ML)-based approaches have also previously been explored, however, the success of these approaches reported thus far has been limited. Additionally, microbial communities have been reported to evolve during the duration of the pregnancy, and capturing such a signature may require higher, more complex modeling paradigms. Thus, alternative approaches are necessary to identify signatures in these microbial communities that are capable of distinguishing PTB from a full-term pregnancy. In this study, we have highlighted the limitations of diversity indices for prediction of PTB in racially diverse cohorts. We applied Deep Learning (DL)-based methods to vaginal microbial abundance profiles obtained at various stages of pregnancy, and Neural Controlled Differential Equations (CDEs) are able to identify a signature in the temporally-evolving vaginal microbiome during trimester 2 and can predict incidences of PTB (mean test set ROC-AUC = 0.81, accuracy = 75%, F1-score = 0.71) significantly better than traditional ML classifiers such as Random Forests (mean test set ROC-AUC = 0.65, accuracy = 66%, F1-score = 0.42) and Decision Trees (mean test set ROC-AUC = 0.48, accuracy = 46%, F1-score = 0.40), thus enabling effective early-stage PTB risk assessment.

show abstract

Microbiome Preterm Birth DREAM Challenge: Crowdsourcing Machine Learning Approaches to Advance Preterm Birth Research

Cited by 10 publications

References 67 publications

Compositional transformations can reasonably introduce phenotype-associated values into sparse features

Compositional transformations can reasonably introduce phenotype-associated values into sparse features

Discovery of sparse, reliable omic biomarkers with Stabl

Neural differential equations enable early-stage prediction of preterm birth using vaginal microbiota

Contact Info

Product

Resources

About