Benchmark of data processing methods and machine learning models for gut microbiome-based diagnosis of inflammatory bowel disease

Abstract

BackgroundInflammatory bowel disease (IBD) patients wait months and undergo numerous invasive procedures between the initial appearance of symptoms and receiving a diagnosis. In order to reduce time until diagnosis and improve patient wellbeing, machine learning algorithms capable of diagnosing IBD from the gut microbiome’s composition are currently being explored. To date, these models have had limited clinical application due to decreased performance when applied to a new cohort of patient samples. Various methods have been developed to analyze microbiome data which may improve the generalizability of machine learning IBD diagnostic tests. With an abundance of methods, there is a need to benchmark the performance and generalizability of various machine learning pipelines (from data processing to training a machine learning model) for microbiome-based IBD diagnostic tools.ResultsWe collected fifteen 16S rRNA microbiome datasets (7707 samples) from North America to benchmark combinations of gut microbiome features, data normalization methods, batch effect reduction methods, and machine learning models. Pipeline generalizability to new cohorts of patients was evaluated with four binary classification metrics following leave-one dataset-out cross validation, where all samples from one study were left out of the training set and tested upon. We demonstrate that taxonomic features obtained from QIIME2 lead to better classification of samples from IBD patients than inferred functional features obtained from PICRUSt2. In addition, machine learning models that identify non-linear decision boundaries between labels are more generalizable than those that are linearly constrained. Prior to training a non-linear machine learning model on taxonomic features, it is important to apply a compositional normalization method and remove batch effects with the naive zero-centering method. Lastly, we illustrate the importance of generating a curated training dataset to ensure similar performance across patient demographics.ConclusionsThese findings will help improve the generalizability of machine learning models as we move towards non-invasive diagnostic and disease management tools for patients with IBD.

Decision Tree Ensembles Utilizing Multivariate Splits Are Effective at Investigating Beta-Diversity in Medically Relevant 16S Amplicon Sequencing Data

“…These models, which are not linearly constrained, have been shown to generalize well to unseen data in more recent amplicon sequencing studies (12).…”

“…For example, RFs have been recently applied to study and identify operational taxonomic units (OTU), which can be considered a class of biomarkers, from the microbiomes of patients suffering from cardiovascular disease, chronic obstructive pulmonary disease, and various immune-mediated inflammatory diseases (3,10,11). These models, which are not linearly constrained, have been shown to generalize well to unseen data in more recent amplicon sequencing studies (12).…”

“…Another classification algorithm, k-nearest neighbors, is sensitive to the number of neighbors and the distance metric (15). Logistic regression, ridge regression, and linear support vector classifiers can only learn linear decision boundaries (12). while neural networks can require a large amount of data and time to learn appropriate weights for each parameter.…”

“…For example, each decision tree in a RF uses a recursive series of axis-orthogonal splits to approximate the underlying data generating function (13,14). However, more complex oblique or non-linear splits often result in more appropriate representations of the data generating function (12,14). Another classification algorithm, k-nearest neighbors, is sensitive to the number of neighbors and the distance metric (15).…”

Conditional forest models built using metagenomic data could accurately predictSalmonellacontamination in Northeastern streams

“…In addition to model selection, the performance of a model may be affected by microbiome data pre-processing, such as data normalization (28). The latter is commonly carried out to account for potential differences in sample library sizes, hence its effect on prediction accuracy needs to be assessed (31)(32)(33).…”