Developing standards for the microbiome field

Abstract

Background: Effective standardisation of methodologies to analyse the microbiome is essential to the entire microbiome community. Despite the microbiome field being established for over a decade, there are no accredited or certified reference materials available to the wider community. In this study, we describe the development of the first reference reagents produced by the National Institute for Biological Standards and Control (NIBSC) for microbiome analysis by next-generation sequencing. These can act as global working standards and will be evaluated as candidate World Health Organization International Reference Reagents. Results: We developed the NIBSC DNA reference reagents Gut-Mix-RR and Gut-HiLo-RR and a four-measure framework for evaluation of bioinformatics tool and pipeline bias. Using these reagents and reporting system, we performed an independent evaluation of a variety of bioinformatics tools by analysing shotgun sequencing and 16S rRNA sequencing data generated from the Gut-Mix-RR and Gut-HiLo-RR. We demonstrate that key measures of microbiome health, such as diversity estimates, are largely inflated by the majority of bioinformatics tools. Across all tested tools, biases were present, with a clear trade-off occurring between sensitivity and the relative abundance of false positives in the final dataset. Using commercially available mock communities, we investigated how the composition of reference reagents may impact benchmarking studies. Reporting measures consistently changed when the same bioinformatics tools were used on different community compositions. This was influenced by both community complexity and taxonomy of species present. Both NIBSC reference reagents, which consisted of gut commensal species, proved to be the most challenging for the majority of bioinformatics tools tested. Going forward, we recommend the field uses site-specific reagents of a high complexity to ensure pipeline benchmarking is fit for purpose. Conclusions: If a consensus of acceptable levels of error can be agreed on, widespread adoption of these reference reagents will standardise downstream gut microbiome analyses. We propose to do this through a large open-invite collaborative study for multiple laboratories in 2020.

Effects of Low and High FODMAP Diets on Human Gastrointestinal Microbiota Composition in Adults with Intestinal Diseases: A Systematic Review

“…To compare findings and allow meta-analyses, it is important that samples are collected, stored, processed, and analyzed in a consistent manner, using appropriate computational biology tools. Standardization efforts conducted by large microbiome consortia could guide study protocols [94][95][96]. In addition, authors should deposit their raw data in public repositories to ensure future data availability.…”

Measuring the microbiome: Best practices for developing and benchmarking microbiomics methods

“…Having a diverse collection of mock communities makes mockrobiota useful for benchmarking purposes, as methods can be tested on multiple communities and multiple sequencing platforms to avoid overfitting to single test datasets (discussed below). Physical mock communities can also be acquired from various government [133] and commercial sources including ATCC ( https://www.lgcstandards-atcc.org/ ) and Zymo Research ( https://www.zymoresearch.com/ ), enabling researchers to use these as positive controls in sequencing, or to analyze with specific technological platforms.…”

“…This should include a “ground truth” of some type, e.g., samples with known composition. This will normally consist of one or more mock communities or simulated datasets (as discussed above) [123] , [133] , [135] . In some cases, a ground truth may not be necessary, for example to measure the consistency among methods rather than their accuracy [170] Test data should be appropriate to the intended usage, and for the different use cases that a user can reasonably expect to encounter (if those different use cases can be expected to impact performance).…”

Species-resolved sequencing of low-biomass microbiomes by 2bRAD-M

“…To further probe 2bRAD-M proficiency in profiling highly complex microbial community at low biomass setting, we used ATCC MSA1002, a standard, commercially available mock community consisting of 20 bacterial species (from 18 genera) with at equal DNA abundance among species 18 (Methods). Samples with total DNA amount range from 10 ng, 1 ng, 100 pg, 10 pg and 1 pg were prepared from this mock for 2bRAD-M profiling.…”