Visibiome: an efficient microbiome search engine based on a scalable, distributed architecture

Azman, Syafiq Kamarul; Anwar, Muhammad Zohaib; Henschel, Andreas

doi:10.1186/s12859-017-1763-0

“…Their approach relies on specific hardware (GPUs) to achieve low latency results for queries. The authors of Reference 14 built Visibiome , a scalable search architecture for microbiomes. Their solution is a distributed web application that only scales horizontally when dealing with increased number of parallel queries.…”

Section: Related Workmentioning

confidence: 99%

Evaluation of a scientific data search infrastructure

Orhean

¹

,

Giannakou

²

,

Antypas

³

et al. 2022

Concurrency and Computation

2

0

View full text Add to dashboard Cite

The ability to search over large scientific datasets has become crucial to next-generation scientific discoveries as data generated from scientific facilities grow dramatically. In previous work, we developed and deployed ScienceSearch, a search infrastructure for scientific data which uses machine learning to automate metadata creation. Our current deployment is deployed atop a container based platform at a HPC center. In this article, we present an evaluation and discuss our experiences with the ScienceSearch infrastructure. Specifically, we present a performance evaluation of ScienceSearch's infrastructure focusing on scalability trends. The obtained resultsshow that ScienceSearch is able to serve up to 130 queries/min with latency under 3 s.We discuss our infrastructure setup and evaluation results to provide our experiences and a perspective on opportunities and challenges of our search infrastructure.

show abstract

“…We finally compared our local samples to the entire dataset using Visibiome (Azman et al (2017)), a UniFrac based search engine for microbial communities. Remarkably, no matches for sabkha were found during an exhaustive search using the popular phylogeny-based distance measure, despite the database containing 35 samples from hypersaline environments, 36 of which have at least 50 OTUs (see Table 1).…”

Section: Figurementioning

confidence: 99%

Untitled

Supplemental Information 3: Sabkha Raw Sequences From Environmental Sequencing (V4 Region From 16S rRNA) Forward Reads

1

0

View full text Add to dashboard Cite

Advances in genomic sequencing technologies resulted in massive microbial diversity data (16S ribosomal gene sequences, rDNA) being generated in every possible environment.However, the majority of microorganisms have never been cultured, and therefore, nor cataloged. This poses a problem for molecular microbial ecologists because a large portion of the marker sequences can not be taxonomically resolved past the phylum taxon level. This tells very little about who or what these microorganisms are doing in relation to their environment. Our study describes an approach to assist in drawing ecological information from a sample when the taxon resolution is poor. We generated 16S rDNA libraries from a hypersaline marine sediment (coastal Sabkha) and saline mangrove soil in Abu Dhabi and then compared the compositional features to a database of 20,470 publicly available microbial community profiles (comprising the entire Earth Microbiome Project, EMP) that were annotated with terms from the Environmental Ontology (EnvO). An accurate taxonomic classification was not possible for 80% of the Sabkha operational taxonomic units (OTUs) beyond phylum level with widely used taxonomy classification tools, but habitat profiling performed on the community revealed strong links to bacterial assemblages of soil and marine origins. To capture the notion of generalist vs. specialist formally, we developed an algorithm to derive empirical probability distributions of OTUs over ecosystems from observed occurrences in the sample database, which then give rise to OTU-specific ecosystem entropies. We observed very low average ecosystem entropy of the Sabkha in contrast to other environmental samples. Based on this concept, the Sabkha community, while of midrange alpha diversity, presented largely specialist characteristics, with most OTUs identified to be unique to the Sabkha habitat. This finding is further corroborated by the observation that the Sabkha sample is unique with respect to the EMP-derived dataset (which contains 74 hypersaline and thousands of marine samples), as

show abstract

“…We calculated the ecosystem entropy to be in the midrange, rather than in the lower or higher extremes. 278 We finally compared our local samples to the entire dataset using Visibiome (Azman et al (2017) Table 3. Ecosystem entropy and additional information for selected saline samples.…”

mentioning

confidence: 99%

Untitled

Supplemental Information 1: Supplementary Material

0

View full text Add to dashboard Cite

Advances in genomic sequencing technologies resulted in massive microbial diversity data (16S ribosomal gene sequences, rDNA) being generated in every possible environment.However, the majority of microorganisms have never been cultured, and therefore, nor cataloged. This poses a problem for molecular microbial ecologists because a large portion of the marker sequences can not be taxonomically resolved past the phylum taxon level.This tells very little about who or what these microorganisms are doing in relation to their environment. Our study describes an approach to assist in drawing ecological information from a sample when the taxon resolution is poor. We generated 16S rDNA libraries from a hypersaline marine sediment (coastal Sabkha) and saline mangrove soil in Abu Dhabi and then compared the compositional features to a database of 20,470 publicly available microbial community profiles (comprising the entire Earth Microbiome Project, EMP) that were annotated with terms from the Environmental Ontology (EnvO). An accurate taxonomic classification was not possible for 80% of the Sabkha operational taxonomic units (OTUs) beyond phylum level with widely used taxonomy classification tools, but habitat profiling performed on the community revealed strong links to bacterial assemblages of soil and marine origins. To capture the notion of generalist vs. specialist formally, we developed an algorithm to derive empirical probability distributions of OTUs over ecosystems from observed occurrences in the sample database, which then give rise to OTU-specific ecosystem entropies. We observed very low average ecosystem entropy of the Sabkha in contrast to other environmental samples. Based on this concept, the Sabkha community, while of midrange alpha diversity, presented largely specialist characteristics, with most OTUs identified to be unique to the Sabkha habitat. This finding is further corroborated by the observation that the Sabkha sample is unique with respect to the EMP-derived dataset (which contains 74 hypersaline and thousands of marine samples), as PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.3286v1 | CC BY 4.0 Open Access | recABSTRACT 15 Advances in genomic sequencing technologies resulted in massive microbial diversity data (16S ribosomal gene sequences, rDNA) being generated for samples from wide-ranging environments. However, the majority of microorganisms have never been cultured, and therefore, are not reflected in current public databases. This poses a problem for molecular microbial ecologists because a large portion of the marker sequences can not be taxonomically resolved past the phylum taxon level. This tells very little about who or what these microorganisms are doing in relation to their environment. Our study describes an approach to assist in drawing ecological information from a sample even when the taxon resolution is poor. We generated 16S rDNA libraries from a hypersaline marine sediment (coastal sabkha) and a moderately hypersaline mangrove soil in Abu Dhabi. Intuitively, our ...

show abstract

Visibiome: an efficient microbiome search engine based on a scalable, distributed architecture

Cited by 3 publications

References 35 publications

Evaluation of a scientific data search infrastructure

Evaluation of a scientific data search infrastructure

Untitled

Untitled

Contact Info

Product

Resources

About