MetaProFi: an ultrafast chunked Bloom filter for storing and querying protein and nucleotide sequence data for accurate identification of functionally relevant genetic variants

Srikakulam, Sanjay Kumar; Keller, Sebastian; Dabbaghie, Fawaz; Bals, Robert; Kalinina, Olga

doi:10.1093/bioinformatics/btad101

Cited by 7 publications

(9 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We evaluated the performances of kmindex together with eight state-of-the-art k-mer indexers: themisto [2]; ggcat [7]; HIBF [18]; PAC [17]; MetaProFi [22]; MetaGraph [13]; Bifrost [12]; and COBS [3]. The dataset for this benchmark is composed of metagenomic seawater sequencing data from 50 Tara Oceans samples, of 1.4TB of gzipped fastq files.…”

Section: Comparative Results Indexing 50 Metagenomic Seawater Samplesmentioning

confidence: 99%

“…We evaluated the performance of kmindex together with state-of-the-art k-mer indexers MetaGraph [11], MetaProFi [20], and PAC [16]. We first indexed raw metagenomic seawater sequencing data from 50 Tara Oceans samples, composed of 1.4TB of gzipped fastq files.…”

Section: Comparative Results Indexing 50 Metagenomic Seawater Samplesmentioning

confidence: 99%

“…MetaGraph [13], BiFrost [12], and ggcat [6]) suffer from low scalability, as highlighted by our results. We are thus left with methods based on Bloom filters (BFs) [4], such as COBS [3], SBT [21], later improved by HowDeSBT [11], and more recently by MetaProFi [22] able to index billions of k-mers using only a few dozen of gigabytes of space.…”

mentioning

confidence: 99%

See 2 more Smart Citations

kmindex and ORA: indexing and real-time user-friendly queries in terabyte-sized complex genomic datasets

Lemane,

Lezzoche,

Lecubin

et al. 2023

Preprint

View full text Add to dashboard Cite

Despite their wealth of biological information, public sequencing databases are largely underutilized. One cannot efficiently search for a sequence of interest in these immense resources. Sophisticated computational methods such as approximate membership query data structures allow searching for fixed-length words (k-mers) in large datasets. Yet they face scalability challenges when applied to thousands of complex sequencing experiments. In this context we propose kmindex, a new approach that uses inverted indexes based on Bloom filters. Thanks to its algorithmic choices and its fine-tuned implementation, kmindex offers the possibility to index thousands of highly complex metagenomes into an index that answers sequences queries in the tenth of a second. Index construction is one order of magnitude faster than previous approaches, and query time is two orders of magnitude faster. Based on Bloom filters, kmindex achieves negligible false positive rates, below 0.01% on average. Its average false positive rate is four orders of magnitude lower than existing approaches, for similar index sizes. It has been successfully used to index 1,393 complex marine seawater metagenome samples of raw sequences from the Tara Oceans project, demonstrating its effectiveness on large and complex datasets. This level of scaling was previously unattainable. Building on the kmindex results, we provide a public web server named "Ocean Read Atlas" (ORA) at https://ocean-read-atlas.mio.osupytheas.fr/ that can answer queries against the entire Tara Oceans dataset in real-time. kmindex is open-source software available at https://github.com/tlemane/kmindex.

show abstract

Section: Comparative Results Indexing 50 Metagenomic Seawater Samplesmentioning

confidence: 99%

Section: Comparative Results Indexing 50 Metagenomic Seawater Samplesmentioning

confidence: 99%

See 1 more Smart Citation

kmindex and ORA: indexing and real-time user-friendly queries in terabyte-sized complex genomic datasets

Lemane,

Lezzoche,

Lecubin

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Here, the graph construction is the main limitation of the methods. Other tools allow false-positive results by using Approximate Membership Queries (AMQ) data structures to enhance space efficiency [6, 4, 20, 14, 27, 16]. They all use trade-offs between size and false-positive rate.…”

Section: Introductionmentioning

confidence: 99%

The Backpack Quotient Filter: a dynamic and space-efficient data structure for queryingk-mers with abundance.

Levallois,

Andreace,

Le Gal

et al. 2024

Preprint

View full text Add to dashboard Cite

Genomic data sequencing has become indispensable for elucidating the complexities of biological systems. As databases storing genomic information, such as the European Nucleotide Archive, continue to grow exponentially, efficient solutions for data manipulation are imperative. One fundamental operation that remains challenging is querying these databases to determine the presence or absence of specific sequences and their abundance within datasets. This paper introduces a novel data structure indexingk-mers (substrings of lengthk), the Backpack Quotient Filter (BQF), which serves as an alternative to the Counting Quotient Filter (CQF). The BQF offers enhanced space efficiency compared to the CQF while retaining key properties, including abundance information and dynamicity, with a negligible false positive rate, below 10-5%. The approach involves a redefinition of how abundance information is handled within the structure, alongside with an independent strategy for space efficiency. We show that the BQF uses 4x less space than the CQF on some of the most complex data to index: sea-water metagenomics sequences. Furthermore, we show that space efficiency increases as the amount of data to be indexed increases, which is in line with the original objective of scaling to ever-larger datasets.

show abstract

“…Several approaches utilize a specialized data structure for information retrieval known as a Bloom filter. FACS [27] uses a Bloom filter to classify DNA sequences. MetaProFi [28] uses a Bloom filter to build indexes of amino acid sequences to provide a memory-efficient and storage-efficient solution for protein sequence comparison.…”

Section: Introductionmentioning

confidence: 99%

MetaBIDx: a new computational approach to bacteria identification in microbiomes

Pham,

Phan

2024

Microbiome Res Rep

View full text Add to dashboard Cite

Objectives: This study introduces MetaBIDx, a computational method designed to enhance species prediction in metagenomic environments. The method addresses the challenge of accurate species identification in complex microbiomes, which is due to the large number of generated reads and the ever-expanding number of bacterial genomes. Bacterial identification is essential for disease diagnosis and tracing outbreaks associated with microbial infections. Methods: MetaBIDx utilizes a modified Bloom filter for efficient indexing of reference genomes and incorporates a novel strategy for reducing false positives by clustering species based on their genomic coverages by identified reads. The approach was evaluated and compared with several well-established tools across various datasets. Precision, recall, and F1-score were used to quantify the accuracy of species prediction. Results: MetaBIDx demonstrated superior performance compared to other tools, especially in terms of precision and F1-score. The application of clustering based on approximate coverages significantly improved precision in species identification, effectively minimizing false positives. We further demonstrated that other methods can also benefit from our approach to removing false positives by clustering species based on approximate coverages. Conclusion: With a novel approach to reducing false positives and the effective use of a modified Bloom filter to index species, MetaBIDx represents an advancement in metagenomic analysis. The findings suggest that the proposed approach could also benefit other metagenomic tools, indicating its potential for broader application in the field. The study lays the groundwork for future improvements in computational efficiency and the expansion of microbial databases.

show abstract

MetaProFi: an ultrafast chunked Bloom filter for storing and querying protein and nucleotide sequence data for accurate identification of functionally relevant genetic variants

Cited by 7 publications

References 15 publications

kmindex and ORA: indexing and real-time user-friendly queries in terabyte-sized complex genomic datasets

kmindex and ORA: indexing and real-time user-friendly queries in terabyte-sized complex genomic datasets

The Backpack Quotient Filter: a dynamic and space-efficient data structure for queryingk-mers with abundance.

MetaBIDx: a new computational approach to bacteria identification in microbiomes

Contact Info

Product

Resources

About