Raptor: A fast and space-efficient pre-filter for querying very large collections of nucleotide sequences

Abstract

HighlightsRaptor is a tool to search through large collections of genomic texts Raptor is 12-144 times faster and uses up to 30 times less RAM than COBS or MantisThe Raptor index is 6-50 times faster to build The use of minimizers and Bloom filters makes Raptor very spaceefficient

Co-Design for Energy Efficient and Fast Genomic Search

“…The runtimes were initially significantly improved by the Patro group with the tool Mantis in [18] and by the Iqbal group with COBS [3]. This year, the Reinert lab introduced the IBF [23], which has proven to be a significant step towards a very time and space efficient in-memory data structure for preprocessing approximate sequence queries, which opens up many possible applications. It improves in runtime by a factor of 12-144 over its competitors COBS and Mantis.…”

“…as a bit mask. Like Seiler et al [23], it uses minimizers to reduce the number of 𝑘-mers to be queried and thus the number of costly memory accesses. We decided on the Intel FPGA SDK for OpenCL version 2021.3 as the implementation environment, as it offers a high-level programming model with an acceptable overhead and encapsulates the entire host interaction in a well-known API, which allowed us to focus on the algorithmic optimizations of the problem.…”

“…One of the main tasks of such pipelines is to (approximately) search large reference data sets for sequencing reads or short sequence patterns like genes. Hence, researchers had to develop novel indexing data structures such as the Interleaved Bloom Filter (IBF) [7] and, based on this, an extension with winnowing minimizers and probabilistic thresholding called Raptor [23] which is currently the state-ofthe-art for distributing approximate queries with an in-memory data structure. The CPU-based IBF implementation can distribute 10 million NGS queries for combined texts of hundreds of Gigabytes in only a few seconds.…”

“…The following section contains a brief introduction to the IBF data structure, details can be found in [23].…”

“…For the approximate search of a query 𝑃, the binning bitvectors of all representative 𝑘-mers in the query are combined into a counting vector and the membership of a query in a bin is determined by applying an appropriate threshold (see [23]). This approach is depicted in Figure 2.…”