Coordinating Computation and I/O in Massively Parallel Sequence Search

Abstract

Abstract-With the explosive growth of genomic information, the searching of sequence databases has emerged as one of the most computation-and data-intensive scientific applications. Our previous studies suggested that parallel genomic sequence-search possesses highly irregular computation and I/O patterns. Effectively addressing these run-time irregularities is thus the key to designing scalable sequence-search tools on massively parallel computers. While the computation scheduling for irregular scientific applications and the optimization of noncontiguous file accesses have been well studied independently, little attention has been paid to the interplay between the two. In this paper, we systematically investigate the computation and I/O scheduling for data-intensive, irregular scientific applications within the context of genomic sequence search. Our study reveals that the lack of coordination between computation scheduling and I/O optimization could result in severe performance issues. We then propose an integrated scheduling approach that effectively improves sequence-search throughput by gracefully coordinating the dynamic load-balancing of computation and highperformance noncontiguous I/O.

cuBLASTP: Fine-Grained Parallelization of Protein Sequence Search on CPU+GPU

“…NCBI BLAST+ [8] uses pthreads to speedup BLAST on a multicore CPU. On CPU clusters, TurboBLAST [24], ScalaBLAST [10], and mpiBLAST [14] have been proposed. Among them, mpiBLAST is a widely-used one based on NCBI BLAST.…”

SCBI_MapReduce, a New Ruby Task-Farm Skeleton for Automated Parallelisation and Distribution in Chunks of Sequences: The Implementation of a Boosted Blast+

“…Improvements in its execution speed will result in significant impact in the practice of genome studies. Therefore, important efforts have been invested in accelerating it for different computers systems (to cite a few, mpiBLAST [6,12], CloudBLAST [28], AzureBlast [29], GPU-Blast [30], and scalaBLAST 2.0 [31]). These Blast parallelisations require computer expertise to produce and adapt a particular Blast code and are tightly bonded to the software version included in the parallelised/distributed code [31].…”

“…Paired sequence comparison is inherently a parallel process in which many sequence pairs can be analysed at the same time by means of functions or algorithms that are iteratively performed over sequences. This is impelling the parallelisation of sequence comparison algorithms [5][6][7][8][9] as well as other bioinformatic algorithms [10,11].…”

Accelerating Protein Sequence Search in a Heterogeneous Computing System

“…On multi-core platforms, the BLAST implementation from National Center for Biotechnology Information (NCBI) has been parallelized with pthreads. On cluster platforms, there are parallel implementations such as TurboBLAST [4], ScalaBLAST [24], and mpiBLAST [6], [13], [14]. Among them, mpiBLAST is a widely used parallelization of NCBI BLAST.…”