Orion: Scaling Genomic Sequence Matching with Fine-Grained Parallelization

Mahadik, Kanak; Chaterji, Somali; Zhou, Bowen; Kulkarni, Milind; Bagchi, Saurabh

doi:10.1109/sc.2014.42

Cited by 14 publications

(16 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Massive volumes of sequences maintained in the database to be searched induces additional computation burden. BLAST is a widely adopted bioinformatics tool for sequence alignment which perform faster alignments, at expense of accuracy (possibly missing some potential hits) [18,19]. Experiment are conducted to evaluate MOHMR and HMR performance for performing gene sequence alignment.…”

Section: Bioinformatics Application Performance Evaluationmentioning

confidence: 99%

“…The dataset for experiment analysis is obtained from NCBI [20]. For performing alignment Drosophila database as a reference database and Query sequence of varied sizes of from Homo sapiens chromosomal sequences and genomic scaffolds is considered similar to [19] which are tabulated in Table 1. All four experiment are conducted using BLAST algorithm on HMR and MOHMR frameworks.…”

Section: Bioinformatics Application Performance Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Optimized memory model for hadoop map reduce framework

Bhaskar¹,

Ranjan

2019

IJECE

View full text Add to dashboard Cite

Map Reduce is the preferred computing framework used in large data analysis and processing applications. Hadoop is a widely used Map Reduce framework across different community due to its open source nature. Cloud service provider such as Microsoft azure HDInsight offers resources to its customer and only pays for their use. However, the critical challenges of cloud service provider is to meet user task Service level agreement (SLA) requirement (task deadline). Currently, the onus is on client to compute the amount of resource required to run a job on cloud. This work present a novel memory optimization model for Hadoop Map Reduce framework namely MOHMR (Optimized Hadoop Map Reduce) to process data in real-time and utilize system resource efficiently. The MOHMR present accurate model to compute job memory optimization and also present a model to provision the amount of cloud resource required to meet task deadline. The MOHMR first build a profile for each job and computes memory optimization time of job using greedy approach. Experiment are conducted on Microsoft Azure HDInsight cloud platform considering different application such as text computing and bioinformatics application to evaluate performance of MOHMR of over existing model shows significant performance improvement in terms of computation time. Experiment are conducted on Microsoft Azure HDInsight cloud. Overall, good correlation is reported between practical memory optimization values and theoretical memory optimization values.

show abstract

Section: Bioinformatics Application Performance Evaluationmentioning

confidence: 99%

Section: Bioinformatics Application Performance Evaluationmentioning

confidence: 99%

Optimized memory model for hadoop map reduce framework

Bhaskar¹,

Ranjan

2019

IJECE

View full text Add to dashboard Cite

show abstract

“…The resulting alignments can then be combined using a custom aggregation function that eliminates duplicates and splices together overlapping alignments. We use the strategy proposed by Mahadik et al [26] to implement the partition and aggregate functions. Figure 11 provides the pseudo code for partition and aggregate functions.…”

Section: Partition-aggregate Functionsmentioning

confidence: 99%

“…As the amount of genomic data rapidly increases, many applications in these categories are facing severe challenges in scaling up to handle these large datasets. For example, the parallel version of the de facto local sequence alignment application called BLAST suffers from an exponential increase in matching time with the increasing size of the query sequence-the knee of the curve is reached at a query size of only 2 Mega base pairs (Mbp) ( Figure 3 in [26]). For calibration, the smallest human chromosome (chromosome 22) is 49 Mbp long.…”

Section: Introductionmentioning

confidence: 99%

Sarvavid

Mahadik

Wright

Zhang

et al. 2016

Proceedings of the 2016 International Conference on Supercomputing

Self Cite

View full text Add to dashboard Cite

Breakthroughs in gene sequencing technologies have led to an exponential increase in the amount of genomic data. Efficient tools to rapidly process such large quantities of data are critical in the study of gene functions, diseases, evolution, and population variation. These tools are designed in an ad-hoc manner, and require extensive programmer effort to develop and optimize them. Often, such tools are written with the currently available data sizes in mind, and soon start to under perform due to the exponential growth in data. Furthermore, to obtain high-performance, these tools require parallel implementations, adding to the development complexity. This paper makes an observation that most such tools contain a recurring set of software modules, or kernels. The availability of efficient implementations of such kernels can improve programmer productivity, and provide effective scalability with growing data. To achieve this goal, the paper presents a domain-specific language, called Sarvavid, which provides these kernels as language constructs. Sarvavid comes with a compiler that performs domain-specific optimizations, which are beyond the scope of libraries and generic compilers. Furthermore, Sarvavid inherently supports exploitation of parallelism across multiple nodes. To demonstrate the efficacy of Sarvavid, we implement five well-known genomics applications-BLAST, MUMmer, E-MEM, SPAdes, and SGA-using Sarvavid. Our versions of BLAST, MUMmer, and E-MEM show a speedup of 2.4X, 2.5X, and 2.1X respectively compared to hand-optimized implementations when run on a single node, while SPAdes and SGA show the same performance as handwritten code. Moreover, Sarvavid applications scale to 1024 cores using a Hadoop backend.

show abstract

“…However, in this approach the scaling of the index table construction is limited by the size of each subgroup. Orion [35] is an improvement over mpiBLAST [36] and scales the sequence matching with fine-grained parallelization. However, Orion uses mpiBLAST's mpiformatdb tool to format and to shard the database and this process is serial.…”

Section: Related Workmentioning

confidence: 99%

merAligner: A Fully Parallel Sequence Aligner

Georganas¹,

Buluç²,

Chapman

et al. 2015

2015 IEEE International Parallel and Distributed Processing Symposium

View full text Add to dashboard Cite

Abstract-Aligning a set of query sequences to a set of target sequences is an important task in bioinformatics. In this work we present merAligner, a highly parallel sequence aligner that implements a seed-and-extend algorithm and employs parallelism in all of its components. MerAligner relies on a high performance distributed hash table (seed index) and uses onesided communication capabilities of the Unified Parallel C to facilitate a fine-grained parallelism. We leverage communication optimizations at the construction of the distributed hash table and software caching schemes to reduce communication during the aligning phase. Additionally, merAligner preprocesses the target sequences to extract properties enabling exact sequence matching with minimal communication. Finally, we efficiently parallelize the I/O intensive phases and implement an effective load balancing scheme. Results show that merAligner exhibits efficient scaling up to thousands of cores on a Cray XC30 supercomputer using real human and wheat genome data while significantly outperforming existing parallel alignment tools.

show abstract

Orion: Scaling Genomic Sequence Matching with Fine-Grained Parallelization

Cited by 14 publications

References 21 publications

Optimized memory model for hadoop map reduce framework

Optimized memory model for hadoop map reduce framework

Sarvavid

merAligner: A Fully Parallel Sequence Aligner

Contact Info

Product

Resources

About