Patterns

McCool, Michael; Robison, Arch D.; Reinders, James

doi:10.1016/b978-0-12-415993-8.00003-7

Cited by 82 publications

(34 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…According to [28], if there are a series of operations, and each operation can be mapped into a specific hardware structure, a pipeline parallel pattern is a good solution.…”

Section: Methodsmentioning

confidence: 99%

PipeMEM: A Framework to Speed Up BWA-MEM in Spark with Low Overhead

Zhang

Liu

Dong

2019

Genes

View full text Add to dashboard Cite

(1) Background: DNA sequence alignment process is an essential step in genome analysis. BWA-MEM has been a prevalent single-node tool in genome alignment because of its high speed and accuracy. The exponentially generated genome data requiring a multi-node solution to handle large volumes of data currently remains a challenge. Spark is a ubiquitous big data platform that has been exploited to assist genome alignment in handling this challenge. Nonetheless, existing works that utilize Spark to optimize BWA-MEM suffer from higher overhead. (2) Methods: In this paper, we presented PipeMEM, a framework to accelerate BWA-MEM with lower overhead with the help of the pipe operation in Spark. We additionally proposed to use a pipeline structure and in-memory-computation to accelerate PipeMEM. (3) Results: Our experiments showed that, on paired-end alignment tasks, our framework had low overhead. In a multi-node environment, our framework, on average, was 2.27× faster compared with BWASpark (an alignment tool in Genome Analysis Toolkit (GATK)), and 2.33× faster compared with SparkBWA. (4) Conclusions: PipeMEM could accelerate BWA-MEM in the Spark environment with high performance and low overhead.

show abstract

“…According to [28], if there are a series of operations, and each operation can be mapped into a specific hardware structure, a pipeline parallel pattern is a good solution.…”

Section: Methodsmentioning

confidence: 99%

PipeMEM: A Framework to Speed Up BWA-MEM in Spark with Low Overhead

Zhang

Liu

Dong

2019

Genes

View full text Add to dashboard Cite

show abstract

“…It means that in the first step we compute singleton parents sets using all available threads, in the second step we compute two-element parents sets in parallel and so on, until we reach parents sets size limit or scoring function limit. However, the set-wise algorithm requires more synchronizations between the threads ( McCool, Reinders & Robison, 2012 ) in comparison with the variable-wise . On top of that allocating large number of cores to the variable whose parents set is very quick to compute may result in lower performance due to the context switching.…”

Section: Methodsmentioning

confidence: 99%

Distributed Bayesian networks reconstruction on the whole genome scale

Frolova

Wilczyński

2018

PeerJ

View full text Add to dashboard Cite

BackgroundBayesian networks are directed acyclic graphical models widely used to represent the probabilistic relationships between random variables. They have been applied in various biological contexts, including gene regulatory networks and protein–protein interactions inference. Generally, learning Bayesian networks from experimental data is NP-hard, leading to widespread use of heuristic search methods giving suboptimal results. However, in cases when the acyclicity of the graph can be externally ensured, it is possible to find the optimal network in polynomial time. While our previously developed tool BNFinder implements polynomial time algorithm, reconstructing networks with the large amount of experimental data still leads to computations on single CPU growing exceedingly.ResultsIn the present paper we propose parallelized algorithm designed for multi-core and distributed systems and its implementation in the improved version of BNFinder—tool for learning optimal Bayesian networks. The new algorithm has been tested on different simulated and experimental datasets showing that it has much better efficiency of parallelization than the previous version. BNFinder gives comparable results in terms of accuracy with respect to current state-of-the-art inference methods, giving significant advantage in cases when external information such as regulators list or prior edge probability can be introduced, particularly for datasets with static gene expression observations.ConclusionsWe show that the new method can be used to reconstruct networks in the size range of thousands of genes making it practically applicable to whole genome datasets of prokaryotic systems and large components of eukaryotic genomes. Our benchmarking results on realistic datasets indicate that the tool should be useful to a wide audience of researchers interested in discovering dependencies in their large-scale transcriptomic datasets.

show abstract

“…The computational performance analysis of Algorithm 2 is evaluated using the metrics speedup (or acceleration) and efficiency. The speedup is the ratio between the execution times of parallel implementations with one core and parallel implementations with two or more cores [ 18 ]. The speedup is represented by the formula S = T 1 / T p .…”

Section: Implementation and Experimental Investigationmentioning

confidence: 99%

“…The speedup is represented by the formula S = T 1 / T p . The efficiency estimates how well utilized the processors are in solving the problem compared to how much effort is wasted in communication and synchronization [ 18 ]. The efficiency is determined by the ratio between the speedup and the number of processing elements, represented by the formula E = T 1 /( pT p ).…”

Section: Implementation and Experimental Investigationmentioning

confidence: 99%

A Parallel Framework with Block Matrices of a Discrete Fourier Transform for Vector‐Valued Discrete‐Time Signals

Soto-Quiros

2015

The Scientific World Journal

View full text Add to dashboard Cite

This paper presents a parallel implementation of a kind of discrete Fourier transform (DFT): the vector-valued DFT. The vector-valued DFT is a novel tool to analyze the spectra of vector-valued discrete-time signals. This parallel implementation is developed in terms of a mathematical framework with a set of block matrix operations. These block matrix operations contribute to analysis, design, and implementation of parallel algorithms in multicore processors. In this work, an implementation and experimental investigation of the mathematical framework are performed using MATLAB with the Parallel Computing Toolbox. We found that there is advantage to use multicore processors and a parallel computing environment to minimize the high execution time. Additionally, speedup increases when the number of logical processors and length of the signal increase.

show abstract

Patterns

Cited by 82 publications

References 0 publications

PipeMEM: A Framework to Speed Up BWA-MEM in Spark with Low Overhead

PipeMEM: A Framework to Speed Up BWA-MEM in Spark with Low Overhead

Distributed Bayesian networks reconstruction on the whole genome scale

A Parallel Framework with Block Matrices of a Discrete Fourier Transform for Vector‐Valued Discrete‐Time Signals

Contact Info

Product

Resources

About