Scalable Genomic Assembly through Parallel
            <i>de Bruijn</i>
            Graph Construction for Multiple K-mers

Mahadik, Kanak; Wright, Christopher; Kulkarni, Milind; Bagchi, Saurabh; Chaterji, Somali

doi:10.1145/3107411.3107482

Cited by 8 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, a reference genome is typically needed to serve as ground truth for such evaluations, making this approach infeasible for de novo sequencing tasks. To the best of our knowledge, existing tools leave the best parameter choice to the end user [9,1], although Kmergenie provides intuitive abundance histograms or heuristics to guide k-value selection, albeit for genome assembly. Further, Kmergenie does not provide the optimal k-value for different tools, only accounting for the dataset for computing the abundance histograms.…”

Section: During Genomic Error Correctionmentioning

confidence: 99%

“…The search space can be large and since the cost of searching shows up as a runtime delay, it is important to reduce the time that it takes to evaluate that metric of each search point. In today's state-of-the-art, to find the best value of a configuration parameter [26,1], e.g., k-value, the method would be to pick a k (a single point in the space), run the EC tool with that value, then perform alignment (with one of several available tools such as Bowtie2), and finally compute the metric (alignment rate or EC gain) for that value. In contrast, with Athena, to explore one point in the search space, we run the EC algorithm with the k-value, and then compute the perplexity metric, which does not involve the time consuming step of alignment.…”

Section: Impact On Assembly Qualitymentioning

confidence: 99%

“…We test the proposed approach against two levels of synthetic errors: low error rate (Uniform in (1,5) bp in error per read), and high error rate (Uniform in (6,10) bp in error per read). In all cases, the language model is trained using the original data set, mentioned in Table 1, with the natural ambient rate of error.…”

Section: Evaluation With Synthetically Injected Errorsmentioning

confidence: 99%

“…The genome analysis workflow needs to be carefully orchestrated so errors in reads are not magnified downstream. Consequently, multiple error-correction (EC) techniques have been developed for improved performance for applications ranging from de novo variant calling to differential expression, iterative k-mer selection for improved genome assembly [1] and for long-read error correction [2,3].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Athena: Automated Tuning of k-mer based Genomic Error Correction Algorithms using Language Models

Abdallah

Mahgoub

Ahmed

et al. 2019

Sci Rep

Self Cite

View full text Add to dashboard Cite

The performance of most error-correction (EC) algorithms that operate on genomics reads is dependent on the proper choice of its configuration parameters, such as the value of k in k-mer based techniques. In this work, we target the problem of finding the best values of these configuration parameters to optimize error correction and consequently improve genome assembly. We perform this in an adaptive manner, adapted to different datasets and to EC tools, due to the observation that different configuration parameters are optimal for different datasets, i.e., from different platforms and species, and vary with the EC algorithm being applied. We use language modeling techniques from the Natural Language Processing (NLP) domain in our algorithmic suite, Athena, to automatically tune the performance-sensitive configuration parameters. Through the use of N-Gram and Recurrent Neural Network (RNN) language modeling, we validate the intuition that the EC performance can be computed quantitatively and efficiently using the “perplexity” metric, repurposed from NLP. After training the language model, we show that the perplexity metric calculated from a sample of the test (or production) data has a strong negative correlation with the quality of error correction of erroneous NGS reads. Therefore, we use the perplexity metric to guide a hill climbing-based search, converging toward the best configuration parameter value. Our approach is suitable for both de novo and comparative sequencing (resequencing), eliminating the need for a reference genome to serve as the ground truth. We find that Athena can automatically find the optimal value of k with a very high accuracy for 7 real datasets and using 3 different k-mer based EC algorithms, Lighter, Blue, and Racer. The inverse relation between the perplexity metric and alignment rate exists under all our tested conditions—for real and synthetic datasets, for all kinds of sequencing errors (insertion, deletion, and substitution), and for high and low error rates. The absolute value of that correlation is at least 73%. In our experiments, the best value of k found by Athena achieves an alignment rate within 0.53% of the oracle best value of k found through brute force searching (i.e., scanning through the entire range of k values). Athena’s selected value of k lies within the top-3 best k values using N-Gram models and the top-5 best k values using RNN models With best parameter selection by Athena, the assembly quality (NG50) is improved by a Geometric Mean of 4.72X across the 7 real datasets.

show abstract

Section: During Genomic Error Correctionmentioning

confidence: 99%

Section: Impact On Assembly Qualitymentioning

confidence: 99%

Section: Evaluation With Synthetically Injected Errorsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Athena: Automated Tuning of k-mer based Genomic Error Correction Algorithms using Language Models

Abdallah

Mahgoub

Ahmed

et al. 2019

Sci Rep

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, there are limitations to this vertical scaling up to the point where the application itself becomes the bottleneck. With the "genomical" data sizes, often noSQL backends are deployed to scale with the increasing sizes and diversity of queries, such as in genomics [101] and IoT-related domains [102]. Further, we have to keep in mind guarantees of safety, availability, and timeliness, either deterministically or stochastically in a complex environment, with humans-inthe-loop.…”

Section: Planet-scale Iotmentioning

confidence: 99%

Resilient Cyberphysical Systems and their Application Drivers: A Technology Roadmap

Chaterji,

Naghizadeh,

Alam

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Cyberphysical systems (CPS) are ubiquitous in our personal and professional lives, and they promise to dramatically improve micro-communities (e.g., urban farms, hospitals), macro-communities (e.g., cities and metropolises), urban structures (e.g., smart homes and cars), and living structures (e.g., human bodies, synthetic genomes). The question that we address in this article pertains to designing these CPS systems to be resilient-from-the-ground-up, and in "learning" iterations, resilientby-reaction. An optimally designed system is resilient to both unique and recurrent attacks with a minimal overhead from "fitting". In the following, our focus is on design and deployment innovations that are broadly applicable across a range of application areas and drivers.We divide our paper into three broad themes. First, we present three prominent application drivers that can lay the basis for the existing and emerging technologies, as follows: smart cities and digital agriculture; planet-scale IoT ; and internet-of-medical-things (IoMT). We select the three application drivers as representative examples of different possible application scenarios that a CPS "designer" may face. These scenarios are orthogonal to each other, operate at different scales, and collectively cover key application domains. Concretely, while the scale of smart cities and digital agriculture on small to large-sized farms can be large, they are not nearly as large as planet-scale IoT, where the scale may pose unique challenges to start with, requiring the consideration of the scale from the outset. For example, a protocol that relies on lots of messages being passed among the participating nodes may work well from an energy efficiency and timeliness standpoint when all the nodes are nearby, say connected through Bluetooth Low Energy (BLE) links. However, when the system is planet-scale and the nodes are dispersed, potentially over large geographic regions, or in highly congested environments, such as urban road environments, such a protocol will become infeasible. In contrast, for IoMT, there is a unique set of challenges with some critical safety requirements as they relate to individuals in medical scenarios.Second, we lay out the foundational technologies-hardware and algorithms-which will power resilient CPS. We base this discussion on two complementary threads for imbuing resilience in these systems, namely, resilience-by-design and resilience-by-reaction. Overall, the notion of resilience can be thought of in the light of three main sources of lack of resilience, as follows: exogenous factors, such as natural variations and attack scenarios; mismatch between engineered designs and exogenous factors ranging from DDoS (distributed denial-of-service) attacks or other cybersecurity

show abstract

MULKSG: MULtiple K Simultaneous Graph Assembly

Wright

Krishnamoorty

Kulkarni

2019

Algorithms for Computational Biology

View full text Add to dashboard Cite

Scalable Genomic Assembly through Parallel de Bruijn Graph Construction for Multiple K-mers

Cited by 8 publications

References 11 publications

Athena: Automated Tuning of k-mer based Genomic Error Correction Algorithms using Language Models

Athena: Automated Tuning of k-mer based Genomic Error Correction Algorithms using Language Models

Resilient Cyberphysical Systems and their Application Drivers: A Technology Roadmap

MULKSG: MULtiple K Simultaneous Graph Assembly

Contact Info

Product

Resources

About