Scalable Genome Assembly through Parallel de Bruijn Graph Construction for Multiple k-mers

Mahadik, Kanak; Wright, Christopher; Kulkarni, Milind; Bagchi, Saurabh; Chaterji, Somali

doi:10.1038/s41598-019-51284-9

Cited by 13 publications

(14 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The new version of GenomeScope 2 is also applicable to polyploid genomes. Finally, some assemblers, such as IDBA [41], IDBA-UD [42] and subsequently SPAdes, SOAPdenovo2, or the recently developed ScalaDBG [43], have implemented innovative ways to deal with the choice of the best k-mer by using a multi-k-mer approach.…”

Section: Wgs Metagenomicsmentioning

confidence: 99%

Metagenomic approaches in microbial ecology: an update on whole-genome and marker gene sequencing analyses

2020

View full text Add to dashboard Cite

Metagenomics and marker gene approaches, coupled with high-throughput sequencing technologies, have revolutionized the field of microbial ecology. Metagenomics is a culture-independent method that allows the identification and characterization of organisms from all kinds of samples. Whole-genome shotgun sequencing analyses the total DNA of a chosen sample to determine the presence of micro-organisms from all domains of life and their genomic content. Importantly, the whole-genome shotgun sequencing approach reveals the genomic diversity present, but can also give insights into the functional potential of the micro-organisms identified. The marker gene approach is based on the sequencing of a specific gene region. It allows one to describe the microbial composition based on the taxonomic groups present in the sample. It is frequently used to analyse the biodiversity of microbial ecosystems. Despite its importance, the analysis of metagenomic sequencing and marker gene data is quite a challenge. Here we review the primary workflows and software used for both approaches and discuss the current challenges in the field.

show abstract

Section: Wgs Metagenomicsmentioning

confidence: 99%

Metagenomic approaches in microbial ecology: an update on whole-genome and marker gene sequencing analyses

2020

View full text Add to dashboard Cite

show abstract

“…It employs DSK [47] to count k-mers and (k +1)-mers which only requires a fixed user-defined amount of memory. Assembler ScalaDBG [48] is a scalable genome assembler through parallel de-Bruijn graph construction for multiple k-mers. This assembler first performs graph construction in parallel for each k-value, then for each pair of graphs, the higher k-valued graph is patched using the lower k-valued graph to generate a single graph.…”

Section: ) De-bruijn Graph-based Assemblersmentioning

confidence: 99%

Graph Theoretical Strategies in De Novo Assembly

2022

View full text Add to dashboard Cite

De novo genome assemblers assume the reference genome is unavailable, incomplete, highly fragmented, or significantly altered as in cancer tissues. Algorithms for de novo assembly have been developed to deal with and assemble a large number of short sequence reads from genome sequencing. In this review paper, we have provided an overview of the graph-theoretical side of de novo genome assembly algorithms. We have investigated the construction of fourteen graph data structures related to OLC-based and DBG-based algorithms in order to compare and discuss their application in different assemblers. In addition, the most significant and recent genome de novo assemblers are classified according to the extensive variety of original, generalized, and specialized versions of graph data structures.INDEX TERMS Combinatorial data structures, de-Bruijn graph, de novo assembly algorithms, high throughput sequencing, overlap graph.

show abstract

“…The k-mers that appear above a certain threshold frequency, and are therefore expected to be legitimate, are solid k-mers, the others are called insolid or untrusted k-mers. It has been found that performance of EC tools is extremely sensitive to the chosen k-value [17,4], and the optimal value is both tooland dataset-dependent. Small values of k result in an increase in the probability of overlap between reads at the cost of not allowing the algorithm to distinguish between erroneous and correct k-mers.…”

Section: K-mer Based Ec Toolsmentioning

confidence: 99%

“…The drop in high-throughput sequencing costs has offered unprecedented opportunities to characterize genomes, metagenomes, and single-cell genomes across the tree-of-life. Third-generation sequencing technologies [1,2] have demonstrated the potential to produce unparalleled genome assemblies due to their capability to generate staggeringly longer reads, at a much faster pace, and remarkably low costs [3,4], albeit, at low signal-to-noise ratios. This fundamental challenge makes genomic sequence identification and assembly, challenging, often necessitating hybrid correction approaches over self-correction from a consensus of long reads [5].…”

Section: Introductionmentioning

confidence: 99%

Lerna: Transformer Architectures for Configuring Error Correction Tools for Short- and Long-Read Genome Sequencing

Sharma¹,

Jain²,

Mahgoub³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Background: Sequencing technologies are prone to errors, making error correction (EC) necessary for downstream applications. EC tools need to be manually configured for optimal performance. We find that the optimal parameters (e.g., k-mer size) are both tool-and dataset-dependent. Moreover, evaluating the performance (i.e., Alignment-rate or Gain) of a given tool usually relies on a reference genome, but quality reference genomes are not always available. We introduce Lerna for the automated configuration of k-mer-based EC tools. Lerna first creates a language model (LM) of the uncorrected genomic reads, and then, based on this LM, calculates a metric called the perplexity metric to evaluate the corrected reads for different parameter choices. Next, it finds the one that produces the highest alignment rate without using a reference genome. The fundamental intuition of our approach is that the perplexity metric is inversely correlated with the quality of the assembly after error correction. Therefore, Lerna leverages the perplexity metric for automated tuning of k-mer sizes without needing a reference genome.Results: First, we show that the best k-mer value can vary for different datasets, even for the same EC tool. This motivates our design that automates k-mer size selection without using a reference genome. Second, we show the gains of our LM using its component attention-based transformers. We show the model's estimation of the perplexity metric before and after error correction. The lower the perplexity after correction, the better the k-mer size. We also show that the alignment rate and assembly quality computed for the corrected reads are strongly negatively correlated with the perplexity, enabling the automated selection of k-mer values for better error correction, and hence, improved assembly quality. We validate our approach on both short and long reads. Additionally, we show that our attention-based models have significant runtime improvement for the entire pipeline -18× faster than previous works, due to parallelizing the attention mechanism and the use of JIT compilation for GPU inferencing. Conclusion:Lerna improves de novo genome assembly by optimizing EC tools. Our code is made available in a public repository at: https://github.com/icanforce/lerna-genomics.

show abstract

Scalable Genome Assembly through Parallel de Bruijn Graph Construction for Multiple k-mers

Cited by 13 publications

References 21 publications

Metagenomic approaches in microbial ecology: an update on whole-genome and marker gene sequencing analyses

Metagenomic approaches in microbial ecology: an update on whole-genome and marker gene sequencing analyses

Graph Theoretical Strategies in De Novo Assembly

Lerna: Transformer Architectures for Configuring Error Correction Tools for Short- and Long-Read Genome Sequencing

Contact Info

Product

Resources

About