CHIC: a short read aligner for pan-genomic references

Valenzuela, Daniel; Mäkinen, Veli

doi:10.1101/178129

Cited by 10 publications

(15 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This allows for an index to be constructed for large collections of chromosome 19s (500 or more); a task that is out of reach of Bowtie, causing it to exhaust memory even with a budget of 512 GB. Our method produces indexes in a smaller memory footprint than a Hybrid Index-based method (CHIC; Valenzuela and Mäkinen, 2017) while providing much faster locate time.…”

Section: Discussionmentioning

confidence: 99%

“…Next, we demonstrate the applicability of our method to short-read alignment. In particular, we compare the memory and time needed by our method to build an index for collections of chromosome 19 with those of Bowtie (Langmead et al, 2008) and CHIC (Valenzuela and Mäkinen, 2017). We also compare the sizes of the resulting indexes as well as the amount of time required to perform several locate queries against the indexes.…”

Section: Complete Index For Pan-genomics Read Alignmentmentioning

confidence: 99%

“…Some of these methods-such as GCSA-use the FM-index to store and query the graph and could capitalize on our approach by building the index in the manner described here. Another set of approaches (Mäkinen et al, 2010;Ferrada et al, 2014;Gagie and Puglisi, 2015;Valenzuela and Mäkinen, 2017) considers the reference pan-genome as the concatenation of individual genomes and exploits redundancy using a compressed index.…”

Section: Related Workmentioning

confidence: 99%

“…We studied how r-index scales to repetitive texts consisting of many similar genomic sequences, comparing it with Bowtie (version 1.2.2) (Langmead et al, 2008), a traditional FM-index-based aligner, and CHIC (Valenzuela and Mäkinen, 2017), a Hybrid Index that uses LZ compression to scale to repetitive texts. We measured indexing memory footprint, indexing time, index size, and locate query time.…”

Section: Comparison With Bowtie and Chicmentioning

confidence: 99%

See 3 more Smart Citations

Efficient Construction of a Complete Index for Pan-Genomics Read Alignment

Kuhnle

Mun²,

Boucher

et al. 2020

Journal of Computational Biology

View full text Add to dashboard Cite

Short-read aligners predominantly use the FM-index, which is easily able to index one or a few human genomes. However, it does not scale well to indexing collections of thousands of genomes. Driving this issue are the two chief components of the index: (1) a rank data structure over the Burrows-Wheeler Transform (BWT) of the string that will allow us to find the interval in the string's suffix array (SA), and (2) a sample of the SA that-when used with the rank data structure-allows us to access the SA. The rank data structure can be kept small even for large genomic databases, by run-length compressing the BWT, but until recently there was no means known to keep the SA sample small without greatly slowing down access to the SA. Now that (SODA 2018) has defined an SA sample that takes about the same space as the run-length compressed BWT, we have the design for efficient FM-indexes of genomic databases but are faced with the problem of building them. In 2018, we showed how to build the BWT of large genomic databases efficiently (WABI 2018), but the problem of building the sample efficiently was left open. We compare our approach to state-of-the-art methods for constructing the SA sample, and demonstrate that it is the fastest and most space-efficient method on highly repetitive genomic databases. Lastly, we apply our method for indexing partial and whole human genomes and show that it improves over the FMindex-based Bowtie method with respect to both memory and time and over the hybrid index-based CHIC method with respect to query time and memory required for indexing.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Complete Index For Pan-genomics Read Alignmentmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Comparison With Bowtie and Chicmentioning

confidence: 99%

See 2 more Smart Citations

Efficient Construction of a Complete Index for Pan-Genomics Read Alignment

Kuhnle

Mun²,

Boucher

et al. 2020

Journal of Computational Biology

View full text Add to dashboard Cite

show abstract

“…Some of these methods -such as the GCSA -use the FM-index to store and query the graph and could capitalize on our approach by building the index in the manner described here. Another set of approaches [24,8,12,32] consider the reference pan-genome as the concatenation of individual genomes and exploits redundancy by using a compressed index. The hybrid index [8] operates on a Lempel-Ziv compression of the reference pan-genome.…”

Section: Introductionmentioning

confidence: 99%

Efficient Construction of a Complete Index for Pan-Genomics Read Alignment

Kuhnle

Mun²,

Boucher

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

While short read aligners, which predominantly use the FM-index, are able to easily index one or a few human genomes, they do not scale well to indexing databases containing thousands of genomes. To understand why, it helps to examine the main components of the FM-index in more detail, which is a rank data structure over the Burrows-Wheeler Transform (BWT) of the string that will allow us to find the interval in the string's suffix array (SA) containing pointers to starting positions of occurrences of a given pattern; second, a sample of the SA that -when used with the rank data structure -allows us access the SA. The rank data structure can be kept small even for large genomic databases, by run-length compressing the BWT, but until recently there was no means known to keep the SA sample small without greatly slowing down access to the SA. Now that Gagie et al. (SODA 2018) have defined an SA sample that takes about the same space as the run-length compressed BWT -we have the design for efficient FM-indexes of genomic databases but are faced with the problem of building them. In 2018 we showed how to build the BWT of large genomic databases efficiently (WABI 2018) but the problem of building Gagie et al.'s SA sample efficiently was left open. We compare our approach to state-of-the-art methods for constructing the SA sample, and demonstrate that it is the fastest and most space-efficient method on highly repetitive genomic databases. Lastly, we apply our method for indexing partial and whole human genomes, and show that it improves over Bowtie with respect to both memory and time.Availability: We note that the implementation of our methods can be found here: https://github. com/alshai/r-index. Equal contribution

show abstract

Scalable Reference Genome Assembly from Compressed Pan-Genome Index with Spark

Maarala

Arasalo

Valenzuela

et al. 2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

High-throughput sequencing (HTS) technologies have enabled rapid sequencing of genomes and large-scale genome analytics with massive data sets. Traditionally, genetic variation analyses have been based on the human reference genome assembled from a relatively small human population. However, genetic variation could be discovered more comprehensively by using a collection of genomes i.e., pan-genome as a reference. The pan-genomic references can be assembled from larger populations or a specific population under study. Moreover, exploiting the pan-genomic references with current bioinformatics tools requires efficient compression and indexing methods. To be able to leverage the accumulating genomic data, the power of distributed and parallel computing has to be harnessed for the new genome analysis pipelines. We propose a scalable distributed pipeline, PanGenSpark, for compressing and indexing pan-genomes and assembling a reference genome from the pan-genomic index. We experimentally show the scalability of the Pan-GenSpark with human pan-genomes in a distributed Spark cluster comprising 448 cores distributed to 26 computing nodes. Assembling a consensus genome of a pan-genome including 50 human individuals was performed in 215 minutes and with 500 human individuals in 1468 minutes. The index of 1.41 TB pan-genome was compressed into a size of 164.5 GB in our experiments.

show abstract

CHIC: a short read aligner for pan-genomic references

Cited by 10 publications

References 22 publications

Efficient Construction of a Complete Index for Pan-Genomics Read Alignment

Efficient Construction of a Complete Index for Pan-Genomics Read Alignment

Efficient Construction of a Complete Index for Pan-Genomics Read Alignment

Scalable Reference Genome Assembly from Compressed Pan-Genome Index with Spark

Contact Info

Product

Resources

About