Engineering a Compressed Suffix Tree Implementation

Abstract

Suffix tree is one of the most important data structures in string algorithms and biological sequence analysis. Unfortunately, when it comes to implementing those algorithms and applying them to real genomic sequences, often the main memory size becomes the bottleneck. This is easily explained by the fact that while a DNA sequence of length n from alphabet Σ = {A, C, G, T } can be stored in n log |Σ| = 2n bits, its suffix tree occupies O(n log n) bits. In practice, the size difference easily reaches factor 50.We report on an implementation of the compressed suffix tree very recently proposed by Sadakane (Theory of Computing Systems, in press). The compressed suffix tree occupies space proportional to the text size, i.e. O(n log |Σ|) bits, and supports all typical suffix tree operations with at most log n factor slowdown. Our experiments show that, e.g. on a 10 MB DNA sequence, the compressed suffix tree takes 10% of the space of the normal suffix tree. At the same time, a representative algorithm is slowed down by factor 30.Our implementation follows the original proposal in spirit, but some internal parts are tailored towards practical implementation. Our construction algorithm has time requirement O(n log n log |Σ|) and uses closely the same space as the final structure while constructing it: on the 10 MB DNA sequence, the maximum space usage during construction is only 1.5 times the final product size. As by-products, we develop a method to create Succinct Suffix Array directly from Burrows-Wheeler transform and a space-efficient version of suffixes-insertion algorithm to build balanced parentheses representation of suffix tree from LCP information.

Um sistema para auxiliar na seleção de regiões específicas de sequências biológicas

“…Aárvore de sufixo comprimida (do inglês compressed suffix trees ou CST)é considerada uma estrutura de dados de indexação própria, com tamanho reduzido, que representa umaárvore de sufixo, através de vetores, simulando as principais funcionalidades convencionais daárvore de sufixo [Välimäki et al 2009].…”

Space Efficient K-MER Algorithm for Generalised Suffix Tree

“…al. [22] designed an algorithm to address the memory challenge by using the Compressed Suffix Tree (CST), there was an improvement on the memory requirements however the memory requirements remained impractically high for it to be adopted and usable. For instance, for the whole human genome, the indexing takes around 4 days, the complete tree takes around 8.5 GB with a maximum amount of memory consumption of 24 GB.…”

Extended suffix array construction using Lyndon factors

“…An array representing all the suffixes of the text in their lexicographic order is called as the suffix array (SA). Improving the space efficiency of suffix array data structure led to a new class of data structure abstract data structures like compressed suffix array [1][2][3][4][5] and then compressed suffix trees [6][7][8][9] with many variations in their implementations. The research in this class of data structure has been oriented towards finding an optimal space/time trade-off in their implementation either during the construction or in their use as an indexing data structure.…”