An elegant algorithm for the construction of suffix arrays

Abstract

The suffix array is a data structure that finds numerous applications in string processing problems for both linguistic texts and biological data. It has been introduced as a memory efficient alternative for suffix trees. The suffix array consists of the sorted suffixes of a string. There are several linear time suffix array construction algorithms (SACAs) known in the literature. However, one of the fastest algorithms in practice has a worst case run time of O(n2). The problem of designing practically and theoretically efficient techniques remains open. In this paper we present an elegant algorithm for suffix array construction which takes linear time with high probability; the probability is on the space of all possible inputs. Our algorithm is one of the simplest of the known SACAs and it opens up a new dimension of suffix array construction that has not been explored until now. Our algorithm is easily parallelizable. We offer parallel implementations on various parallel models of computing. We prove a lemma on the ℓ-mers of a random string which might find independent applications. We also present another algorithm that utilizes the above algorithm. This algorithm is called RadixSA and has a worst case run time of O(n log n). RadixSA introduces an idea that may find independent applications as a speedup technique for other SACAs. An empirical comparison of RadixSA with other algorithms on various datasets reveals that our algorithm is one of the fastest algorithms to date. The C++ source code is freely available at http://www.engr.uconn.edu/~man09004/radixSA.zip.

On String Matching with Mismatches

“…All convolutions were performed using the fftw [28] library Version 3.3.3. We used the suffix array algorithm RadixSAof [29]. Figure 1 shows run times for varying the length of the text n. All algorithms scale linearly with the length of the text.…”

Improving the sensitivity of long read overlap detection using grouped short k-mer matches

“…We implemented k -mer counting using a generalized suffix array and the derived longest common prefix (LCP) array. The generalized suffix array S A is created from the concatenated reads (delimited by special characters such as $) using a linear algorithm [33]. Then, we create the LCP using both the suffix array S A and the reversed suffix array S A ′ [34, 35].…”

“…For each position i in the LCP, LCP[i] contains the size of the longest common prefix between S A [ i ] and S A [ i −1]. The key observation [33] for efficient computation of LCP[i] is: for a position j in T , if L C P [ S A ′ [ j −1]] is L , L C P [ S A ′ [ j ]]≥ L −1. The whole LCP array construction takes linear time to the size of T [33].…”

Homology-aware Phylogenomics at Gigabase Scales

“…This data structure requires \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}${\sim}10$\end{document} bytes/nucleotide. This array is constructed by a new, fast sorting method that is highly scalable ( Rajasekaran and Nicolae 2014 ), having worst case run times of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$O(L$\end{document} log \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$L)$\end{document} and usually much better than this in practice. Once the suffix array is sorted, exact \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$k$\end{document} -mer matches form contiguous blocks in the array.…”

“…To keep the speed and simplicity of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$k$\end{document} -mer based approaches but retain information about positional homology, we combined and extended several well-tested ideas in new ways ( Gardner and Hall 2013 ; Leimeister and Morgenstern 2014 ; Fan et al 2015 ; Haubold et al 2015 ) and leveraged recent improvements in engineering of a key data structure ( Rajasekaran and Nicolae 2014 ). From a set of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$N$\end{document} genomes, which may be at various stages of assembly, our algorithm builds short multiple sequence alignments, or “ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$k$\end{document} -mer blocks,” starting from approximately matching \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$k$\end{document} -mer “seeds” ( Fig.…”