Suffix Arrays: A New Method for On-Line String Searches

Abstract

A new and conceptually simple data structure, called a suffix array, for on-line string searches is intro

Optimal In-place Sorting of Vectors and Records

“…With sophisticated techniques for proving upper and lower bounds on the complexity of searching V in lexicographic order, Andersson, Hagerup, Håstad and Petersson have proved in [1] that it requires Θ k log log n log log(4 + k log log n log n ) + k + log n time. This bound is worse than Θ(k + log n), obtained by searching V plus O(n) auxiliary locations (e.g., Manber and Myers [18]). Using permutations other than those resulting from sorting is a way to reach optimality: Franceschini and Grossi [10] have shown that for any set V of n vectors in lexicographic order, there exists a permutation of them allowing for Θ(k + log n) search time using O(1) auxiliary data locations.…”

“…The pivots inside M B are kept searchable by a suitable blend of the techniques in [10,13,18], requiring to decode O(log n) heavy bits per inserted vector (which is fine since decoding takes O(1 + k/ log n) time). In particular, we logically divide each vector x into a concatenation of O(log m ) = O(log n) equally sized chunks.…”

Fast Plagiarism Detection System

“…A suffix array allows us to rapidly find a file (or files), containing any given substring. This is achieved with a binary search, and requires O(m + log 2 n) time on average, where m is the length of the substring (it is also possible to make this the worst case complexity, see [4]). The array can be constructed in time O(n log n), assuming atomic comparison of two tokens.…”

“…We use suffix array as an index structure. A suffix array is a lexicographically sorted array of all suffixes of a given string [4]. The suffix array for the whole document collection is of size O(n).…”

“…For example, a typical tokenization scheme might involve replacing all identifiers with the <IDT> token, all numbers by <VALUE> and any loops by generic <BEGIN LOOP>...<END LOOP> tokens. Our algorithm also makes use of tokenised versions of the input files and we use suffix arrays [4] as our index data structure to enable efficient comparisons.…”