Collage system: a unifying framework for compressed pattern matching

Kida, Takuro; Matsumoto, Tetsuya; Shibata, Yusuke; Takeda, Masayuki; Shinohara, Ayumi; Arikawa, Setsuo

doi:10.1016/s0304-3975(02)00426-7

Cited by 80 publications

(61 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been successfully used in different scenarios related to wordbased text compression [17], searching compressed text [8], compression of Web graphs [4], and suffix array compression [7], to give some examples.…”

Section: Introductionmentioning

confidence: 99%

Re-pair Achieves High-Order Entropy

Navarro

S.Russo

2008

Data Compression Conference (Dcc 2008)

View full text Add to dashboard Cite

Re-Pair is a dictionary-based compression method invented in 1999 by Larsson and Moffat. Although its practical performance has been established through experiments, the method has resisted all attempts of formal analysis. In this paper we show that Re-Pair compresses a sequence T [1, n] over an alphabet of size σ and k-th order entropy H k , to at most 2H k + o(n log σ) bits, for any k = o(log σ n).

show abstract

Section: Introductionmentioning

confidence: 99%

Re-pair Achieves High-Order Entropy

Navarro

S.Russo

2008

Data Compression Conference (Dcc 2008)

View full text Add to dashboard Cite

show abstract

“…Navarro and Raffinot [13] developed more general technique, which abstracts both LZ77 and LZ78 and runs in O(nm/w + m + occ), where w is the machine word length and occ is the number of pattern occurrences. Kida et al [7] proposed the collage systems: a formal system to represent a string by dictionary D and sequence S of variables, which unifies various dictionary-based compressions such as LZ family (LZ77, LZSS, LZ78, LZW), CFG transform based compressions, Run-Length encoding, etc. They also presented a general CPM algorithm on collage systems which…”

Section: Introductionmentioning

confidence: 99%

“…The aim of compression in the context of Goal 2 is not only to reduce disk storage requirement or data transmission cost but also to speed up string searching. In this section, we consider the CPM problem for restricted Σ-sensitive grammars and show a CPM algorithm based on [7]. We then discuss a Goal 2-oriented implementation of it.…”

mentioning

confidence: 99%

Context-Sensitive Grammar Transform: Compression and Pattern Matching

MARUYAMA

Tanaka

Sakamoto

et al. 2010

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

SUMMARYA framework of context-sensitive grammar transform for speeding-up compressed pattern matching (CPM) is proposed. A greedy compression algorithm with the transform model is presented as well as a Knuth-Morris-Pratt (KMP)-type compressed pattern matching algorithm. The compression ratio is a match for gzip and Re-Pair, and the search speed of our CPM algorithm is almost twice faster than the KMP-type CPM algorithm on Byte-Pair-Encoding by Shibata et al. [18], and in the case of short patterns, faster than the Boyer-Moore-Horspool algorithm with the stopper encoding by Rautio et al. [14], which is regarded as one of the best combinations that allows a practically fast search.

show abstract

“…There are textual substitution compression methods which are more powerful than those CFG-based [17]. A well-known one is LZ77 [30], which cannot be directly expressed using CFGs.…”

Section: Introduction and Related Workmentioning

confidence: 99%

“…As for finding patterns, there has been much work on sequential compressed pattern matching [1], that is, scanning the whole grammar. The most attractive result is that of Kida et al [17], which can search general SLPs/CFGs in time O(n + m 2 + occ). This may be o(u), but still linear in the size of the compressed text.…”

Section: Introduction and Related Workmentioning

confidence: 99%

Self-indexed Text Compression Using Straight-Line Programs

Claude

Navarro

2009

Mathematical Foundations of Computer Science 2009

View full text Add to dashboard Cite

Abstract. Straight-line programs (SLPs) offer powerful text compression by representing a text T [1, u] in terms of a restricted context-free grammar of n rules, so that T can be recovered in O(u) time. However, the problem of operating the grammar in compressed form has not been studied much. We present a grammar representation whose size is of the same order of that of a plain SLP representation, and can answer other queries apart from expanding nonterminals. This can be of independent interest. We then extend it to achieve the first grammar representation able of extracting text substrings, and of searching the text for patterns, in time o(n). We also give byproducts on representing binary relations.

show abstract

Collage system: a unifying framework for compressed pattern matching

Cited by 80 publications

References 30 publications

Re-pair Achieves High-Order Entropy

Re-pair Achieves High-Order Entropy

Context-Sensitive Grammar Transform: Compression and Pattern Matching

Self-indexed Text Compression Using Straight-Line Programs

Contact Info

Product

Resources

About