Information compression by factorising common strings

Mayne, Alan J.

doi:10.1093/comjnl/18.2.157

Cited by 21 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wagner [1973] gave an algorithm based on dynamic programming for optimally parsing the source into a skeleton, given a collection of phrases (similar to a dictionary, though less flexible), but did not show how the phrases were best generated. A heuristic algorithm for generating a dictionary was presented in Mayne and James [1975]. Storer and Szymanski [1982] showed that the problem of deciding whether the length of the shortest possible compressed form is less than k is NP-complete.…”

Section: Data Compressionmentioning

confidence: 99%

A text-compression-based method for code size minimization in embedded systems

Liao

Devadas

Keutzer

1999

ACM Trans. Des. Autom. Electron. Syst.

View full text Add to dashboard Cite

We address the problem of code-size minimization in VLSI systems with embedded DSP processors. Reducing code size reduces the production cost of embedded systems.We use data-compression methods to develop code-size minimization strategies. In our framework, the compressed program consists of a skeleton and a dictionary. We show that the dictionary can be computed by solving a set-covering problem derived from the original program. To execute the compressed code, we describe two methods that have different performance characteristics and different degrees of freedom in compressing the code. We also address performance considerations, and show that they can be incorporated easily into the set-covering formulation, and present experimental results obtained with Texas Instruments' optimizing TMS320C25 compiler.

show abstract

Section: Data Compressionmentioning

confidence: 99%

A text-compression-based method for code size minimization in embedded systems

Liao

Devadas

Keutzer

1999

ACM Trans. Des. Autom. Electron. Syst.

View full text Add to dashboard Cite

show abstract

“…The task of finding an optimal phrase dictionary for a set of data is NP-hard [Storer and Szymanski 1982]. Therefore, approaches such as LZ78 and alternative phrase-based approaches [Lynch 1973;Mayne and James 1975;Schuegraf and Heaps 1974;Wagner 1973;Wolff 1978] require the introduction of heuristics to address the complexity issues. Our RAY [Cannane and Williams 2001] algorithm also uses heuristics and is similiar to other recent approaches [Larsson and Moffat 2000;Nakamura and Murashima 1996] which are closely related to the work of Rubin and the incremental construction of phrases.…”

Section: Phrase-based Compressionmentioning

confidence: 99%

A general-purpose compression scheme for large collections

Cannane

Williams

2002

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

Compression of large collections can lead to improvements in retrieval times by offsetting the CPU decompression costs with the cost of seeking and retrieving data from disk. We propose a semistatic phrase-based approach called XRAY that builds a model offline using sample training data extracted from a collection, and then compresses the entire collection online in a single pass. The particular benefits of XRAY are that it can be used in applications where individual records or documents must be decompressed, and that decompression is fast. The XRAY scheme also allows new data to be added to a collection without modifying the semistatic model. Moreover, XRAY can be used to compress general-purpose data such as genomic, scientific, image, and geographic collections without prior knowledge of the structure of the data. We show that XRAY is effective on both text and general-purpose collections. In general, XRAY is more effective than the popular GZIP and COMPRESS schemes, while being marginally less effective than BZIP2. We also show that XRAY is efficient: of the popular schemes we tested, it is typically only slower than GZIP in decompression. Moreover, the query evaluation costs of retrieval of documents from a large collection with our search engine is improved by more than 30% when XRAY is incorporated compared to an uncompressed approach. We use simple techniques for obtaining the training data from the collection to be compressed and show that with just over 4% of data the entire collection can be effectively compressed. We also propose four schemes for phrase-match selection during the single pass compression of the collection. We conclude that with these novel approaches XRAY is a fast and effective scheme for compression and decompression of large general-purpose collections.

show abstract

“…Wagner [7] gave an algorithm based on dynamic programming for optimally parsing the source into a skeleton given a collection of phrases (similar to a dictionary, though less flexible), but did not show how the phrases were best generated. A heuristic algorithm for generating a dictionary was presented in Mayne and James [5]. Storer and Szymanski [6] showed that the problem of deciding whether the length of the shortest possible compressed form is less than k is NP-complete.…”

Section: 1: Data Compressionmentioning

confidence: 99%

Code density optimization for embedded DSP processors using data compression techniques

Liao

Devadas²,

Keutzer

1998

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

We address the problem of code size minimization in VLSI systems with embedded DSP processors. Reducing code size reduces the production cost of embedded systems. We use data compression methods to develop code size minimization strategies. We present a framework for code size minimization where the compressed data consists of a dictionary and a skeleton. The dictionary can be computed using popular text compression algorithms. We describe two methods to execute the compressed code that have varying performance characteristics and varying degrees of freedom in compressing the code. Experimental results obtained with a TMS320C25 code generator are presented.

show abstract

Information compression by factorising common strings

Cited by 21 publications

References 0 publications

A text-compression-based method for code size minimization in embedded systems

A text-compression-based method for code size minimization in embedded systems

A general-purpose compression scheme for large collections

Code density optimization for embedded DSP processors using data compression techniques

Contact Info

Product

Resources

About