Compression of unicode files

Fenwick, Peter; Brierley, Scott

doi:10.1109/dcc.1998.672274

Cited by 3 publications

(2 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach yields a 12.2% average improvement in compression effectiveness for LZW over a corpus of UTF-8 files. This result compares favourably with the Unicode-aware LZ77 variant from Fenwick and Brierley that achieved only a 2% improvement [6]. Unfortunately their corpus is not available to us, precluding a direct comparison.…”

Section: Introductionmentioning

confidence: 63%

Making Compression Algorithms for Unicode Text

Gleave

Steinruecken

2017

2017 Data Compression Conference (DCC)

View full text Add to dashboard Cite

The majority of online content is written in languages other than English, and is most commonly encoded in UTF-8, the world's dominant Unicode character encoding. Traditional compression algorithms typically operate on individual bytes. While this approach works well for the single-byte ASCII encoding, it works poorly for UTF-8, where characters often span multiple bytes. Previous research has focused on developing Unicode compressors from scratch, which often failed to outperform established algorithms such as bzip2.We develop a technique to modify byte-based compressors to operate directly on Unicode characters, and implement variants of LZW and PPM that apply this technique. We find that our method substantially improves compression effectiveness on a UTF-8 corpus, with our PPM variant outperforming the state-of-the-art PPMII compressor. On ASCII and binary files, our variants perform similarly to the original unmodified compressors.

show abstract

Section: Introductionmentioning

confidence: 63%

Making Compression Algorithms for Unicode Text

Gleave

Steinruecken

2017

2017 Data Compression Conference (DCC)

View full text Add to dashboard Cite

show abstract

“…This fact necessitates the need of a peculiar compression technique for natural languages. A small corpus of Unicode files has been compressed on several widely available text compressors of the various types by Fenwick and Brierley (1998), confirming that Unicode files have different compression characteristics from those known for 8-bit data. The Malayalam text compression by variable length encoding was explained by Divakaran et al (2013), after an informational analysis of Malayalam Language.…”

Section: Introductionmentioning

confidence: 90%

A Compression System for Unicode Files Using an Enhanced Lzw Method

Anto¹,

Ramachandran²

2020

JST

View full text Add to dashboard Cite

Data compression plays a vital and pivotal role in the process of computing as it helps in space reduction occupied by a file as well as to reduce the time taken to access the file. This work relates to a method for compressing and decompressing a UTF-8 encoded stream of data pertaining to Lempel-Ziv-welch (LZW) method. It is worth to use an exclusive-purpose LZW compression scheme as many applications are utilizing Unicode text. The system of the present work comprises a compression module, configured to compress the Unicode data by creating the dictionary entries in Unicode format. This is accomplished with adaptive characteristic data compression tables built upon the data to be compressed reflecting the characteristics of the most recent input data. The decompression module is configured to decompress the compressed file with the help of unique Unicode character table obtained from the compression module and the encoded output. We can have remarkable gain in compression, wherein the knowledge that we gather from the source is used to explore the decompression process.

show abstract

On the Costs of Multilingualism in Database Systems

Kumaran

Haritsa

2003

Proceedings 2003 VLDB Conference

View full text Add to dashboard Cite

Compression of unicode files

Cited by 3 publications

References 4 publications

Making Compression Algorithms for Unicode Text

Making Compression Algorithms for Unicode Text

A Compression System for Unicode Files Using an Enhanced Lzw Method

On the Costs of Multilingualism in Database Systems

Contact Info

Product

Resources

About