A syllable-based method for Vietnamese text compression

Nguyen, Vu; Nguyen, Hien; Duong, Hieu N.; Snášel, Václav

doi:10.1145/2857546.2857564

Cited by 2 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Table 9, we show the results of the current method in 10 cases in comparison with the methods in [13, 14]. As shown in Table 9 and Figure 3, the compression ratio of our method is better than the methods presented in [13, 14] for any size of text in our test cases.…”

Section: Methodsmentioning

confidence: 84%

“…In order to evaluate our method with the methods presented in [13, 14], we compress the input files using these methods. In Table 9, we show the results of the current method in 10 cases in comparison with the methods in [13, 14].…”

Section: Methodsmentioning

confidence: 99%

“…Tables 11 and 12 and Figures 5 and 6 show the compression and decompression time of our method in comparison with the methods in [13, 14] and WinRAR, respectively. In these tables and figures, we have some abbreviations and meanings as follows: CT: compression time; DT: decompression time; RAR: WinRAR; O: our method; ms: millisecond.…”

Section: Methodsmentioning

confidence: 99%

“…In our previous papers for Vietnamese text compression [13, 14], we proposed a syllable-based method based on morphology and syllable dictionaries in [13]. With each morphosyllable, it is split into a consonant and a syllable, and they are compressed based on their corresponding dictionaries.…”

Section: Related Workmentioning

confidence: 99%

“…This paper presents the first attempt at text compression using n -gram dictionaries, and the contribution has three attributes; that is, (1) it is a method for text compression using n -gram dictionaries, (2) it collects the text corpus of the Vietnamese language from the Internet and builds five n -gram dictionaries with nearly 500,000,000 n -grams, and (3) a test set of 10 different text files with different sizes to evaluate our new system and compare it with my two previous methods [13, 14] and also with other methods. The rest of this paper is organized as follows.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

n-Gram-Based Text Compression

Nguyen

Duong

et al. 2016

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

We propose an efficient method for compressing Vietnamese text using n-gram dictionaries. It has a significant compression ratio in comparison with those of state-of-the-art methods on the same dataset. Given a text, first, the proposed method splits it into n-grams and then encodes them based on n-gram dictionaries. In the encoding phase, we use a sliding window with a size that ranges from bigram to five grams to obtain the best encoding stream. Each n-gram is encoded by two to four bytes accordingly based on its corresponding n-gram dictionary. We collected 2.5 GB text corpus from some Vietnamese news agencies to build n-gram dictionaries from unigram to five grams and achieve dictionaries with a size of 12 GB in total. In order to evaluate our method, we collected a testing set of 10 different text files with different sizes. The experimental results indicate that our method achieves compression ratio around 90% and outperforms state-of-the-art methods.

show abstract

Section: Methodsmentioning

confidence: 84%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

n-Gram-Based Text Compression

Nguyen

Duong

et al. 2016

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

show abstract

A Syllable-Based Technique for Uyghur Text Compression

Abliz

Maimaiti

et al. 2020

Information

View full text Add to dashboard Cite

To improve utilization of text storage resources and efficiency of data transmission, we proposed two syllable-based Uyghur text compression coding schemes. First, according to the statistics of syllable coverage of the corpus text, we constructed a 12-bit and 16-bit syllable code tables and added commonly used symbols—such as punctuation marks and ASCII characters—to the code tables. To enable the coding scheme to process Uyghur texts mixed with other language symbols, we introduced a flag code in the compression process to distinguish the Unicode encodings that were not in the code table. The experiments showed that the 12-bit coding scheme had an average compression ratio of 0.3 on Uyghur text less than 4 KB in size and that the 16-bit coding scheme had an average compression ratio of 0.5 on text less than 2 KB in size. Our compression schemes outperformed GZip, BZip2, and the LZW algorithm on short text and could be effectively applied to the compression of Uyghur short text for storage and applications.

show abstract

A syllable-based method for Vietnamese text compression

Cited by 2 publications

References 14 publications

n-Gram-Based Text Compression

n-Gram-Based Text Compression

A Syllable-Based Technique for Uyghur Text Compression

Contact Info

Product

Resources

About