A Survey on Data Compression Methods for Biological Sequences

Abstract

Abstract:The ever increasing growth of the production of high-throughput sequencing data poses a serious challenge to the storage, processing and transmission of these data. As frequently stated, it is a data deluge. Compression is essential to address this challenge-it reduces storage space and processing costs, along with speeding up data transmission. In this paper, we provide a comprehensive survey of existing compression approaches, that are specialized for biological data, including protein and DNA sequences. Also, we devote an important part of the paper to the approaches proposed for the compression of different file formats, such as FASTA, as well as FASTQ and SAM/BAM, which contain quality scores and metadata, in addition to the biological sequences. Then, we present a comparison of the performance of several methods, in terms of compression ratio, memory usage and compression/decompression time. Finally, we present some suggestions for future research on biological data compression.

Representation ofk-mer sets using spectrum-preserving string sets

“…DSK [18] uses an HDF5-based encoding, KMC3 [17] combines a dense storage of prefixes with a sparse storage of suffixes, and Squeakr [20] uses a counting quotient filter [21]. The compression of read data, on the other hand, stored in either unaligned or aligned formats, has received a lot of attention [22][23][24]. In the scenario where the k-mer set to be compressed was originally generated from FASTA files by a k-mer counter, an alternate to k-mer compression is to compress the original FASTA file and use a k-mer counter as part of the decompression to extract the k-mers on the fly.…”

Design of genomic signatures for pathogen identification and characterization

“…It is possible that new concepts and terminology will be needed to map existing taxonomic categories into the genomic reality of the 21st century. Similarly, compressed genome data storage techniques, including just mapping differences relative to reference genomes of one or more species, may be leveraged to reduce data storage, transfer, and computation bottlenecks (Hosseini et al, 2016).…”

GTO: a toolkit to unify pipelines in genomic and proteomic research

“…For example, new models can be tested under this framework, namely with extended alphabets [8]. In general, any data compressor able to output local estimations can be used in the pipeline as an alternative [9].…”