Compressing Genomic Sequence Fragments Using SlimGene

Abstract

Abstract. With the advent of next generation sequencing technologies, the cost of sequencing whole genomes is poised to go below $1000 per human individual in a few years. As more and more genomes are sequenced, analysis methods are undergoing rapid development, making it tempting to store sequencing data for long periods of time so that the data can be re-analyzed with the latest techniques. The challenging open research problems, huge influx of data, and rapidly improving analysis techniques have created the need to store and transfer very large volumes of data. Compression can be achieved at many levels, including trace level (image data), sequence level (compressing a non-redundant genomic sequence), and fragmentlevel, which involves compressing a set of short, and redundant collection of fragment reads, along with quality-values on the base-calls. We focus on fragmentlevel compression, which is the pressing need today. Our paper makes two contributions, implemented in a tool, SLIMGENE. First, we introduce a set of domain specific loss-less compression schemes that achieve over 40× compression of fragments, outperforming bzip2 by over 6×. Including quality values, we show a 5× compression using less running time than bzip2. Second, given the discrepancy between the compression factor obtained with and without quality values, we initiate the study of using 'lossy' quality values. Specifically, we show that a lossy quality value quantization results in 14× compression but has minimal impact on downstream applications like SNP calling that use the quality values. Discrepancies between SNP calls made between the lossy and lossless versions of the data are limited to low coverage areas where even the SNP calls made by the lossless version are marginal.

Trends in Genome Compression

“…Following a similar approach, [68] propose SLIMGENE, a lossless or lossy reference-based compression scheme focusing on how to find encodings of integers in order to minimize storage. In their work, they employ Huffman and arithmetic encoding.…”

Identification of four novel genes contributing to familial elevated plasma HDL cholesterol in humans

“…To reduce the identifi cation of false variant calls, the following quality control fi lters were applied: A ) at least 75-fold sequence coverage per sample pool of fi ve probands (at least ‫ف‬ 15-fold per proband DNA); B ) a minor allele frequency >0.05 in sample pools; C ) detection in pooled high quality sequence data: major allele BaCON score >10, minor allele BaCON score >6, and BaCON score ratio <15:1 (the BaCON score is generated by Illumina and represents the discrepancy of variants. A score of р 6 indicates variants below the threshold of detection) ( 24 ); and D ) found only in LHDL or HHDL sample pools ( Fig. 2 ).…”

Compressive genomics

“…2), 1,000 human genomes contain less than twice the unique information of one genome. Thus, although individual genomes are not very compressible 12,13 , collections of related genomes are extremely compressible [14][15][16][17] .…”

“…As more divergent genomes are added to a database, Many algorithms exist for the compression of genomic data sets purely to reduce the space required for storage and transmission [12][13][14][15]17,18 . Hsi-Yang Fritz et al 18 provide a particularly instructive discussion of the concerns involved.…”