QVZ: lossy compression of quality values

Malysa, Greg; Hernáez, Mikel; Ochoa, Idoia; Mv, Rao; Ganesan, Karthik; Weissman, Tsachy

doi:10.1093/bioinformatics/btx654

Cited by 21 publications

(37 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This latter test demonstrates that quality values are important, but we only need a small quality budget to achieve comparable results to lossless compression. Indeed, we observe that vertical quality score compression can marginally improve variant calling by standard callers, as has been noted previously in the QVZ (Malysa et al, 2015) and Leon (Benoit et al, 2015) papers. Table 1 shows the GATK lossless results on the CHM pair along with the changes caused by lossy compression using a variety of Crumble options on both the full Syndip data and a low coverage subset.…”

Section: Resultssupporting

confidence: 84%

See 1 more Smart Citation

Crumble: reference free lossy compression of sequence quality values

Bonfield

McCarthy

Durbin

2018

Preprint

View full text Add to dashboard Cite

Motivation: The bulk of space taken up by NGS sequencing CRAM files consists of per-base quality values. Most of these are unnecessary for variant calling, offering an opportunity for space saving. Results: On the CHM1+CHM13 test set, a 17 fold reduction in quality storage can be achieved while maintaining variant calling accuracy. Availability: Crumble is OpenSource and can be obtained from https://github.com/jkbonfield/crumble.

show abstract

Section: Resultssupporting

confidence: 84%

“…"Horizontal" compression smooths qualities along each sequence in turn, as implemented in libCSAM (Cánovas et al, 2014), QVZ (Malysa et al, 2015) and FaStore (Roguski et al, 2017) or via quantisation (Illumina, 2014). This type of compression can be applied before alignment and is entirely reference free.…”

Section: Introductionmentioning

confidence: 99%

Crumble: reference free lossy compression of sequence quality values

Bonfield

McCarthy

Durbin

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…P-Block and R-Block are two state-of-the-art lossy quality score compressors 17 which have also been shown to improve variant calling performance. Another state-of-the-art lossy compressor, QVZ 18 , has been omitted from the analysis since it is unable to compress SAM files with variable length reads. Again we focus our discussion on data set 2 and observe that in Supplementary Figure 4 shows that the effect of lossy compression on the variant call set is almost indistinguishable from that of SAMDUDE.…”

Section: Comparison Of Samdude Denoising With State-of-the-art Genomimentioning

confidence: 99%

Denoising of Aligned Genomic Data

Fischer-Hwang

Ochoa

Weissman

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Noise in genomic sequencing data is known to have effects on various stages of genomic data analysis pipelines. Variant identification is an important step of many of these pipelines, and is increasingly being used in clinical settings to aid medical practices. We propose a denoising method, dubbed SAMDUDE, which operates on aligned genomic data in order to improve variant calling performance. Denoising human data with SAMDUDE resulted in improved variant identification in both individual chromosome as well as whole genome sequencing (WGS) data sets. In the WGS data set, denoising led to identification of almost 2,000 additional true variants, and elimination of over 1,500 erroneously identified variants. In contrast, we found that denoising with other state-of-the-art denoisers significantly worsens variant calling performance. SAMDUDE is written in Python and is freely available at https://github.com/ihwang/SAMDUDE. 2/12

show abstract

“…Still, little effort has been made to compress MS data. This is in contrast to other omics disciplines, such as genomics, that have experienced an increasing effort in designing specialized compression schemes to lessen the storage burden [17,18,19].…”

Section: Introductionmentioning

confidence: 99%

MassComp, a lossless compressor for mass spectrometry data

Yang

Chen

Ochoa

2019

Preprint

Self Cite

View full text Add to dashboard Cite

Background: Mass Spectrometry (MS) is a widely used technique in biology research, and has become key in proteomics and metabolomics analyses. As a result, the amount of MS data has significantly increased in recent years. For example, the MS repository MassIVE contains more than 123TB of data. Somehow surprisingly, these data are stored uncompressed, hence incurring a significant storage cost. Efficient representation of these data is therefore paramount to lessen the burden of storage and facilitate its dissemination. Results: We present MassComp, a lossless compressor optimized for the numerical (m/z)intensity pairs that account for most of the MS data. We tested MassComp on several MS data and show that it delivers on average a 46% reduction on the size of the numerical data, and up to 89%. These results correspond to an average improvement of more than 27% when compared to the general compressor gzip and of 40% when compared to the state-of-the-art numerical compressor FPC. When tested on entire files retrieved from the MassIVE repository, MassComp achieves on average a 59% size reduction. MassComp is written in C++ and freely available at https://github.com/iochoa/MassComp. Conclusions: The compression performance of MassComp demonstrates its potential to significantly reduce the footprint of MS data, and shows the benefits of designing specialized compression algorithms tailored to MS data. MassComp is an addition to the family of omics compression algorithms designed to lessen the storage burden and facilitate the exchange and dissemination of omics data.

show abstract

QVZ: lossy compression of quality values

Cited by 21 publications

References 0 publications

Crumble: reference free lossy compression of sequence quality values

Crumble: reference free lossy compression of sequence quality values

Denoising of Aligned Genomic Data

MassComp, a lossless compressor for mass spectrometry data

Contact Info

Product

Resources

About