CoLoRd: Compressing long reads

Kokot, Marek; Gudyś, Adam; Li, H.; Deorowicz, Sebastian

doi:10.1101/2021.07.17.452767

Cited by 4 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in Section 3, all the experimented pipelines yield very similar performance on the original and quantized versions of each data set. These results complement and reinforce the aforementioned conclusion in [9], indicating that the effect of information loss caused by quality score quantization is not significant in practice. For the assembly polishing of a mock microbial community, setting all quality scores to the fixed value 10 results in a number of mismatches that is, on average over three independent runs, less than 1.2 % higher that that obtained with the original data.…”

Section: Introductionsupporting

confidence: 89%

“…Nanopore sequencing, however, is a much more recent technology and few specific data compressors suitable for nanopore data are available, developed by our group [6,7] and others [8,9]. Moreover, the lossy compression of quality scores for nanopore data has only been explored in [9], where the impact of quality score information loss is assessed for some downstream analyses. Specifically, in [9], it is shown that this information loss has non or little impact on the construction of consensus sequences with Racon [10] for long CHM13 reads, either for HiFi or for Nanopore data.…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…Moreover, the lossy compression of quality scores for nanopore data has only been explored in [9], where the impact of quality score information loss is assessed for some downstream analyses. Specifically, in [9], it is shown that this information loss has non or little impact on the construction of consensus sequences with Racon [10] for long CHM13 reads, either for HiFi or for Nanopore data. The impact is also shown to be small for variant calling accuracy with HiFi reads, evaluated with DeepVariant [11] on HG002 HiFi reads against the human reference genome GRCh38; variant calling for nanopore data is not evaluated.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Nanopore quality score resolution can be reduced with little effect on downstream analysis

Rivara-Espasandín

Balestrazzi

Álvarez

et al. 2022

Preprint

View full text Add to dashboard Cite

We investigate the effect of quality score information loss on downstream analysis from nanopore sequencing FASTQ files. We polished denovo assemblies for a mock microbial community and a human genome, and we called variants on a human genome. We repeated these experiments using various pipelines, under various coverage level scenarios, and various quality score quantizers. In all cases we found that the quantization of quality scores cause little difference on (or even improves) the results obtained with the original (non-quantized) data. This suggests that the precision that is currently used for nanopore quality scores is unnecessarily high, and motivates the use of lossy compression algorithms for this kind of data. Moreover, we show that even a non-specialized compressor, like gzip, yields large storage space savings after quantization of quality scores.

show abstract

Section: Introductionsupporting

confidence: 89%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Nanopore quality score resolution can be reduced with little effect on downstream analysis

Rivara-Espasandín

Balestrazzi

Álvarez

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For the analysis of Tuberculosis data, a newer version of the SPLASH pipeline was run, SPLASH2 (26), to generate the contingency tables. Then, the same optimization procedures were run to generate the optimized p-value bounds.…”

Section: E Tuberculosis Additional Plotsmentioning

confidence: 99%

OASIS: An interpretable, finite-sample valid alternative to Pearson’sX²for scientific discovery

Baharav¹,

Tse²,

Salzman

2023

Preprint

View full text Add to dashboard Cite

Contingency tables, data represented as counts matrices, are ubiquitous across quantitative research and data-science applications. Existing statistical tests are insufficient however, as none are simultaneously computationally efficient and statistically valid for a finite number of observations. In this work, motivated by a recent application in reference-genome-free inference (Chaung et al., 2022), we develop OASIS (Optimized Adaptive Statistic for Inferring Structure), a family of statistical tests for contingency tables. OASIS constructs a test-statistic which is linear in the normalized data matrix, providing closed form p-value bounds through classical concentration inequalities. In the process, OASIS provides a decomposition of the table, lending interpretability to its rejection of the null. We derive the asymptotic distribution of the OASIS test statistic, showing that these finite-sample bounds correctly characterize the p-value bound derived up to a variance term. Experiments on genomic sequencing data highlight the power and interpretability of OASIS. The same method based on OASIS significance calls detects SARS-CoV-2 and Mycobacterium Tuberculosis strains de novo, which cannot be achieved with current approaches. We demonstrate in simulations that OASIS is robust to overdispersion, a common feature in genomic data like single cell RNA-sequencing, where under accepted noise models OASIS still provides good control of the false discovery rate, while Pearson's X2 test consistently rejects the null. Additionally, we show on synthetic data that OASIS is more powerful than Pearson's X2 test in certain regimes, including for some important two group alternatives, which we corroborate with approximate power calculations.

show abstract

NanoSpring: reference-free lossless compression of nanopore sequencing reads using an approximate assembly approach

Meng

Chandak

Zhu

et al. 2021

Preprint

View full text Add to dashboard Cite

Motivation: The amount of data produced by genome sequencing experiments has been growing rapidly over the past several years, making compression important for efficient storage, transfer and analysis of the data. In recent years, nanopore sequencing technologies have seen increasing adoption since they are portable, real-time and provide long reads. However, there has been limited progress on compression of nanopore sequencing reads obtained in FASTQ files. Previous work ENANO focuses mostly on quality score compression and does not achieve significant gains for the compression of read sequences over general-purpose compressors. RENANO achieves significantly better compression for read sequences but is limited to aligned data with a reference available. Results: We present NanoSpring, a reference-free compressor for nanopore sequencing reads, relying on an approximate assembly approach. We evaluate NanoSpring on a variety of datasets including bacterial, metagenomic, plant, animal, and human whole genome data. For recently basecalled high quality nanopore datasets, NanoSpring achieves close to 3x improvement in compression over state-of-the-art reference-free compressors. The computational requirements of NanoSpring are practical, although it uses more time and memory during compression than previous tools to achieve the compression gains. Availability: NanoSpring is available on GitHub at https://github.com/qm2/NanoSpring.

show abstract

CoLoRd: Compressing long reads

Cited by 4 publications

References 33 publications

Nanopore quality score resolution can be reduced with little effect on downstream analysis

Nanopore quality score resolution can be reduced with little effect on downstream analysis

OASIS: An interpretable, finite-sample valid alternative to Pearson’sX²for scientific discovery

NanoSpring: reference-free lossless compression of nanopore sequencing reads using an approximate assembly approach

Contact Info

Product

Resources

About

CoLoRd: Compressing long reads

Cited by 4 publications

References 33 publications

Nanopore quality score resolution can be reduced with little effect on downstream analysis

Nanopore quality score resolution can be reduced with little effect on downstream analysis

OASIS: An interpretable, finite-sample valid alternative to Pearson’sX2for scientific discovery

NanoSpring: reference-free lossless compression of nanopore sequencing reads using an approximate assembly approach

Contact Info

Product

Resources

About

OASIS: An interpretable, finite-sample valid alternative to Pearson’sX²for scientific discovery