CloudRS: An error correction algorithm of high-throughput sequencing data based on scalable framework

Chen, Chien-Chih; Chang, Yu-Jung; Chung, Wei-Chun; Lee, Der-Tsai; Ho, Jan-Ming

doi:10.1109/bigdata.2013.6691642

Cited by 22 publications

(17 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As of now, there are two fully distributed methods using the Map-Reduce (Hadoop) paradigm. [125,127] An interesting addition to the field is FADE [115], the FPGA error corrector which unleashes the massive parallelism available on FPGA devices to tackle the error correction.…”

Section: Single Threaded Vs Parallelmentioning

confidence: 99%

“…[103] use GPGPUs to generate the k-mer spectrum. A rare feature is the support for variable k-mers for grouping reads as in [125], where the corrector uses a wildcard based k-mer.…”

Section: K-mermentioning

confidence: 99%

“…Chung textitet al [127]aim to improve the efficiency and effectiveness of CloudRS [125] by introducing the read-message (RM) diagram trimmed using the Gradient-number Votes (GNV) scheme. They define the RM diagram of a read as the set of the records i represented by Arm left (i), Kernel(i), Arm right (i).…”

Section: Multiple Sequence Alignment Based (Msab)mentioning

confidence: 99%

“…In contrast, non-repetitive regions may not even participate in alignments, because of the uniqueness resulted from the Pacific Biosciences's high error rate. [125] makes use of a high frequency k-mer filter to avoid stacking reads from repetitive regions. Column "Rep" from table 3.1 contains the support for repetitive regions in the correctors.…”

Section: Repetitive Regionsmentioning

confidence: 99%

“…CloudRS [125] implements the ALLPATHS-LG's [126] error correction algorithm on top of Hadoop. The corrector create stacks of reads sharing a k-mer.…”

Section: Multiple Sequence Alignment Based (Msab)mentioning

confidence: 99%

See 4 more Smart Citations

Improved Error Correction of NGS Data

Alic¹

View full text Add to dashboard Cite

SummaryThe work done for this doctorate thesis focuses on error correction of Next Generation Sequencing (NGS) data in the context of High Performance Computing (HPC).Due to the reduction in sequencing cost, the increasing output of the sequencers and the advancements in the biological and medical sciences, the amount of NGS data has increased tremendously. Humans alone are not able to keep pace with this explosion of information, therefore computers must assist them to ease the handle of the deluge of information generated by the sequencing machines. Since NGS is no longer just a research topic (used in clinical routine to detect cancer mutations, for instance), requirements in performance and accuracy are more stringent. For sequencing to be useful outside research, the analysis software must work accurately and fast. This is where HPC comes into play. NGS processing tools should leverage the full potential of multi-core and even distributed computing, as those platforms are extensively available. Moreover, as the performance of the individual core has hit a barrier, current computing tendencies focus on adding more cores and explicitly split the computation to take advantage of them.This thesis starts with a deep analysis of all these problems in a general and comprehensive way (to reach out to a very wide audience), in the form of an exhaustive and objective review of the NGS error correction field. We dedicate a chapter to this topic to introduce the reader gradually and gently into the world of sequencing. It presents real problems and applications of NGS that demonstrate the impact this technology has on science. The review results in the following conclusions: the need of understanding of the specificities of NGS data samples (given the high variety of technologies and features) and the need of flexible, efficient and accurate tools for error correction as a preliminary step of any NGS postprocessing.As a result of the explosion of NGS data, we introduce MuffinInfo. It is a piece of software capable of extracting information from the raw data produced by the sequencer to help the user understand the data. MuffinInfo uses HTML5, therefore it runs in almost any software and hardware environment. It supports custom statistics to mould itself to specific requirements. MuffinInfo can reload the results of a run which are stored in JSON format for easier integration with third party applications. Finally, our application uses threads to perform the calculations, to load the data from the disk and to handle the UI.In continuation to our research and as a result of the single core performance limitation, we leverage the power of multi-core computers to develop a new error correction tool. The error correction of the NGS data is normally the first step of any analysis targeting NGS. As we conclude from the review performed within the frame of this thesis, many projects in different real-life applications have opted for this step before further analysis. In this sense, we propose MuffinEC, a multi-technology (Illu...

show abstract

Section: Single Threaded Vs Parallelmentioning

confidence: 99%

“…[103] use GPGPUs to generate the k-mer spectrum. A rare feature is the support for variable k-mers for grouping reads as in [125], where the corrector uses a wildcard based k-mer.…”

Section: K-mermentioning

confidence: 99%

Section: Multiple Sequence Alignment Based (Msab)mentioning

confidence: 99%

Section: Repetitive Regionsmentioning

confidence: 99%

“…CloudRS [125] implements the ALLPATHS-LG's [126] error correction algorithm on top of Hadoop. The corrector create stacks of reads sharing a k-mer.…”

Section: Multiple Sequence Alignment Based (Msab)mentioning

confidence: 99%

See 3 more Smart Citations

Improved Error Correction of NGS Data

Alic¹

View full text Add to dashboard Cite

show abstract

Objective review of de novo stand‐alone error correction methods for NGS data

Alic

Ruzafa

Dopazo

et al. 2016

WIREs Comput Mol Sci

View full text Add to dashboard Cite

The sequencing market has increased steadily over the last years, with different approaches to read DNA information, prone to different types of errors. Multiple studies demonstrated the impact of sequencing errors on different applications of Next Generation Sequencing (NGS), making error correction a fundamental initial step. Different methods in the literature use different approaches and fit different types of problems. We analysed a number of 50 methods divided into five main approaches (k-spectrum, suffix arrays, multiple sequence alignment, read clustering and probabilistic models). They are not published as a part of a suite (stand-alone) and target raw, unprocessed data without an existing reference genome (de Novo). These correctors handle one or more sequencing 1 Correspondence to: asalic@posgrado.upv.es 1 technologies using the same or different approaches. They face general challenges (sometimes with specific traits for specific technologies) such as repetitive regions, uncalled bases and ploidy. Even assessing their performance is a challenge in itself because of the approach taken by various authors, the unknown factor (de Novo) and the behaviour of the third party tools employed in the benchmarks. This work aims at helping the researcher in the field to advance the state-of-the-art, the educator to have a brief but comprehensive companion and the bioinformatician to choose the right tool for the right job.The Next Generation Sequencing (NGS) appeared in 2005 and since then its market has increased steadily, with various technologies being developed. The NGS has evolved faster than the the Moore's law in Computer Science, allowing us to sequence and assemble large genomes like the Loblolly Pine with 22 Gb 1 or the Norway Spruce with 20 Gb 2 for a reasonable cost in time and resources. However, there are many other species (e.g. the Amoeba Dubia with a 670 Gb estimated genome size 3 , 200x human genome's size) that are still challenging to assemble. The errors introduced by the sequencing process are one of the main reasons NGS data has to be corrected before any further use. Multiple studies have demonstrated the impact of sequencing errors on different applications of NGS, making error correction a fundamental initial step. [4][5][6][7] There are many error correction tools in the literature that cope with different technologies and error types. However, to our knowledge, there is no complete, objective review of the modern methods that could help researchers, educators and users at the same time. There are benchmarks summarizing a number of methods, but there is none extensively focusing on the implementation, features and the overall domain (including challenges). Our work synthesises 50 de Novo stand-alone error-correction software. The Supplementary Material includes the description of the approach used to search the literature along with the inclusion criteria.The article continues with the motivation (also containing a brief description of the sequencing technologies and various err...

show abstract