HTSFinder: Powerful Pipeline of DNA Signature Discovery by Parallel and Distributed Computing

Abstract

Comprehensive effort for low-cost sequencing in the past few years has led to the growth of complete genome databases. In parallel with this effort, a strong need, fast and cost-effective methods and applications have been developed to accelerate sequence analysis. Identification is the very first step of this task. Due to the difficulties, high costs, and computational challenges of alignment-based approaches, an alternative universal identification method is highly required. Like an alignment-free approach, DNA signatures have provided new opportunities for the rapid identification of species. In this paper, we present an effective pipeline HTSFinder (high-throughput signature finder) with a corresponding k-mer generator GkmerG (genome k-mers generator). Using this pipeline, we determine the frequency of k-mers from the available complete genome databases for the detection of extensive DNA signatures in a reasonably short time. Our application can detect both unique and common signatures in the arbitrarily selected target and nontarget databases. Hadoop and MapReduce as parallel and distributed computing tools with commodity hardware are used in this pipeline. This approach brings the power of high-performance computing into the ordinary desktop personal computers for discovering DNA signatures in large databases such as bacterial genome. A considerable number of detected unique and common DNA signatures of the target database bring the opportunities to improve the identification process not only for polymerase chain reaction and microarray assays but also for more complex scenarios such as metagenomics and next-generation sequencing analysis.

Mining Public Metagenomes for Environmental Surveillance of Parasites: A Proof of Principle

“…To increase specificity and to alleviate computing limitations, metagenomes may be queried using signature sequences, which are specific for the taxon of interest (inclusion ancestor) and are not shared with closely related species (exclusion ancestor). Selection of signature sequences can be achieved by tools such as the high-throughput signature finder (HTSFinder) ( Karimi and Hajdu, 2016 ) and Neptune ( Marinier et al, 2017 ). However, these bioinformatics tools have been developed to identify genomic variation in bacteria.…”

The emergence of new trends in clinical laboratory diagnosis

“…The advantage of using this technique is its ability to screen for all identified as well as unidentified pathogens, which helps prevent the occurrence of diseases caused by unknown pathogens as well. 19 , 20 …”

Optimization of hadoop cluster foranalyzing large-scale sequence data inbioinformatics

“…DNA signature is a unique short fragment (k-mer) of DNA, which is specific for a species that is selected from a target genome database. DNA signature can be obtained for every individual species in the genome databases by screening and counting the frequency of k-mers through the entire database [10].…”

“…• Number of processed blocks are odd (1,3,5,7,9,11,13) • Number of processed blocks are even (2,4,6,8,10,12,14) Figure 8: CPU usage of N2…”

“…Each k-mer that appears once in a genome database is a unique DNA signature for the related sequence containing that k-mer. Since comparing k-mers frequencies are computationally easier than sequence alignment, the method can also be used as a first stage analysis before an alignment [10].…”