Facing the genome data deluge

Abstract

Next-generation sequencing enables whole genome sequencing within a few hours at a minimum of cost. However, this technology imposes new challenges to computational genome analysis tasks in terms of efficiently processing an increasing amount of error-prone data.In this work, we focus on addressing these challenges for identifying Single Nucleotide Polymorphisms as one type of genetic variants in genome data. We propose the application of a column-store in-memory database for efficient data processing to profit from built-in compression and parallelization techniques and accessing data directly from main memory instead of slower disk space. We provide a statistical model that is sensitive to input data quality and utilizes knowledge from language population studies. Comparisons with state-of-the-art tools show that our approach outperforms traditional procedures on average by magnitudes of speed whilst requiring less administration efforts.