Shared data science infrastructure for genomics data

Abstract

Background Creating a scalable computational infrastructure to analyze the wealth of information contained in data repositories is difficult due to significant barriers in organizing, extracting and analyzing relevant data. Shared data science infrastructures like Boa g is needed to efficiently process and parse data contained in large data repositories. The main features of Boa g are inspired from existing languages for data intensive computing and can easily integrate data from biological data repositories. Results As a proof of concept, Boa for genomics, Boa g , has been implemented to analyze RefSeq’s 153,848 annotation (GFF) and assembly (FASTA) file metadata. Boa g provides a massive improvement from existing solutions like Python and MongoDB, by utilizing a domain-specific language that uses Hadoop infrastructure for a smaller storage footprint that scales well and requires fewer lines of code. We execute scripts through Boa g to answer questions about the genomes in RefSeq. We identify the largest and smallest genomes deposited, explore exon frequencies for assemblies after 2016, identify the most commonly used bacterial genome assembly program, and address how animal genome assemblies have improved since 2016. Boa g databases provide a significant reduction in required storage of the raw data and a significant speed up in its ability to query large datasets due to automated parallelization and distribution of Hadoop infrastructure during computations. Conclusions In order to keep pace with our ability to produce biological data, innovative methods are required. The Shared Data Science Infrastructure, Boa g , provides researchers a greater access to researchers to efficiently explore data in new ways. We demonstrate the potential of a the domain specific language Boa g using the RefSeq database to explore how deposited genome assemblies and annotations are changing over time. This is a small example of how Boa g could be used with large biological datasets.

Shifting Gears in Precision Oncology—Challenges and Opportunities of Integrative Data Analysis

“…Multiple sequencing techniques/platforms and the vast amount of information obtained using genomics also pose added challenges to data integration [ 123 ]. Early interactions between researchers and clinicians would be recommended to first of all select the best diagnostic test so that all relevant information are covered.…”

“…Early interactions between researchers and clinicians would be recommended to first of all select the best diagnostic test so that all relevant information are covered. Secondly, prioritizing relevant variants for a specific disease would allow for reduction of enormous amount of data to a reasonable volume [ 123 , 124 ]. This does not mean that researchers should only analyze the defined gene sets, for example.…”