Framework for Parallel Preprocessing of Microarray Data Using Hadoop

Abstract

Nowadays, microarray technology has become one of the popular ways to study gene expression and diagnosis of disease. National Center for Biology Information (NCBI) hosts public databases containing large volumes of biological data required to be preprocessed, since they carry high levels of noise and bias. Robust Multiarray Average (RMA) is one of the standard and popular methods that is utilized to preprocess the data and remove the noises. Most of the preprocessing algorithms are time-consuming and not able to handle a large number of datasets with thousands of experiments. Parallel processing can be used to address the above-mentioned issues. Hadoop is a well-known and ideal distributed file system framework that provides a parallel environment to run the experiment. In this research, for the first time, the capability of Hadoop and statistical power of R have been leveraged to parallelize the available preprocessing algorithm called RMA to efficiently process microarray data. The experiment has been run on cluster containing 5 nodes, while each node has 16 cores and 16 GB memory. It compares efficiency and the performance of parallelized RMA using Hadoop with parallelized RMA using affyPara package as well as sequential RMA. The result shows the speed-up rate of the proposed approach outperforms the sequential approach and affyPara approach.

Identification of key genes and functional enrichment analysis of liver fibrosis in nonalcoholic fatty liver disease through weighted gene co-expression network analysis

“…This microarray data provided a gene expression profile of the liver from 32 patients with advanced NAFLD (fibrosis stages 3–4) and 40 patients with mild NAFLD (fibrosis stages 0–1). The GSE49541 dataset underwent independent normalization using robust multiarray analysis (RMA) [ 14 ] at the NCBI, followed by log2 transformation and quantile normalization. To mitigate batch effects, ComBat was applied to the normalized combined dataset.…”

Identification and validation of hub genes for diabetic retinopathy

“…Then, microarray analyses and experiments were completed, following which, the Agilent custom algorithm was used to design the probe sets that were printed on the GPL18056 platform. The robust multiarray average algorithm ( Sahlabadi et al, 2018 ) was used to perform quartile data standardization of the downloaded data and background correction. We filtered the lack of corresponding gene symbols for the probes, and reserved the maximum values of the gene symbols using multiple probes.…”

Debunking the “junk”: Unraveling the role of lncRNA–miRNA–mRNA networks in fetal hemoglobin regulation

“…For lncRNA-based classification, datasets were selected according to the following criteria: (1) dataset having high HbF vs. normal HbF expression profiles ;(2) dataset with sample no.>3; and (3) only Affymetrix HG-U133 plus 2.0 arrays expression profiles were included in our study. The raw (.CEL) files were downloaded, quantile normalized, Log2transformed and background corrected by using Robust Multichip Average (RMA, Windows Version) (Sahlabadi et al 2018). Finally, we obtained probe ID-centric gene expression datasets.…”