A systematic evaluation of 41 DNA methylation predictors across 101 data preprocessing and normalization strategies highlights considerable variation in algorithm performance

Abstract

Background DNA methylation (DNAm) based predictors hold great promise to serve as clinical tools for health interventions and disease management. While these algorithms often have high prediction accuracy and are associated with many disease-related phenotypes, the reliability of their performance remains to be determined. We therefore conducted a systematic evaluation across 101 different data processing strategies that preprocess and normalize DNAm data and assessed how each analytical strategy affects the reliability and prediction accuracy of 41 DNAm-based predictors. Results Our analyses were conducted in a large EPIC DNAm sample of the Jackson Heart Study (N=2,053) that included 146 pairs of technical replicate samples. By estimating the average absolute agreement between replicate pairs, we show that 32 out of 41 predictors (78%) demonstrate excellent test-retest reliability when appropriate data processing and normalization steps are implemented. Across all pairs of predictors, we find a moderate correlation in performance across analytical strategies (mean rho=0.40, SD=0.27), highlighting significant heterogeneity in performance across algorithms within a choice of an analytical pipeline. (Un)successful removal of technical variation furthermore significantly impacts downstream phenotypic association analysis, such as all-cause mortality risk associations. Conclusions We show that DNAm-based algorithms are sensitive to technical variation. The right choice of data processing and normalization pipeline is important to achieve reproducible estimates and improve prediction accuracy in downstream phenotypic association analyses. For each of the 41 DNAm predictors, we report its test-retest reliability and provide the best performing analytical strategy as a guideline for the research community. As DNAm-based predictors become more and more widely used, both for research purposes as well as for clinic applications, our work helps improve their performance and standardize their implementation.

Inflammation and Epigenetic Aging Are Largely Independent Markers of Biological Aging and Mortality

“…Another limitation is measurement error. Although the reliability of AgeAccel and the inflammatory markers was not assessed in this study, previous research indicates that epigenetic age variables have high reliability ( 44 ) while inflammatory marker variables are measured with more error (ICCs ~ 0.6) ( 10 ). Such measurement error was likely nondifferential and would likely have introduced bias towards the null, leading to an underestimation of the association between inflammation and epigenetic aging.…”

R methylCIPHER: A Methylation Clock Investigational Package for Hypothesis-Driven Evaluation & Research

“…Consequently, there may exist particular sets of CpGs which are essential to the function of for tissue specific aging clocks [49]. However, our results suggest that the majority of the CpGs have significant redundancy even for use in a tissue specific age predictor.…”

“…Preprocessing and normalization of methylation data is typically performed within R using the minfi [46], wateRmelon [47], or SeSAMe [48] packages, however methylCIPHER functions regardless of normalization protocol. For more details regarding the effects of choice of normalization, refer to Ori et al [49]. The user must simply have an object of matrix or data frame class, with named columns corresponding to the Illumina CpG names, and cells containing methylation beta ratios between 0 and 1.…”

Epigenetic Clocks for Mice Based on Age-Associated Regions That are Conserved Between Mouse Strains and Human

“…BeadChip technology makes large signatures easily applicable since all relevant CpGs are addressed in each sample. However, adaptation and integration of different microarray datasets remains a major hurdle and age-predictors may become outdated if a BeadChip release is discontinued (Ori et al, 2021). It may therefore be advantageous to rather focus on individual CpGs by targeted methods, such as pyrosequencing, digital droplet PCR or barcoded amplicon sequencing (Wagner, 2022).…”