A Comprehensive Evaluation of Methods for Mendelian Randomization Using Realistic Simulations and an Analysis of 38 Biomarkers for Risk of Type-2 Diabetes

Abstract

Mendelian randomization (MR) has provided major opportunities for understanding the causal relationship among complex traits. Previous studies have often evaluated MR methods based on simulations that do not adequately reflect the data-generating mechanism in GWAS and thus there is often discrepancies in performance of MR methods in simulation studies and in real datasets. We use a simulation framework that generates data on full GWAS for two traits under realistic model for effect-size distribution coherent with heritability, co-heritability and polygenicity typically observed for complex traits. We select instruments based on SNPs which reach genome-wide significance in the underlying study of one of the traits the causal effect of which to be tested on the other. Results show that the weighted mode method and MRMix are the only two methods which maintain correct type I error rate in a diverse set of scenarios.Between the two methods, MRMix tends to be more powerful for larger GWAS while the opposite being true for smaller sample sizes. Among other methods, random effect based IVW analysis, MR analyses based on robust loss function (MR-Robust) and robust profile-scores (MR-RAPS) tend to perform best in terms of maintaining low MSE when the InSIDE assumption is satisfied. However, when the InSIDE assumption is violated, all methods except weighted mode and MRMix can produce large bias and correspondingly large mean squared errors (MSE). In conclusion, the results show that relative performance of different methods depends heavily on sample sizes of underlying GWAS, proportion of valid instruments and validity of the InSIDE assumption.

Bayesian variable selection with a pleiotropic loss function in Mendelian randomization

“…Although it is perhaps counterintuitive that some methods may perform better with smaller sample sizes, similar observations have been made elsewhere (e.g. in some of the simulations in Qi and Chatterjee, 2019).…”

“…For an extensive comparison of pleiotropy-robust Mendelian randomization algorithms using summary data, we refer the reader to two recent review papers (Slob and Burgess, 2019;Qi and Chatterjee, 2019).…”

“…Similar to JAM-MR, this approach relies on model averaging, but instead of our algorithm's stochastic search it uses an exhaustive search over possible models, and is therefore only applicable for small numbers of genetic variants. We have also not considered the mixture modelling method of Qi and Chatterjee (2018); this method was shown to perform well in terms of Type I error calibration in a recent study (Qi and Chatterjee, 2019), but in order to function efficiently it also requires access to summary statistics for additional genetic variants throughout the genome that are not associated with the risk factor. We did not generate such variants in our simulations, since it would have dramatically increased their computational cost.…”

“…Traditional approaches include MR-Egger regression (Bowden et al, 2015) and median estimation , while several algorithms for pleiotropy-robust Mendelian randomization have been developed recently (Hartwig et al, 2017;Kang et al, 2016;Rees et al, 2019;Burgess et al, 2018;Verbanck et al, 2018;Zhao et al, 2018;Qi and Chatterjee, 2018;Bowden et al, 2018;Burgess et al, 2019). Recent reviews and comparison of the various methods can be found in Slob and Burgess (2019) and Qi and Chatterjee (2019).…”

Kidney Function and Blood Pressure: A Bi-directional Mendelian Randomisation Study

“…The second being the assumption of the lack of horizontal pleiotropy of the genetic instruments variants, which is usually difficult to assess and verify 18 . To address this issue, we analyzed the data using multiple Mendelian randomisation methods and prioritized the method that are known to be most robust to the presence of horizontal pleiotropy 19 .…”