Comparative analysis of methods for detecting interacting loci

Abstract

BackgroundInteractions among genetic loci are believed to play an important role in disease risk. While many methods have been proposed for detecting such interactions, their relative performance remains largely unclear, mainly because different data sources, detection performance criteria, and experimental protocols were used in the papers introducing these methods and in subsequent studies. Moreover, there have been very few studies strictly focused on comparison of existing methods. Given the importance of detecting gene-gene and gene-environment interactions, a rigorous, comprehensive comparison of performance and limitations of available interaction detection methods is warranted.ResultsWe report a comparison of eight representative methods, of which seven were specifically designed to detect interactions among single nucleotide polymorphisms (SNPs), with the last a popular main-effect testing method used as a baseline for performance evaluation. The selected methods, multifactor dimensionality reduction (MDR), full interaction model (FIM), information gain (IG), Bayesian epistasis association mapping (BEAM), SNP harvester (SH), maximum entropy conditional probability modeling (MECPM), logistic regression with an interaction term (LRIT), and logistic regression (LR) were compared on a large number of simulated data sets, each, consistent with complex disease models, embedding multiple sets of interacting SNPs, under different interaction models. The assessment criteria included several relevant detection power measures, family-wise type I error rate, and computational complexity. There are several important results from this study. First, while some SNPs in interactions with strong effects are successfully detected, most of the methods miss many interacting SNPs at an acceptable rate of false positives. In this study, the best-performing method was MECPM. Second, the statistical significance assessment criteria, used by some of the methods to control the type I error rate, are quite conservative, thereby limiting their power and making it difficult to fairly compare them. Third, as expected, power varies for different models and as a function of penetrance, minor allele frequency, linkage disequilibrium and marginal effects. Fourth, the analytical relationships between power and these factors are derived, aiding in the interpretation of the study results. Fifth, for these methods the magnitude of the main effect influences the power of the tests. Sixth, most methods can detect some ground-truth SNPs but have modest power to detect the whole set of interacting SNPs.ConclusionThis comparison study provides new insights into the strengths and limitations of current methods for detecting interacting loci. This study, along with freely available simulation tools we provide, should help support development of improved methods. The simulation tools are available at: http://code.google.com/p/simulation-tool-bmc-ms9169818735220977/downloads/list.

Genetic studies of complex human diseases: Characterizing SNP-disease associations using Bayesian networks

“…Our definition of power prohibits any false positives and any false negatives and reflects the ability to precisely detect whole interactions [35]. Although we consider both type I error and type II error and the performance comparison is fair for each method in Figure 2, this type of definition of power seems stringent.…”

KNN-MDR: a learning approach for improving interactions mapping performances in genome wide association studies

“…Nevertheless, since experimental data has been used, genotyping errors might be present. Presence of LD in the data was checked using simple association tests between consecutive markers (data not shown).Since many different individuals are needed in the simulations, we used a trick similar to [42] to generate new individuals based on the few available genotypes: each individual genotype was chopped into 10 SNP windows, leading to 200 windows with (maximum) 197 different 10 loci genotypes. Each simulated individual genotype was then built by randomly sampling a genotype for each window and concatenating the 200 genotypes into a new complete genotype with 2000 markers.…”

Comparative Analysis of Methods for Identifying Recurrent Copy Number Alterations in Cancer

“…To quantitatively evaluate the performance of the methods, we test four commonly used criteria [13], [28], [29], [30] based on a large number of simulation datasets. The criteria are described in detail below.…”

“…The purpose of assessing type I error rate is to investigate the meaning of the significance levels resulted from the statistical methods for detecting recurrent CNAs [13], [30]. If type I error rate is too conservative or too aggressive, the intended meaning of the p -values (or q -values) would be reduced or lost, and it doesn’t agree with the real false positive rate in results.…”

“…For a convenient assessment, we use the middle marker of the recurrent CNA unit to determine whether the unit is declared, i.e., if the middle marker is detected, then we suppose that the unit is successfully detected, otherwise, it is not. Accordingly, the CNA unit-based detection power of a method can be calculated by [30] where is the total number of ground truth CNA units in each simulated dataset, and indicates the number of ground truth CNA units that are declared significant in the i -th dataset.…”