Power analysis for generalized linear mixed models in ecology and evolution

Abstract

‘Will my study answer my research question?’ is the most fundamental question a researcher can ask when designing a study, yet when phrased in statistical terms – ‘What is the power of my study?’ or ‘How precise will my parameter estimate be?’ – few researchers in ecology and evolution (EE) try to answer it, despite the detrimental consequences of performing under- or over-powered research. We suggest that this reluctance is due in large part to the unsuitability of simple methods of power analysis (broadly defined as any attempt to quantify prospectively the ‘informativeness’ of a study) for the complex models commonly used in EE research. With the aim of encouraging the use of power analysis, we present simulation from generalized linear mixed models (GLMMs) as a flexible and accessible approach to power analysis that can account for random effects, overdispersion and diverse response distributions.We illustrate the benefits of simulation-based power analysis in two research scenarios: estimating the precision of a survey to estimate tick burdens on grouse chicks and estimating the power of a trial to compare the efficacy of insecticide-treated nets in malaria mosquito control. We provide a freely available R function, sim.glmm, for simulating from GLMMs.Analysis of simulated data revealed that the effects of accounting for realistic levels of random effects and overdispersion on power and precision estimates were substantial, with correspondingly severe implications for study design in the form of up to fivefold increases in sampling effort. We also show the utility of simulations for identifying scenarios where GLMM-fitting methods can perform poorly.These results illustrate the inadequacy of standard analytical power analysis methods and the flexibility of simulation-based power analysis for GLMMs. The wider use of these methods should contribute to improving the quality of study design in EE.

How social opinion influences syntactic processing—An investigation using virtual reality

“…Statistical power was calculated using simulated priming data produced by the sim.glmm package [39] in R [40]. For our simulated data set we assumed 15 repetitions per prime type (active, passive, baseline; see below).…”

The Relationship Among Breastfeeding, Postpartum Depression, and Postpartum Weight in Mexican American Women

“…For tests using the dichotomous EPDS depression risk measure, we drew on our observed data and estimated fixed and random effect parameter values to derive power estimates using a simulation-based approach in R as described by Johnson, Barry, Ferguson, and Müller (2015). Effect size ( f 2 ) estimates along with sample size and alpha values were entered into G*Power to arrive at estimates of the observed power for each test involving continuous outcomes.…”

Optimising survey effort to monitor environmental variables: A case study using New Zealand kiwifruit orchards

“…Statistical power is one of a range of possible criteria used to inform the indicator selection and prioritisation process (Herzog et al, 2012;OECD, 2001;Sommerville et al, 2011). With recent advances in computational power and software, there is no excuse not to deploy and integrate power analyses into design and decision making to reduce the risk of potentially dreadful loss of efficiency, direct costs and opportunity costs (Field et al, 2007;Green and MacLeod, 2015;Johnson et al, 2015;Raffaelli and Moller, 2000). Knowing when there is enough evidence to change farming practice would also be very valuable, as experimentation can be expensive and can have several indirect costs (including the opportunity costs of not learning fast enough).…”

“…This is often technically and logistically challenging, especially when treatment impacts vary from site to site (Raudenbush and Liu, 2000) and/or resources are limited (Lindenmayer and Likens, 2010;Magurran et al, 2010). Careful planning and executing of sophisticated analyses of monitoring data are recommended for identifying: cost-effective and robust designs (Field et al, 2007;Geijzendorffer et al, 2015;Johnson et al, 2015); monitoring efforts that have no realistic chance of detecting relevant changes, and options for improving them (Collen and Nicholson, 2014;Field et al, 2007;Legg and Nagy, 2006); and trade-offs between spatial and temporal replication (Rhodes and Jonz en, 2011;Urquhart et al, 1998).…”

“…This requires a sufficient sample size in relation to the variability inherent in the system (Sims et al, 2007); setting an ecologically appropriate level of power as a target; and a flexible design that allows for learning and improvement in the future (Field et al, 2007). A large survey might be overpowered and more expensive than necessary (Johnson et al, 2015). However, it is more likely that the level of survey effort is restricted by the resources available and the monitoring has insufficient power, with nonsignificant results erroneously interpreted as showing no trend or change when in fact there is real change.…”