When power analyses based on pilot data are biased: Inaccurate effect size estimators and follow-up bias

Albers, Casper J.; Lakens, Daniël

doi:10.1016/j.jesp.2017.09.004

Cited by 273 publications

(216 citation statements)

References 28 publications

Supporting

Mentioning

192

Contrasting

Order By: Relevance

“…As a corollary, the effect‐size estimates obtained in our exploratory study might not provide a reliable basis for calculating the sample size of our confirmatory study (cf. Albers & Lakens, ). This implies that our confirmatory study might not have been adequately powered to detect, for example, the correlation between cWCST and PEBT performance.…”

Section: Discussionmentioning

confidence: 99%

Cognitive Flexibility and Pro–Environmental Behaviour: A Multimethod Approach

Lange

Dewitte

2019

Eur J Pers

View full text Add to dashboard Cite

While cognitive flexibility has been hypothesized to relate to adaptive, long‐term‐oriented behaviour, empirical support for such a relationship is scarce. In the present article, we examine the role of cognitive flexibility as a correlate of long‐term‐oriented behaviour in the domain of environmental conservation. In a first study (N = 143), we explored potential associations between multiple self‐report scales and performance‐based measures of cognitive flexibility and pro‐environmental behaviour. Based on its results, we conducted a pre‐registered confirmatory study (N = 264) focusing on the most promising correlations found in our exploratory study. This study generated conclusive support for a positive medium‐sized correlation between cognitive flexibility and pro‐environmental behaviour. Exploratory hierarchical regression analyses further revealed that cognitive flexibility accounted for a small portion of variance in pro‐environmental behaviour above and beyond the Big Five. While appearing robust on the level of self‐reports, the relationship between cognitive flexibility and pro‐environmental behaviour did not reliably extend to the level of performance tasks, possibly because of their relatively specific scope. These results illustrate the contribution of cognitive flexibility to understanding individual differences in long‐term‐oriented, pro‐environmental behaviour. © 2019 European Association of Personality Psychology

show abstract

Section: Discussionmentioning

confidence: 99%

Cognitive Flexibility and Pro–Environmental Behaviour: A Multimethod Approach

Lange

Dewitte

2019

Eur J Pers

View full text Add to dashboard Cite

show abstract

“…During the entire analysis, ω 2 was preferred as the effect size for the factorial and repeated-measures ANOVAs over partial eta square, which has been reported to be an upwardly biased estimate of the true population effect size. 40,41 The degrees of freedom were corrected using either the Greenhouse-Geisser or Huynh-Feldt estimates based on Mauchly's test of sphericity and the Greenhouse-Geisser epsilon (ε). 41 3 | RESULTS…”

Section: Discussionmentioning

confidence: 99%

Perceiving speed—The color can matter

Mentzel

Schücker

Hagemann

et al. 2019

Color Research & Application

View full text Add to dashboard Cite

This study focuses on examining color effects of perceived speed in an individual noncompetitive sport context, treadmill running. In addition, the presence of individual trends for color effects are inspected by observing the connection of color effects in a sport‐specific video rating task with those found in an individual color association task. For this, 32 participants rated the perceived running speed of 48 videos depicting runners on a treadmill at seven different speed settings. Furthermore, participants rated a range of additional sport‐specific performance parameters. The runners in the video were shown wearing either a red, blue, or gray jersey, gray being used to strengthen the cover story. As a secondary task, the participants performed a modified Stroop task to assess implicit color associations. The results showed a significant color effect for speed; runners depicted in red were perceived as running at higher speeds compared to blue. No significant color effects were found for the other sport‐specific parameters. Finally, there was no significant covariate effect of the modified Stroop task for the speed perception color effect. These findings indicate that, in situations in which speed must be judged, red might be perceived as going faster.

show abstract

“…207 The targeted effect size should be biologically plausible, and based on the degree of benefit that patients, consumers, or practitioners would consider sufficiently beneficial to be worthwhile, given other considerations such as cost, inconvenience, side effects, and potential for serious adverse effects 213 (Table 5). Effect sizes should not be based on estimates from the literature as a whole, which almost always leads to biased estimates, 214 inasmuch as the published literature contains a biased selection of all empirical research that is performed. Meta-analytic evaluations of the literature have consistently found that studies yielding null or mixed results are substantially less likely to appear in the published scientific record 215,216 than those studies with strong statistical support for the tested hypothesis.…”

Section: Sample and Effect Sizementioning

confidence: 99%

“…Use best current meta-analytic approaches to assess bias in the literature, and to correct for bias in reported effect sizes 214. Base sample sizes on the smallest effect size of interest, in the context of existing options and consumer burden.…”

mentioning

confidence: 99%

Improving natural product research translation: From source to clinical trial

et al. 2019

View full text Add to dashboard Cite

While great interest in health effects of natural product (NP) including dietary supplements and foods persists, promising preclinical NP research is not consistently translating into actionable clinical trial (CT) outcomes. Generally considered the gold standard for assessing safety and efficacy, CTs, especially phase III CTs, are costly and require rigorous planning to optimize the value of the information obtained. More effective bridging from NP research to CT was the goal of a Septe mber, 2018 trans disci plina ry workshop. Participants emphasized that replicability and likelihood of successful translation depend on rigor in experimental design, interpretation, and reporting across the continuum of NP research. Discussions spanned good practices for NP characterization and quality control; use and interpretation of models (computational through in vivo) with strong clinical predictive validity; controls for experimental artefacts, especially for in vitro interrogation of bioactivity and mechanisms of action; rigorous assessment and interpretation of prior research; transparency in all reporting; and prioritization of research questions. Natural product clinical trials prioritized based on rigorous, convergent supporting data and current public health needs are most likely to be informative and ultimately affect public health. Thoughtful, coordinated implementation of these practices should enhance the knowledge gained from future NP research. K E Y W O R D S clinical predictive validity, dietary supplements, model systems, rigor and replicability, value of information | 43 SORKIN et al.still provide invaluable guidance, the results of such trials are often met with concerns that different design choiceswhether of product, dose, timing, participant eligibility criteria, outcomes, etc.-might have yielded a substantially different result. Each interventional trial represents a substantial investment, with well-powered Phase III CTs typically consuming more than $10 million and many person-years. 9,10 Additionally, as time and funds are limited, and participant pools may also be constrained, a decision to invest in a CT often precludes pursuing other research questions, 11 which may provide more useful public health-relevant knowledge than an RCT with an insufficient evidence base.

show abstract

When power analyses based on pilot data are biased: Inaccurate effect size estimators and follow-up bias

Cited by 273 publications

References 28 publications

Cognitive Flexibility and Pro–Environmental Behaviour: A Multimethod Approach

Cognitive Flexibility and Pro–Environmental Behaviour: A Multimethod Approach

Perceiving speed—The color can matter

Improving natural product research translation: From source to clinical trial

Contact Info

Product

Resources

About