Small studies are more heterogeneous than large ones: a meta-meta-analysis

IntHout, Joanna; Ioannidis, John P. A.; Borm, George F.; Goeman, Jelle J.

doi:10.1016/j.jclinepi.2015.03.017

Cited by 224 publications

(165 citation statements)

References 42 publications

Supporting

Mentioning

159

Contrasting

Order By: Relevance

“…The present analyses, however, suggest this decline effect may be a result of low power and selective reporting: Cues that have rarely been reported will demonstrate wide variation in effect sizes, some of which will appear to be quite large because low-powered studies occasionally produce highly overestimated significant effects, which are more likely to be published than the nonsignificant effects that would attenuate the overestimates (IntHout et al, 2015). In contrast, cues that have been studied more extensively should better approximate the true effect.…”

Section: Trouble In the Land Of Toyscontrasting

confidence: 54%

Lessons from Pinocchio: Cues to deception may be highly exaggerated

Luke¹

2018

Preprint

View full text Add to dashboard Cite

Deception researchers widely acknowledge that cues to deception -observable behaviors that may differ between truthful and deceptive messagestend to be weak. Nevertheless, several deception cues have been reported with unusually large effect sizes, and some researchers have advocated the use of such cues as tools for detecting deceit and assessing credibility in practical contexts. Examining data from empirical deception cue research and using a series of Monte Carlo simulations, I demonstrate that many estimated effect sizes of deception cues may be greatly inflated by publication bias, small numbers of estimates, and low power. Indeed, simulations indicate the informational value of the present deception literature is quite low, such that it is not possible to determine whether any given effect is real or a false positive. I warn against the hazards of relying on potentially illusory cues to deception and offer some recommendations for improving the state of the science of deception. A preprint of this document is available at https://osf.io/xt8fq/."That Marionette," continued the Talking Cricket, "is a rascal of the worst kind." (Collodi, 1883) In an effort to temper our hopes of catching lies with unerring accuracy, deception researchers often say there is no "Pinocchio's nose" (e.g., Frank, Menasco, & O'Sullivan, 2008;Hartwig & Bond, 2011;Vrij, 2006Vrij, , 2004. That is, there is no behavior that perfectly discriminates between truthful and deceptive messages. For all the talk about Pinocchio's nose, there are other more urgent lessons from the story of Pinocchio pertinent to deception research.In his adventures, Pinocchio causes mischief and mayhem, and among his many foibles is the ease with which he is tempted to do what is easy and immediately satisfying but ultimately dangerous, rather than what is difficult and responsible. * timothy.luke@psy.gu.se. For their helpful comments, suggestions, and inspiration, I am grateful to Fabi Alceste, Karl Ask, Charlie Bond, Will Crozier, Bella DePaulo, Emelie Ernberg, Pär Anders Granhag, Maria Hartwig, Lorraine Hope, Emily Joseph, Louise Jupe, Erik Mac Giolla, Patty Sanchez, Rebecca Willén, and three anonymous reviewers, as well as the many supportive others who provided encouragement and tolerated my ravings and rantings.

show abstract

Section: Trouble In the Land Of Toyscontrasting

confidence: 54%

Lessons from Pinocchio: Cues to deception may be highly exaggerated

Luke¹

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…This, however, is an argument in favour of a meta-analysis strategy, since statistical power is greater and more subtle associations can be identified when results from multiple small studies can be combined in meta-analysis. Although small studies tend to be impacted by publication bias to a greater degree than larger studies [15, 43], we did not find evidence of that in this analysis. Selection bias may be present across all of the selected publications because HACT is often used as a supplementary treatment in relatively stable asthma, whereas patients with severe or difficult to control asthma may be excluded from such investigations.…”

Section: Discussioncontrasting

confidence: 75%

High-altitude alpine therapy and lung function in asthma: systematic review and meta-analysis

et al. 2016

View full text Add to dashboard Cite

We used meta-analysis to measure the effect of high-altitude climate therapy (HACT) on lung function outcomes in asthma, and systematically searched PubMed, Embase and www.elibrary.ru for publications appearing from 1970 to mid-2015. We included studies carried out with children or adults with an exposure of up to 12 weeks at an altitude of ≥1500 m above sea level. Changes in forced expiratory volume in 1 s (FEV1), FEV1/vital capacity ratio or peak expiratory flow rate as the HACT intervention outcomes were analysed. We included data for 907 participants (age range 4–58 years) from 21 studies, altogether including 28 substrata based on asthma type or severity. Only three of 21 included studies had high quality, whereas 93% of substudies reported lung function improvement with an overall pooled standardised mean difference (SMD) of 0.53 (95% CI 0.43–0.62). The measured effect of HACT was greater in adults (SMD 0.75, 95% CI 0.63–0.88, n=14) than in children (SMD 0.24, 95% CI 0.09–0.38, n=14). Studies at altitude >2000 m above sea level yielded the same effect as those at lower altitude. Based on a cut-point of a 0.50 change in SMD to define a meaningful clinical difference, HACT appears to have efficacy as an intervention. This extent of benefit appears to be limited to adults with asthma.

show abstract

“…This subcategory is not used as standard in the assessment of bias in Cochrane Reviews, despite the increasing volume of research available suggesting that small studies of fewer than 100 participants per arm (Moore 2010; Nüesch 2010) are at increased risk of succumbing to the random effects in estimating both direction and magnitude of treatment effects (Moore 1998; Turner 2013) due to greater heterogeneity within and between small studies (IntHout 2015). …”

Section: Discussionmentioning

confidence: 99%

Physical activity and exercise for chronic pain in adults: an overview of Cochrane Reviews

Geneen

Moore²,

Clarke

et al. 2017

Cochrane Database of Systematic Reviews

546

274

View full text Add to dashboard Cite

BackgroundChronic pain is defined as pain lasting beyond normal tissue healing time, generally taken to be 12 weeks. It contributes to disability, anxiety, depression, sleep disturbances, poor quality of life, and healthcare costs. Chronic pain has a weighted mean prevalence in adults of 20%.For many years, the treatment choice for chronic pain included recommendations for rest and inactivity. However, exercise may have specific benefits in reducing the severity of chronic pain, as well as more general benefits associated with improved overall physical and mental health, and physical functioning.Physical activity and exercise programmes are increasingly being promoted and offered in various healthcare systems, and for a variety of chronic pain conditions. It is therefore important at this stage to establish the efficacy and safety of these programmes, and furthermore to address the critical factors that determine their success or failure.ObjectivesTo provide an overview of Cochrane Reviews of adults with chronic pain to determine (1) the effectiveness of different physical activity and exercise interventions in reducing pain severity and its impact on function, quality of life, and healthcare use; and (2) the evidence for any adverse effects or harm associated with physical activity and exercise interventions.MethodsWe searched theCochrane Database of Systematic Reviews (CDSR) on the Cochrane Library (CDSR 2016, Issue 1) for systematic reviews of randomised controlled trials (RCTs), after which we tracked any included reviews for updates, and tracked protocols in case of full review publication until an arbitrary cut-off date of 21 March 2016 (CDSR 2016, Issue 3). We assessed the methodological quality of the reviews using the AMSTAR tool, and also planned to analyse data for each painful condition based on quality of the evidence.We extracted data for (1) self-reported pain severity, (2) physical function (objectively or subjectively measured), (3) psychological function, (4) quality of life, (5) adherence to the prescribed intervention, (6) healthcare use/attendance, (7) adverse events, and (8) death.Due to the limited data available, we were unable to directly compare and analyse interventions, and have instead reported the evidence qualitatively.Main resultsWe included 21 reviews with 381 included studies and 37,143 participants. Of these, 264 studies (19,642 participants) examined exercise versus no exercise/minimal intervention in adults with chronic pain and were used in the qualitative analysis.Pain conditions included rheumatoid arthritis, osteoarthritis, fibromyalgia, low back pain, intermittent claudication, dysmenorrhoea, mechanical neck disorder, spinal cord injury, postpolio syndrome, and patellofemoral pain. None of the reviews assessed 'chronic pain' or 'chronic widespread pain' as a general term or specific condition. Interventions included aerobic, strength, flexibility, range of motion, and core or balance training programmes, as well as yoga, Pilates, and tai chi.Reviews were well performed an...

show abstract

Small studies are more heterogeneous than large ones: a meta-meta-analysis

Abstract: Heterogeneity between small studies is larger than between larger studies. The large imprecision with which τ is estimated in a typical small-studies' meta-analysis is another reason for concern, and sensitivity analyses are recommended.

Cited by 224 publications

References 42 publications

Lessons from Pinocchio: Cues to deception may be highly exaggerated

Lessons from Pinocchio: Cues to deception may be highly exaggerated

High-altitude alpine therapy and lung function in asthma: systematic review and meta-analysis

Physical activity and exercise for chronic pain in adults: an overview of Cochrane Reviews

Contact Info

Product

Resources

About