Double-counting of populations in evidence synthesis in public health: a call for awareness and future methodological development

Hussein, Humaira; Nevill, Clareece R; Meffen, Anna; Abrams, Keith R.; Bujkiewicz, Sylwia; Sutton, Alex; Gray, Laura J.

doi:10.1186/s12889-022-14213-6

Cited by 10 publications

(8 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…75,76 Nonindependence commonly results from including multiple effect estimates with overlapping samples from the same study (i.e., double counting). 75 Multiple estimates can be found in a single report or across multiple reports of the same study population or sample. 77,78 To avoid double counting participants, it is recommended that reviewers select the most relevant, independent estimate(s) from each unique study.…”

Section: Discussionmentioning

confidence: 99%

“…78 The approach to selecting estimates might introduce bias, such as if estimates are selected in a way that confirms reviewers' perceptions or beliefs. 75,76 We therefore followed the decision rules described in Appendix 5. The estimates from each unique study and corresponding reports selected for our primary meta-analyses are described in Appendices 9 and 10.…”

Section: Discussionmentioning

confidence: 99%

“…The effect estimates included in a meta‐analysis are typically assumed to be independent, as pooling together nonindependent estimates can overestimate their precision when synthesized 75,76 . Nonindependence commonly results from including multiple effect estimates with overlapping samples from the same study (i.e., double counting) 75 . Multiple estimates can be found in a single report or across multiple reports of the same study population or sample 77,78 .…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Causal Assessment of Income Inequality on Self‐Rated Health and All‐Cause Mortality: A Systematic Review and Meta‐Analysis

SHIMONOVICH,

CAMPBELL,

THOMSON

et al. 2024

Milbank Quarterly

View full text Add to dashboard Cite

Policy Points Income is thought to impact a broad range of health outcomes. However, whether income inequality (how unequal the distribution of income is in a population) has an additional impact on health is extensively debated. Studies that use multilevel data, which have recently increased in popularity, are necessary to separate the contextual effects of income inequality on health from the effects of individual income on health. Our systematic review found only small associations between income inequality and poor self‐rated health and all‐cause mortality. The available evidence does not suggest causality, although it remains methodologically flawed and limited, with very few studies using natural experimental approaches or examining income inequality at the national level. ContextWhether income inequality has a direct effect on health or is only associated because of the effect of individual income has long been debated. We aimed to understand the association between income inequality and self‐rated health (SRH) and all‐cause mortality (mortality) and assess if these relationships are likely to be causal.MethodsWe searched Medline, ISI Web of Science, Embase, and EconLit (PROSPERO: CRD42021252791) for studies considering income inequality and SRH or mortality using multilevel data and adjusting for individual‐level socioeconomic position. We calculated pooled odds ratios (ORs) for poor SRH and relative risk ratios (RRs) for mortality from random‐effects meta‐analyses. We critically appraised included studies using the Risk of Bias in Nonrandomized Studies – of Interventions tool. We assessed certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation framework and causality using Bradford Hill (BH) viewpoints.FindingsThe primary meta‐analyses included 2,916,576 participants in 38 cross‐sectional studies assessing SRH and 10,727,470 participants in 14 cohort studies of mortality. Per 0.05‐unit increase in the Gini coefficient, a measure of income inequality, the ORs and RRs (95% confidence intervals) for SRH and mortality were 1.06 (1.03‐1.08) and 1.02 (1.00‐1.04), respectively. A total of 63.2% of SRH and 50.0% of mortality studies were at serious risk of bias (RoB), resulting in very low and low certainty ratings, respectively. For SRH and mortality, we did not identify relevant evidence to assess the specificity or, for SRH only, the experiment BH viewpoints; evidence for strength of association and dose–response gradient was inconclusive because of the high RoB; we found evidence in support of temporality and plausibility.ConclusionsIncreased income inequality is only marginally associated with SRH and mortality, but the current evidence base is too methodologically limited to support a causal relationship. To address the gaps we identified, future research should focus on income inequality measured at the national level and addressing confounding with natural experiment approaches.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Causal Assessment of Income Inequality on Self‐Rated Health and All‐Cause Mortality: A Systematic Review and Meta‐Analysis

SHIMONOVICH,

CAMPBELL,

THOMSON

et al. 2024

Milbank Quarterly

View full text Add to dashboard Cite

show abstract

“…Thirdly, the issue of double-counting of individuals in NMAs including observational studies was not considered in this study. As the number of real-world and observational studies using large electronic health care databases increase, it is likely that individuals could be included multiple times in evidence synthesis due to the same database being used or individuals included in the databases also taking part in RCTs, thus artificially inflating precision [ 22 ]. However, allowing for further heterogeneity across study designs and by introducing a bias factor, may mitigate the impact of this issue due to the allowance for increased uncertainty.…”

Section: Discussionmentioning

confidence: 99%

Hierarchical network meta-analysis models for synthesis of evidence from randomised and non-randomised studies

Hussein

Abrams

Gray

et al. 2023

BMC Med Res Methodol

View full text Add to dashboard Cite

Background With the increased interest in the inclusion of non-randomised data in network meta-analyses (NMAs) of randomised controlled trials (RCTs), analysts need to consider the implications of the differences in study designs as such data can be prone to increased bias due to the lack of randomisation and unmeasured confounding. This study aims to explore and extend a number of NMA models that account for the differences in the study designs, assessing their impact on the effect estimates and uncertainty. Methods Bayesian random-effects meta-analytic models, including naïve pooling and hierarchical models differentiating between the study designs, were extended to allow for the treatment class effect and accounting for bias, with further extensions allowing for bias terms to vary depending on the treatment class. Models were applied to an illustrative example in type 2 diabetes; using data from a systematic review of RCTs and non-randomised studies of two classes of glucose-lowering medications: sodium-glucose co-transporter 2 inhibitors and glucagon-like peptide-1 receptor agonists. Results Across all methods, the estimated mean differences in glycated haemoglobin after 24 and 52 weeks remained similar with the inclusion of observational data. The uncertainty around these estimates reduced when conducting naïve pooling, compared to NMA of RCT data alone, and remained similar when applying hierarchical model allowing for class effect. However, the uncertainty around these effect estimates increased when fitting hierarchical models allowing for the differences in study design. The impact on uncertainty varied between treatments when applying the bias adjustment models. Hierarchical models and bias adjustment models all provided a better fit in comparison to the naïve-pooling method. Conclusions Hierarchical and bias adjustment NMA models accounting for study design may be more appropriate when conducting a NMA of RCTs and observational studies. The degree of uncertainty around the effectiveness estimates varied depending on the method but use of hierarchical models accounting for the study design resulted in increased uncertainty. Inclusion of non-randomised data may, however, result in inferences that are more generalisable and the models accounting for the differences in the study design allow for more detailed and appropriate modelling of complex data, preventing overly optimistic conclusions.

show abstract

“…Publisher Correction: Double-counting of populations in evidence synthesis in public health: a call for awareness and future methodological development Humaira Hussein 1 , Clareece R. Nevill 1 , Anna Mefen 1 , Keith R. Abrams 1,2 , Sylwia Bujkiewicz 1 , Alex J. Sutton 1 and Laura J. Gray In the original publication of this article [1]: Box 1 was omitted during the publication process. Box 1 has been included in this correction article, the original article has been updated.…”

Section: Open Accessmentioning

confidence: 99%

Publisher Correction: Double-counting of populations in evidence synthesis in public health: a call for awareness and future methodological development

et al. 2022

Self Cite

View full text Add to dashboard Cite

Double-counting of populations in evidence synthesis in public health: a call for awareness and future methodological development

Cited by 10 publications

References 48 publications

Causal Assessment of Income Inequality on Self‐Rated Health and All‐Cause Mortality: A Systematic Review and Meta‐Analysis

Causal Assessment of Income Inequality on Self‐Rated Health and All‐Cause Mortality: A Systematic Review and Meta‐Analysis

Hierarchical network meta-analysis models for synthesis of evidence from randomised and non-randomised studies

Publisher Correction: Double-counting of populations in evidence synthesis in public health: a call for awareness and future methodological development

Contact Info

Product

Resources

About