Three risk of bias tools lead to opposite conclusions in observational research synthesis

Losilla, Josep-María; Oliveras, Isabel; Marín-García, Juán A.; Vives, Jaume

doi:10.1016/j.jclinepi.2018.05.021

Cited by 79 publications

(80 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, the present meta-analysis was unable to assess risks by hormone receptor status because only one study provided stratified analysis for ER, which showed that soybean product intake was associated with a significantly decreased risk of ER-positive breast cancer, with odds ratios in the top tertile of intake of 0.74 (95% CI, 0.58-0.94), but the result were not significant in ER-negative breast cancer [43]. Fifth, there are more advanced tools available for quality assessment of studies cross-sectional and cohort studies, such as quality of cohort studies (Q-Coh) and risk of bias in nonrandomized studies of interventions (ROBINS-I), however, we only used the NOS to rate the individual articles, which might underestimate the risk of bias of studies [54,55]. Sixth, tofu is very versatile as a food; it can be further processed into various secondary tofu products, including deep-fried tofu, grilled tofu, frozen tofu, dried-frozen tofu, fermented tofu, which may have different effects on breast cancer.…”

Section: Discussionmentioning

confidence: 99%

Tofu intake is inversely associated with risk of breast cancer: A meta-analysis of observational studies

2020

View full text Add to dashboard Cite

Observational studies on the association between tofu intake and breast cancer incidence have reported inconsistent results. We reviewed the current evidence and quantitatively assessed this association by conducting a dose-response meta-analysis. The electronic databases PubMed and EMBASE were searched for relevant studies published up to August, 2018. We included epidemiological studies that reported relative risks (RRs) or odds ratios (ORs) with 95% confidence intervals (CIs) for the association between tofu intake and breast cancer risk. A total of 14 studies (2 cohort studies, 12 case-control studies) were included in the meta-analysis. The overall OR of breast cancer for highest vs lowest intake of tofu was 0.78 (95% CI 0.69-0.88), with moderate heterogeneity (P = 0.011, I 2 = 49.7%). Dose-response analysis based on 5 case-control studies revealed that each 10 g/d increase in tofu intake was associated with 10% reduction in the risk of breast cancer (95% CI 7%-13%, P = 0.037, I 2 = 40.8%). In summary, our findings suggest an inverse doseresponse association between tofu intake and risk of breast cancer. However, owing to the limitations of case-control studies, more properly designed prospective studies are warranted to confirm this association.PLOS ONE | https://doi.org/10.1371/journal.pone.0226745 January 7, 2020 1 / 13 OPEN ACCESSCitation: Wang Q, Liu X, Ren S (2020) Tofu intake is inversely associated with risk of breast cancer: A meta-analysis of observational studies. PLoS ONE 15(1): e0226745. https://doi.org/10.

show abstract

Section: Discussionmentioning

confidence: 99%

Tofu intake is inversely associated with risk of breast cancer: A meta-analysis of observational studies

2020

View full text Add to dashboard Cite

show abstract

“…In addition, the forms contained a set of prespecified questions assessing study design, analytical methods, transparency of reporting, and potential for bias. While some tools for assessing quality of observational studies have been proposed, most of them are poorly suited to evaluating the variety of design options and possible sources of bias present in pharmacoepidemiological research . In addition, they are often not specific enough for a particular clinical research area; thus, we developed a customized list of questions, that was compiled based on the authors' expertise and published literature .…”

Section: Methodsmentioning

confidence: 99%

“…While some tools for assessing quality of observational studies have been proposed, most of them are poorly suited to evaluating the variety of design options and possible sources of bias present in pharmacoepidemiological research. 23 In addition, they are often not specific enough for a particular clinical research area; thus, we developed a customized list of questions, that was compiled based on the authors' expertise and published literature. 11,12,14,20,24,25 In short, we assessed each study on: inclusion of incident or prevalent users; choice of comparator/control group;…”

Section: Data Extraction and Reviewmentioning

confidence: 99%

Glucose‐lowering medications and the risk of cancer: A methodological review of studies based on real‐world data

Bykov

Franklin

et al. 2019

Diabetes Obesity Metabolism

View full text Add to dashboard Cite

Aim: Recent years have witnessed a proliferation of observational research examining the association between glucose-lowering medications and cancer, paired with an increasing concern on the quality of the produced evidence. Our objective was to review published studies in this area to identify the most common methodological challenges and sources of bias. Methods: We systematically searched PubMed to identify observational studies on glucose-lowering medications and cancer published between January 2000 and January 2016. We assessed the design and analytical methods used in each study, with a focus on their ability to achieve study validity, and further evaluated the prevalence of major methodological choices over time. Results: Out of 155 studies evaluated, only 26% implemented a new user design, 41% used an active comparator, 33% implemented a lag or latency period, 51% adjusted for diabetes duration. Potential for immortal person-time bias was identified in 63% of the studies; 55% of the studies adjusted for variables measured during the follow-up without appropriate statistical methods. Aside from a decreasing trend in adjusting for variables measured during the follow-up, we observed no trends in methodological choices over time. Conclusions: The prevalence of well-known design and analysis flaws that may lead to biased results remains high among observational studies on glucose-lowering medications and cancer, limiting the conclusions that can be drawn from these studies. Avoiding known pitfalls could substantially improve the quality and validity of real-world evidence in this field.

show abstract

“…As with any elaborate tool, it is important to evaluate concerns regarding the practical use of ROBINS-I. To the best of our knowledge, there are two studies [29,30] that have assessed the IRR of ROBINS-I. In Losilla et al [29], using a sample of studies on health psychology, the IRR for ROBINS-I was reported to range from slight to an almost perfect agreement for various items and domains using the kappa statistic.…”

Section: Strengths and Challengesmentioning

confidence: 99%

Inter-rater reliability and concurrent validity of ROBINS-I: protocol for a cross-sectional study

et al. 2020

View full text Add to dashboard Cite

Background: The Cochrane Bias Methods Group recently developed the "Risk of Bias (ROB) in Non-randomized Studies of Interventions" (ROBINS-I) tool to assess ROB for non-randomized studies of interventions (NRSI). It is important to establish consistency in its application and interpretation across review teams. In addition, it is important to understand if specialized training and guidance will improve the reliability of the results of the assessments. Therefore, the objective of this cross-sectional study is to establish the inter-rater reliability (IRR), interconsensus reliability (ICR), and concurrent validity of ROBINS-I. Furthermore, as this is a relatively new tool, it is important to understand the barriers to using this tool (e.g., time to conduct assessments and reach consensusevaluator burden). Methods: Reviewers from four participating centers will appraise the ROB of a sample of NRSI publications using the ROBINS-I tool in two stages. For IRR and ICR, two pairs of reviewers will assess the ROB for each NRSI publication. In the first stage, reviewers will assess the ROB without any formal guidance. In the second stage, reviewers will be provided customized training and guidance. At each stage, each pair of reviewers will resolve conflicts and arrive at a consensus. To calculate the IRR and ICR, we will use Gwet's AC 1 statistic. For concurrent validity, reviewers will appraise a sample of NRSI publications using both the New-castle Ottawa Scale (NOS) and ROBINS-I. We will analyze the concordance between the two tools for similar domains and for the overall judgments using Kendall's tau coefficient. To measure the evaluator burden, we will assess the time taken to apply the ROBINS-I (without and with guidance), and the NOS. To assess the impact of customized training and guidance on the evaluator burden, we will use the generalized linear models. We will use Microsoft Excel and SAS 9.4 to manage and analyze study data, respectively. Discussion: The quality of evidence from systematic reviews that include NRS depends partly on the study-level ROB assessments. The findings of this study will contribute to an improved understanding of the ROBINS-I tool and how best to use it.

show abstract

Three risk of bias tools lead to opposite conclusions in observational research synthesis

Cited by 79 publications

References 44 publications

Tofu intake is inversely associated with risk of breast cancer: A meta-analysis of observational studies

Tofu intake is inversely associated with risk of breast cancer: A meta-analysis of observational studies

Glucose‐lowering medications and the risk of cancer: A methodological review of studies based on real‐world data

Inter-rater reliability and concurrent validity of ROBINS-I: protocol for a cross-sectional study

Contact Info

Product

Resources

About