Comparison of risk-of-bias assessment approaches for selection of studies reporting prevalence for economic analyses

Abstract

ObjectivesWithin cost-effectiveness models, prevalence figures can inform transition probabilities. The methodological quality of studies can inform the choice of prevalence figures but no single obvious candidate tool exists for assessing quality of the observational epidemiological studies for selecting prevalence estimates. We aimed to compare different tools to assess the risk of bias of studies reporting prevalence, and develop and compare possible numerical scoring systems using these tools to set a threshold for inclusion of reports of prevalence in an economic analysis of neonatal hypoglycaemia.DesignAssessments of bias using two tools (Joanna Briggs Institute (JBI) Checklist for Prevalence Studies and a modified version of Risk Of Bias In Non-randomised Studies-of Interventions (ROBINS-I)) were compared for 18 studies relevant to a single setting (neonatal hypoglycaemia). Inclusions of studies for use in a decision analysis model were considered based on summary scores derived from these tools.ResultsBoth tools were considered easy to use and produced dispersed scores for each of the 40 study–outcome combinations. The modified ROBINS-I scores were more skewed than the JBI scores, particularly at higher thresholds. The studies selected for inclusion are generally the same using either tool; if 50% was used as the cut-off threshold using the Applicable Score both tools would yield the same results. However, the JBI tool is shorter and may be easier to interpret and apply to studies that do not involve a control group, while the modified ROBINS-I tool assesses more methodological detail in studies that include a control group.ConclusionBoth tools performed well for systematically assessing studies that report on outcome prevalence and provided similar discrimination between studies for risk of bias. This convergent validity supports use of both tools for the purpose of assessing risk of bias and selecting studies that report prevalence for inclusion in economic analyses.

Changes in quality of life following hypoglycaemia in adults with type 2 diabetes: A systematic review of longitudinal studies

“…Quality of included studies was rated by a reviewer (KM) using critical appraisal tools from the Joanna Briggs Institute (JBI) specific to RCTs and cohort studies, 16 and ratings were checked by a second reviewer (MVJ). To provide a concise metric for quantitative comparisons across designs, an overall quality proportion ( α q ) was calculated for each study using a count score 17 whereby the number of ‘yes’ items was divided by the total number of items on the appraisal tool (‘unclear’ items were counted as half, while ‘not applicable’ items were excluded). Proportions ranged from 0 to 1 and were interpreted in a manner similar to Cronbach's α 18 : α q ≥ 0.9 ( excellent ), α q ≥ 0.8 ( good ), α q ≥ 0.7 ( acceptable ) and α q < 0.7 ( poor ).…”

Quality of economic evaluations of drug-coated balloons and drug-eluting stents in peripheral artery disease: a systematic review

“…However, the Cooper checklist does not allow an in-depth analysis of the quality of the clinical data used, especially for retrospective clinical data that are at a high risk of bias. Exploring this point, a recent study suggested using two checklists, the Joanna Briggs Institute (JBI) Checklist for Prevalence Studies and a modified version of Risk Of Bias In Non-randomised Studies of Interventions (ROBINS-I) (25); it would be relevant to combine the Cooper checklist with these two checklists. However, it must also be kept in mind that the more tools used for a single study, the more complicated and longer the assessment is.…”

Factors and strategies affecting motor imagery ability in people with multiple sclerosis: a systematic review

“…Risk of bias for case-control and analytical cross-sectional studies was assessed using the Joanna Briggs Institute (JBI) Checklist for Case-Control [13] and Analytical Cross-Sectional Studies [14], respectively. A ≥50% cut-off score was used to indicate a low risk of J o u r n a l P r e -p r o o f bias [15]. For randomised controlled trials (RCTs), the revised Cochrane "Risk of bias" (RoB) tool (RoB 2.0) was utilised [16], based on which an overall summary RoB judgement (low; some concerns; high) for each specific outcome was derived [17].…”