A scoping review of online repositories of quality improvement projects, interventions and initiatives in healthcare

Bytautas, Jessica P; Gheihman, Galina; Dobrow, Mark

doi:10.1136/bmjqs-2015-005092

Cited by 8 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Empirical evidence on selective outcome reporting has primarily been obtained from RCTs as study protocols are made available in the trial registration process [4]. Calls for prospective registration of study protocols of observational studies have been made [30] and repositories of quality improvement projects are emerging [31]. HSR and quality improvement communities will need to consider and evaluate the feasibility and values of adopting these practices.…”

Section: Discussionmentioning

confidence: 99%

Publication and related biases in health services research: a systematic review of empirical evidence

Ayorinde

Williams

Mannion

et al. 2020

BMC Med Res Methodol

View full text Add to dashboard Cite

Background: Publication and related biases (including publication bias, time-lag bias, outcome reporting bias and p-hacking) have been well documented in clinical research, but relatively little is known about their presence and extent in health services research (HSR). This paper aims to systematically review evidence concerning publication and related bias in quantitative HSR. Methods: Databases including MEDLINE, EMBASE, HMIC, CINAHL, Web of Science, Health Systems Evidence, Cochrane EPOC Review Group and several websites were searched to July 2018. Information was obtained from: (1) Methodological studies that set out to investigate publication and related biases in HSR; (2) Systematic reviews of HSR topics which examined such biases as part of the review process. Relevant information was extracted from included studies by one reviewer and checked by another. Studies were appraised according to commonly accepted scientific principles due to lack of suitable checklists. Data were synthesised narratively. Results: After screening 6155 citations, four methodological studies investigating publication bias in HSR and 184 systematic reviews of HSR topics (including three comparing published with unpublished evidence) were examined. Evidence suggestive of publication bias was reported in some of the methodological studies, but evidence presented was very weak, limited in both quality and scope. Reliable data on outcome reporting bias and p-hacking were scant. HSR systematic reviews in which published literature was compared with unpublished evidence found significant differences in the estimated intervention effects or association in some but not all cases. Conclusions: Methodological research on publication and related biases in HSR is sparse. Evidence from available literature suggests that such biases may exist in HSR but their scale and impact are difficult to estimate for various reasons discussed in this paper. Systematic review registration: PROSPERO 2016 CRD42016052333.

show abstract

Section: Discussionmentioning

confidence: 99%

Publication and related biases in health services research: a systematic review of empirical evidence

Ayorinde

Williams

Mannion

et al. 2020

BMC Med Res Methodol

View full text Add to dashboard Cite

show abstract

“…1 Stakeholders in our interviews generally supported registration of HSDR, albeit with some reservations, and quality improvement projects and implementation studies were identified as being potentially at high risk of these bias. As repositories of quality improvement projects emerge, 100 HSDR and quality improvement communities will need to consider and evaluate the feasibility and values of adopting these practices for studies of varied types and purposes.…”

Section: Overall Learning From the Project And Study Limitationsmentioning

confidence: 99%

Publication and related bias in quantitative health services and delivery research: a multimethod study

Ayorinde

Williams

Mannion

et al. 2020

Health Serv Deliv Res

View full text Add to dashboard Cite

Background Bias in the publication and reporting of research findings (referred to as publication and related bias here) poses a major threat in evidence synthesis and evidence-based decision-making. Although this bias has been well documented in clinical research, little is known about its occurrence and magnitude in health services and delivery research. Objectives To obtain empirical evidence on publication and related bias in quantitative health services and delivery research; to examine current practice in detecting/mitigating this bias in health services and delivery research systematic reviews; and to explore stakeholders’ perception and experiences concerning such bias. Methods The project included five distinct but interrelated work packages. Work package 1 was a systematic review of empirical and methodological studies. Work package 2 involved a survey (meta-epidemiological study) of randomly selected systematic reviews of health services and delivery research topics (n = 200) to evaluate current practice in the assessment of publication and outcome reporting bias during evidence synthesis. Work package 3 included four case studies to explore the applicability of statistical methods for detecting such bias in health services and delivery research. In work package 4 we followed up four cohorts of health services and delivery research studies (total n = 300) to ascertain their publication status, and examined whether publication status was associated with statistical significance or perceived ‘positivity’ of study findings. Work package 5 involved key informant interviews with diverse health services and delivery research stakeholders (n = 24), and a focus group discussion with patient and service user representatives (n = 8). Results We identified only four studies that set out to investigate publication and related bias in health services and delivery research in work package 1. Three of these studies focused on health informatics research and one concerned health economics. All four studies reported evidence of the existence of this bias, but had methodological weaknesses. We also identified three health services and delivery research systematic reviews in which findings were compared between published and grey/unpublished literature. These reviews found that the quality and volume of evidence and effect estimates sometimes differed significantly between published and unpublished literature. Work package 2 showed low prevalence of considering/assessing publication (43%) and outcome reporting (17%) bias in health services and delivery research systematic reviews. The prevalence was lower among reviews of associations than among reviews of interventions. The case studies in work package 3 highlighted limitations in current methods for detecting these biases due to heterogeneity and potential confounders. Follow-up of health services and delivery research cohorts in work package 4 showed positive association between publication status and having statistically significant or positive findings. Diverse views concerning publication and related bias and insights into how features of health services and delivery research might influence its occurrence were uncovered through the interviews with health services and delivery research stakeholders and focus group discussion conducted in work package 5. Conclusions This study provided prima facie evidence on publication and related bias in quantitative health services and delivery research. This bias does appear to exist, but its prevalence and impact may vary depending on study characteristics, such as study design, and motivation for conducting the evaluation. Emphasis on methodological novelty and focus beyond summative assessments may mitigate/lessen the risk of such bias in health services and delivery research. Methodological and epistemological diversity in health services and delivery research and changing landscape in research publication need to be considered when interpreting the evidence. Collection of further empirical evidence and exploration of optimal health services and delivery research practice are required. Study registration This study is registered as PROSPERO CRD42016052333 and CRD42016052366. Funding This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 33. See the NIHR Journals Library website for further project information.

show abstract

“…7 Accordingly, only a fraction of QI projects implemented in practice are reported in peer-reviewed journals. 8 There are some online QI reporting repositories to disseminate learning, however, variation in the classification of projects makes comparisons difficult. 8 Furthermore, while the Standards for Quality Improvement Reporting Excellence guidelines advise to report costs, 9 barriers to this include disparate and limited formal guidance for improvement teams on the measurement of costs associated with QI, Open access and the accessibility of organisational and patient cost data.…”

Section: Introductionmentioning

confidence: 99%

“…8 There are some online QI reporting repositories to disseminate learning, however, variation in the classification of projects makes comparisons difficult. 8 Furthermore, while the Standards for Quality Improvement Reporting Excellence guidelines advise to report costs, 9 barriers to this include disparate and limited formal guidance for improvement teams on the measurement of costs associated with QI, Open access and the accessibility of organisational and patient cost data. 7 10-15 To support effective decision-making, robust information is required about the content and context of QI initiatives, the expected outcomes, initial costs of implementation and the subsequent impact on longterm costs for the health service.…”

Section: Introductionmentioning

confidence: 99%

Reporting standards, outcomes and costs of quality improvement studies in Ireland: a scoping review

McCarthy¹,

Jabakhanji²,

Martin

et al. 2021

BMJ Open Qual

View full text Add to dashboard Cite

ObjectivesTo profile the aims and characteristics of quality improvement (QI) initiatives conducted in Ireland, to review the quality of their reporting and to assess outcomes and costs.DesignScoping review.Data sourcesSystematic searches were conducted in PubMed, Web of Science, Embase, Google Scholar, Lenus and rian.ie. Two researchers independently screened abstracts (n=379) and separately reviewed 43 studies identified for inclusion using a 70-item critique tool. The tool was based on the Quality Improvement Minimum Quality Criteria Set (QI-MQCS), an appraisal instrument for QI intervention publications, and health economics reporting criteria. After reaching consensus, the final dataset was analysed using descriptive statistics. To support interpretations, findings were presented at a national stakeholder workshop.Eligibility criteriaQI studies implemented and evaluated in Ireland and published between January 2015 and April 2020.ResultsThe 43 studies represented various QI interventions. Most studies were peer-reviewed publications (n=37), conducted in hospitals (n=38). Studies mainly aimed to improve the ‘effectiveness’ (65%), ‘efficiency’ (53%), ‘timeliness’ (47%) and ‘safety’ (44%) of care. Fewer aimed to improve ‘patient-centredness’ (30%), ‘value for money’ (23%) or ‘staff well-being’ (9%). No study aimed to increase ‘equity’. Seventy per cent of studies described 14 of 16 QI-MQCS dimensions. Least often studies reported the ‘penetration/reach’ of an initiative and only 35% reported health outcomes. While 53% of studies expressed awareness of costs, only eight provided at least one quantifiable figure for costs or savings. No studies assessed the cost-effectiveness of the QI.ConclusionIrish QI studies included in our review demonstrate varied aims and high reporting standards. Strategies are needed to support greater stimulation and dissemination of QI beyond the hospital sector and awareness of equity issues as QI work. Systematic measurement and reporting of costs and outcomes can be facilitated by integrating principles of health economics in QI education and guidelines.

show abstract

A scoping review of online repositories of quality improvement projects, interventions and initiatives in healthcare

Cited by 8 publications

References 14 publications

Publication and related biases in health services research: a systematic review of empirical evidence

Publication and related biases in health services research: a systematic review of empirical evidence

Publication and related bias in quantitative health services and delivery research: a multimethod study

Reporting standards, outcomes and costs of quality improvement studies in Ireland: a scoping review

Contact Info

Product

Resources

About