Growing Literature, Stagnant Science? Systematic Review, Meta-Regression and Cumulative Analysis of Audit and Feedback Interventions in Health Care

Abstract

BACKGROUNDThis paper extends the findings of the Cochrane systematic review of audit and feedback on professional practice to explore the estimate of effect over time and examine whether new trials have added to knowledge regarding how optimize the effectiveness of audit and feedback.METHODSWe searched the Cochrane Central Register of Controlled Trials, MEDLINE, and EMBASE for randomized trials of audit and feedback compared to usual care, with objectively measured outcomes assessing compliance with intended professional practice. Two reviewers independently screened articles and abstracted variables related to the intervention, the context, and trial methodology. The median absolute risk difference in compliance with intended professional practice was determined for each study, and adjusted for baseline performance. The effect size across studies was recalculated as studies were added to the cumulative analysis. Meta-regressions were conducted for studies published up to 2002, 2006, and 2010 in which characteristics of the intervention, the recipients, and trial risk of bias were tested as predictors of effect size.RESULTSOf the 140 randomized clinical trials (RCTs) included in the Cochrane review, 98 comparisons from 62 studies met the criteria for inclusion. The cumulative analysis indicated that the effect size became stable in 2003 after 51 comparisons from 30 trials. Cumulative meta-regressions suggested new trials are contributing little further information regarding the impact of common effect modifiers. Feedback appears most effective when: delivered by a supervisor or respected colleague; presented frequently; featuring both specific goals and action-plans; aiming to decrease the targeted behavior; baseline performance is lower; and recipients are non-physicians.DISCUSSIONThere is substantial evidence that audit and feedback can effectively improve quality of care, but little evidence of progress in the field. There are opportunity costs for patients, providers, and health care systems when investigators test quality improvement interventions that do not build upon, or contribute toward, extant knowledge.

Primary Care Physicians' Experiences With and Attitudes Toward Pediatric Quality Reporting

“…This finding underscores that production and distribution of quality reports might be insufficient for practices to use them as a tool for quality improvement without some other forms of support or incentives, such as technical assistance on the use of reports or financial incentives for improvement. [13][14][15][21][22][23] Consistent with other research, 6,14,18 the survey and qualitative findings in this study suggest that physicians are more likely to be receptive to and use quality reports when reports align with physicians' priorities, contain information specific to their patients with clear benchmarks for comparison, are timely, provide recommendations for improvement, and are developed in consultation with practicing physicians. 6,14,18 Furthermore, physicians need to have the skills and time to do the improvement work, which is not typically a reimbursed activity.…”

“…[13][14][15][21][22][23] Consistent with other research, 6,14,18 the survey and qualitative findings in this study suggest that physicians are more likely to be receptive to and use quality reports when reports align with physicians' priorities, contain information specific to their patients with clear benchmarks for comparison, are timely, provide recommendations for improvement, and are developed in consultation with practicing physicians. 6,14,18 Furthermore, physicians need to have the skills and time to do the improvement work, which is not typically a reimbursed activity. Historically, the primary care delivery system and its financing has provided few supports for providers to traverse the gap between quality measurement and action.…”

“…6,9,13,14,18 The primary dependent variables included: 1) receipt of any quality reports; 2) receipt of reports with key measures relevant to pediatric care; 3) engaging in any QI efforts; 4) engaging in QI in response to quality reports; and 5) perceiving quality reports as an effective tool in QI efforts. Additionally, we asked physicians about their opinions on the usefulness of specific types of information used to create quality reports, such as the populations of children included in the report and comparisons to benchmarks.…”

“…[1][2][3][4] Challenges in delivering high quality care are particularly significant for providers caring for children who face increased risks for health care problems, including publicly insured children. [2][3][4] Quality measurement and reporting at the physician level is a common, potentially effective approach to improve the quality of health care, 5,6 and physician recertification programs have included a requirement for involvement in measuring the quality of care and quality improvement activities. 7,8 Quality measurement and reporting in child health care has lagged behind efforts in adult health care, but many recent state and federal initiatives have sought to close that gap.…”

“…6 Physicians' experiences with and attitudes toward quality reporting are key influences on the effectiveness of these efforts, 16 but few studies have assessed the experiences with and attitudes toward quality reporting for primary care providers for children. 6,17 To address this gap, we conducted a survey of physicians in three states to examine the degree to which primary care providers for children report receiving quality reports, the sources and content of reports, related QI efforts, and attitudes about quality reporting. We assessed the associations between these experiences and attitudes and physician characteristics, inclduing exposure to demonstration states' projects.…”