Lipstick on a pig: Understanding efforts to redesign audit and feedback reports for primary care.

Rouleau, Geneviève; Reis, Catherine; Ivers, Noah; Desveaux, Laura

doi:10.21203/rs.3.rs-1350094/v1

Cited by 4 publications

(6 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We observed that participants without prior use of A&F were more sceptical of A&F as a tool than participants who had used A&F prior to the focus group. In line with previous research, this suggests being unknown with A&F, makes it unloved 18 19. While A&F has been used as a QI intervention in healthcare for years, studies consistently report physicians’ incapability to interpret population-level feedback 19–22.…”

Section: Discussionsupporting

confidence: 80%

Do medical specialists accept claims-based Audit and Feedback for quality improvement? A focus group study

de Weerdt,

Ybema,

Repping

et al. 2024

BMJ Open

View full text Add to dashboard Cite

ObjectivesAudit and Feedback (A&F) is a widely used quality improvement (QI) intervention in healthcare. However, not all feedback is accepted by professionals. While claims-based feedback has been previously used for A&F interventions, its acceptance by medical specialists is largely unknown. This study examined medical specialists’ acceptance of claims-based A&F for QI.DesignQualitative design, with focus group discussions. Transcripts were analysed using discourse analysis.Setting and participantsA total of five online focus group discussions were conducted between April 2021 and September 2022 with 21 medical specialists from varying specialties (urology; paediatric surgery; gynaecology; vascular surgery; orthopaedics and trauma surgery) working in academic or regional hospitals in the Netherlands.ResultsParticipants described mixed views on using claims-based A&F for QI. Arguments mentioned in favour were (1) A&F stimulates reflective learning and improvement and (2) claims-based A&F is more reliable than other A&F. Arguments in opposition were that (1) A&F is insufficient to create behavioural change; (2) A&F lacks clinically meaningful interpretation; (3) claims data are invalid for feedback on QI; (4) claims-based A&F is unreliable and (5) A&F may be misused by health insurers. Furthermore, participants described several conditions for the implementation of A&F which shape their acceptance.ConclusionsUsing claims-based A&F for QI is, for some clinical topics and under certain conditions, accepted by medical specialists. Acceptance of claims-based A&F can be shaped by how A&F is implemented into clinical practice. When designing A&F for QI, it should be considered whether claims data, as the most resource-efficient data source, can be used or whether it is necessary to collect more specific data.

show abstract

Section: Discussionsupporting

confidence: 80%

Do medical specialists accept claims-based Audit and Feedback for quality improvement? A focus group study

de Weerdt,

Ybema,

Repping

et al. 2024

BMJ Open

View full text Add to dashboard Cite

show abstract

“…Thus, even when the quality of the feedback itself is sufficient, recipients may still need education on how to interpret and translate A&F into change. 41 …”

Section: Discussionmentioning

confidence: 99%

“…Experts state that the optimal design of A&F likely depends on recipient factors and context, but it is yet unclear which thresholds exists for quality of feedback to be ‘sufficient’ to contribute to motivation to change6 Furthermore, Desveaux et al 35 describe that ‘A&F may not be translated into actual change, due to incapability of participants to interpret A&F in an actionable way’. Thus, even when the quality of the feedback itself is sufficient, recipients may still need education on how to interpret and translate A&F into change 41…”

Section: Discussionmentioning

confidence: 99%

“…Thus, even when the quality of the feedback itself is sufficient, recipients may still need education on how to interpret and translate A&F into change. 41 Multiple studies throughout the years describe that GPs find A&F reports unusable, while these same studies do find that GPs are motivated to use A&F for quality improvement. [41][42][43] In addition in our study, GPs state that they receive a multitude of A&F reports, many of which they hardly use.…”

Section: Open Accessmentioning

confidence: 99%

“…41 Multiple studies throughout the years describe that GPs find A&F reports unusable, while these same studies do find that GPs are motivated to use A&F for quality improvement. [41][42][43] In addition in our study, GPs state that they receive a multitude of A&F reports, many of which they hardly use. Using low-quality A&F creates waste of resources and leads to ineffective use of clinician's time.…”

Section: Open Accessmentioning

confidence: 99%

See 2 more Smart Citations

‘Give me something meaningful’: GPs perspectives on how to improve an audit and feedback report provided by health insurers – an exploratory qualitative study

et al. 2022

View full text Add to dashboard Cite

BackgroundAudit and feedback (A&F) is a valuable quality improvement strategy, which can contribute to de-implementation of low-value care. In the Netherlands, all health insurers collaboratively provide A&F to general practitioners (GPs), the ‘Primary Care Practice Report’ (PCPR). Unfortunately, the use of this report by GPs is limited. This study examined the thoughts of GPs on the usability of the PCPR and GPs recommendations for improving the PCPR.MethodWe used an interpretative qualitative design, with think-aloud tasks to uncover thoughts of GPs on the usability of the PCPR and semistructured interview questions to ask GPs’ recommendations for improvement of the PCPR. Interviews were audiorecorded and transcribed ad verbatim. Data were analysed using thematic content analysis.ResultsWe identified two main themes: ‘poor usability of the PCPR’, and ‘minimal motivation to change based on the PCPR’. The GPs found the usability of the PCPR poor due to the feedback not being clinically meaningful, the data not being recent, individual and reliable, the performance comparators offer insufficient guidance to assess clinical performance, the results are not discussed with peers and the definitions and visuals are unclear. The GPs recommended improving these issues. The GPs motivation to change based on the PCPR was minimal.ConclusionsThe GPs evaluated the PCPR as poorly usable and were minimally motivated to change. The PCPR seems developed from the perspective of the reports’ commissioners, health insurers, and does not meet known criteria for effective A&F design and user-centred design. Importantly, the GPs did state that well-designed feedback could contribute to their motivation to improve clinical performance.Furthermore, the GPs stated that they receive a multitude of A&F reports, which they hardly use. Thus, we see a need for policy makers to invest in less, but more usable A&F reports.

show abstract

Developing an Audit and Feedback Dashboard for Family Physicians: User-Centered Design Process

Shuldiner¹,

Kiran²,

Agarwal³

et al. 2023

JMIR Hum Factors

View full text Add to dashboard Cite

Background Audit and feedback (A&F), the summary and provision of clinical performance data, is a common quality improvement strategy. Successful design and implementation of A&F—or any quality improvement strategy—should incorporate evidence-informed best practices as well as context-specific end user input. Objective We used A&F theory and user-centered design to inform the development of a web-based primary care A&F dashboard. We describe the design process and how it influenced the design of the dashboard. Methods Our design process included 3 phases: prototype development based on A&F theory and input from clinical improvement leaders; workshop with family physician quality improvement leaders to develop personas (ie, fictional users that represent an archetype character representative of our key users) and application of those personas to design decisions; and user-centered interviews with family physicians to learn about the physician’s reactions to the revised dashboard. Results The team applied A&F best practices to the dashboard prototype. Personas were used to identify target groups with challenges and behaviors as a tool for informed design decision-making. Our workshop produced 3 user personas, Dr Skeptic, Frazzled Physician, and Eager Implementer, representing common users based on the team’s experience of A&F. Interviews were conducted to further validate findings from the persona workshop and found that (1) physicians were interested in how they compare with peers; however, if performance was above average, they were not motivated to improve even if gaps compared to other standards in their care remained; (2) burnout levels were high as physicians are trying to catch up on missed care during the pandemic and are therefore less motivated to act on the data; and (3) additional desired features included integration within the electronic medical record, and more up-to-date and accurate data. Conclusions We found that carefully incorporating data from user interviews helped operationalize generic best practices for A&F to achieve an acceptable dashboard that could meet the needs and goals of physicians. We demonstrate such a design process in this paper. A&F dashboards should address physicians’ data skepticism, present data in a way that spurs action, and support physicians to have the time and capacity to engage in quality improvement work; the steps we followed may help those responsible for quality improvement strategy implementation achieve these aims.

show abstract

Lipstick on a pig: Understanding efforts to redesign audit and feedback reports for primary care.

Cited by 4 publications

References 21 publications

Do medical specialists accept claims-based Audit and Feedback for quality improvement? A focus group study

Do medical specialists accept claims-based Audit and Feedback for quality improvement? A focus group study

‘Give me something meaningful’: GPs perspectives on how to improve an audit and feedback report provided by health insurers – an exploratory qualitative study

Developing an Audit and Feedback Dashboard for Family Physicians: User-Centered Design Process

Contact Info

Product

Resources

About