Framing Bias in the Interpretation of Quality Improvement Data: Evidence From an Experiment

Ballard, Andrew

doi:10.15171/ijhpm.2019.08

Cited by 15 publications

(10 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is little evidence that the considerable effort and resources invested in quality registrations, including those used to benchmark healthcare providers, and incentives lead to improved health outcomes. 4,[10][11][12] Moreover, quality indicators, as seemingly objective data, may be prone to validity and reliability issues. As these data are often collected by healthcare professionals and managers, they are subject to interpretation issues and even gaming to improve institutions' results.…”

Section: Key Messagesmentioning

confidence: 99%

Perceived Burden Due to Registrations for Quality Monitoring and Improvement in Hospitals: A Mixed Methods Study

Zegers¹,

Veenstra

Gerritsen

et al. 2020

Int J Health Policy Manag

View full text Add to dashboard Cite

Background: Quality indicators are registered to monitor and improve the quality of care. However, the number and effectiveness of quality indicators is under debate, and may influence the joy in work of physicians and nurses. Empirical data on the nature and consequences of the registration burden are lacking. The aim of this study was to identify and explore healthcare professionals’ perceived burden due to quality registrations in hospitals, and the effect of this burden on their joy in work. Methods: A mixed methods observational study, including participative observations, a survey and semi-structured interviews in two academic hospitals and one teaching hospital in the Netherlands. Study participants were 371 healthcare professionals from an intensive care unit (ICU), a haematology department and others involved in the care of elderly patients and patients with prostate or gastrointestinal cancer. Results: On average, healthcare professionals spend 52.3 minutes per working day on quality registrations. The average number of quality measures per department is 91, with 1380 underlying variables. Overall, 57% are primarily registered for accountability purposes, 19% for institutional governance and 25% for quality improvement objectives. Only 36% were perceived as useful for improving quality in everyday practice. Eight types of registration burden were identified, such as an excessive number of quality registrations, and the lack of usefulness for improving quality and inefficiencies in the registration process. The time healthcare professionals spent on quality registrations was not correlated with any measure of joy in work. Perceived unreasonable registrations were negatively associated with healthcare professionals’ joy in work (intrinsic motivation and autonomy). Healthcare professionals experienced quality registrations as diverting time from patient care and from actually improving quality. Conclusion: Registering fewer quality indicators, but more of what really matters to healthcare professionals, is key to increasing the effectiveness of registrations for quality improvement and governance. Also the efficiency of quality registrations should be increased through staffing and information and communications technology solutions to reduce the registration burden experienced by nurses and physicians.

show abstract

Section: Key Messagesmentioning

confidence: 99%

Perceived Burden Due to Registrations for Quality Monitoring and Improvement in Hospitals: A Mixed Methods Study

Zegers¹,

Veenstra

Gerritsen

et al. 2020

Int J Health Policy Manag

View full text Add to dashboard Cite

show abstract

“…Although a criticism of all health worker motivation studies, quantitative data were self-reported and may be subject to acceptability biases. QI data have been found previously to be influenced by whether questions were framed positively or negatively ( Ballard, 2019 ), yet although such biases would overestimate cross-sectional estimates of motivation, these would likely cancel out in our analyses over time or by QI or comparison area, assuming biases were not affected by the programme. Additionally, data were not available on potentially important contextual factors which may have enhanced or inhibited the impact of the QI intervention, e.g.…”

Section: Discussionmentioning

confidence: 89%

Changes in health worker knowledge and motivation in the context of a quality improvement programme in Ethiopia

Quaife

Estafinos

Keraga

et al. 2021

Health Policy and Planning

View full text Add to dashboard Cite

A knowledgeable and motivated workforce is critical for health systems to provide high-quality services. Many low- and middle-income countries face shortages in human resources and low health worker motivation but are also home to a burgeoning number of quality improvement (QI) programmes. This study evaluates whether and how motivation and clinical knowledge in three cadres of health workers changed in the context of a QI programme for maternal and newborn health in Ethiopia. This mixed-methods study used a pre–post comparison group design with matched comparison areas. We interviewed 395 health workers at baseline in April 2018 and 404 at endline in June 2019 from seven districts (woredas) with QI and seven comparison woredas. Three cadres were interviewed: health extension workers, facility-based skilled midlevel maternal and newborn care providers, and non-patient-facing staff. A qualitative component sought to triangulate and further elucidate quantitative findings using in-depth interviews with 22 health workers. Motivation was assessed quantitatively, exploratory factor analysis was used to categorize motivation dimensions, and regression-based difference-in-difference analyses were conducted. Knowledge was assessed through a clinical vignette. Qualitative data were analysed in a deductive process based on a framework derived from quantitative results. Although knowledge of the QI programme was high (79%) among participants from QI woreda at endline, participation in QI teams was lower (56%). There was strong evidence that health worker knowledge increased more in areas with QI than comparison areas. Three motivation dimensions emerged from the data: (1) ‘helping others’, (2) ‘pride and satisfaction’ and (3) ‘external recognition and support’. We found strong evidence that motivation across these factors improved in both QI and comparison areas, with weak evidence of greater increases in comparison areas. Qualitative data suggested the QI programme may have improved motivation by allowing staff to provide better care. This study suggests that although QI programmes can increase health worker knowledge, there may be little effect on motivation. Programme evaluations should measure a wide range of outcomes to fully understand their impact.

show abstract

“…This challenge is not unique to our study or its design, however, and it is impossible to know how or if such differences in interpretation would impact the results. [17][18] One unique confounder to our study that could have affected both the response rate and potential interpretation of the results was the timing of survey administration. This survey was completed during the early stages of the COVID-19 pandemic in April 2020 to May 2020, just as national and local lockdowns were in effect and many educational institutions switched to virtual learning.…”

Section: Discussionmentioning

confidence: 99%

Assessment of Impostor Phenomenon in Student Pharmacists and Faculty at Two Doctor of Pharmacy Programs

Boyle

Malcom

Barker³

et al. 2022

American Journal of Pharmaceutical Education

View full text Add to dashboard Cite

The objective of this study was to evaluate the prevalence of impostor phenomenon (IP) in student pharmacists and faculty members at two educational institutions in the US.Methods. An electronic self-report survey instrument including the validated Clance Impostor Phenomenon Scale (CIPS) was delivered anonymously from April 2020 to May 2020. Demographic data including age range, gender, and other characteristics were collected. CIPS scores were reported as overall mean (SD) and data were compared between institutions and demographic groups using Student's t tests.Results. The overall mean CIPS score for all survey respondents (N=209, 35.5% response rate) was 63.8 (SD=15.1). Mean student pharmacist CIPS score for NEOMED was 64.7 (SD=14.4) vs. 63.8 (SD=16.1) for SUCOPHS, which was similar. Mean faculty CIPS score for NEOMED was 59.2 (SD=14.0) vs. 64.7 (SD=16.8) for SUCOPHS, which was similar. Mean CIPS score for the combined student pharmacist group (NEOMED and SUCOPHS) was 64.3 (SD=15.1) vs. 61.2 (SD=15.1) for the combined group of faculty members, which was similar. Overall, most respondents fell in the "moderate" to "frequent" CIPS score classification range (36.4% and 44%, respectively). Conclusion.Our study found that IP feelings were common among responding student pharmacists and faculty members at the institutions surveyed, though little is known currently about the implications of these findings. Future research should seek to elucidate factors predictive of or associated with IP as well as assess the impact of strategies to prevent or manage IP.

show abstract

Framing Bias in the Interpretation of Quality Improvement Data: Evidence From an Experiment

Cited by 15 publications

References 40 publications

Perceived Burden Due to Registrations for Quality Monitoring and Improvement in Hospitals: A Mixed Methods Study

Perceived Burden Due to Registrations for Quality Monitoring and Improvement in Hospitals: A Mixed Methods Study

Changes in health worker knowledge and motivation in the context of a quality improvement programme in Ethiopia

Assessment of Impostor Phenomenon in Student Pharmacists and Faculty at Two Doctor of Pharmacy Programs

Contact Info

Product

Resources

About