Validation of a method to measure resident doctors’ reflections on quality improvement

Abstract

Validity evidence supports MERIT as a meaningful measure of resident reflection on QI opportunities. Our findings suggest that dimensions of resident reflection on QI opportunities may include personal, system and Problem of Merit factors. Additionally, residents may be more effective at reflecting on 'problems of merit' than personal and systems factors.

Factor Analysis Methods and Validity Evidence

“…For instance, a number of authors suggested that their findings established an instrument's construct validity. However, from a contemporary 59,90 ). It is unclear why the transition from the traditional to the contemporary validity framework, which was introduced in 1999, has yet to occur in medical education.…”

Measuring Faculty Reflection on Adverse Patient Events: Development and Initial Validation of a Case-Based Learning System

“…The Mayo Evaluation of Reflection on Improvement Tool (MERIT) assesses resident physicians' reflections on adverse events encountered in practice. 22 However, MERIT reflections are scored by external raters, and are thus cumbersome and biased. The Kember instrument, which was the basis for this study's assessment tool, measures health sciences students' reflections on their courses.…”

“…An established validity paradigm states that construct validity is upheld by evidence from the following sources: content, response process, internal structure, relations to other variables (criterion), and consequences. [25][26][27][28][29][30] In this study, content evidence draws from items that were created based on a previously validated reflection assessment instrument, 15,16 findings from our prior research, 22 and revision by a panel of experts with experience in teaching and measuring reflection and QI. Internal Structure evidence is supported by factor analysis showing a two-dimensional assessment of physician reflection that generally verifies the reflection levels found in the original Kember instrument, 15,16 and by excellent overall internal consistency reliability.…”

Use of the Quality Improvement (QI) Knowledge Application Tool in Assessing Pediatric Resident QI Education

“…[4][5][6][7] However, few instruments exist that objectively assess gains in QI skills, knowledge, and behavior postinstruction. [8][9][10] A tool with evidence of preliminary validity was published during this study but has not been evaluated further. 11 The Quality Improvement Knowledge Application Tool (QIKAT) has been used to evaluate QI knowledge acquisition after curricular instruction.…”