BACKGROUND: Residency programs involve trainees in quality improvement (QI) projects to evaluate competency in systems-based practice and practice-based learning and improvement. Valid approaches to assess QI proposals are lacking. OBJECTIVE:We developed an instrument for assessing resident QI proposals-the Quality Improvement Proposal Assessment Tool (QIPAT-7)-and determined its validity and reliability.DESIGN: QIPAT-7 content was initially obtained from a national panel of QI experts. Through an iterative process, the instrument was refined, pilot-tested, and revised.PARTICIPANTS: Seven raters used the instrument to assess 45 resident QI proposals. MEASUREMENTS:Principal factor analysis was used to explore the dimensionality of instrument scores. Cronbach's alpha and intraclass correlations were calculated to determine internal consistency and interrater reliability, respectively. RESULTS: QIPAT-7 items comprised a single factor (eigenvalue=3.4) suggesting a single assessment dimension. Interrater reliability for each item (range 0.79 to 0.93) and internal consistency reliability among the items (Cronbach's alpha=0.87) were high. CONCLUSIONS:This method for assessing resident physician QI proposals is supported by content and internal structure validity evidence. QIPAT-7 is a useful tool for assessing resident QI proposals. Future research should determine the reliability of QIPAT-7 scores in other residency and fellowship training programs. Correlations should also be made between assessment scores and criteria for QI proposal success such as implementation of QI proposals, resident scholarly productivity, and improved patient outcomes.KEY WORDS: quality improvement; systems-based practice; practicebased learning and improvement; assessment; evaluation study; validation study. INTRODUCTIONThe Accreditation Council for Graduate Medical Education (ACGME) released its Outcome Project in 1999 requiring residents and fellows to demonstrate competency in systemsbased practice (SBP) and practice-based learning and improvement (PBLI).1 Involvement of trainees in quality improvement (QI) efforts has been proposed as a means of addressing these two competencies. Indeed, resident physicians are in an excellent position to identify health care systems improvements, 2-4 yet they often lack the skills to identify the best interventions for improvement and the time and mentorship necessary to implement them. Consequently, many internal medicine residencies have developed QI curricula. We searched the literature for examples of health systems and QI curricula and their assessment. 3,5-10 Some published curricula involve seminar and lecture formats that are typically evaluated using pre-and posttests of knowledge. 5,7,8,10 Other curricula incorporate independent-study QI proposals and/or QI projects.3,5-10However, reports of QI project assessments, and QI proposals in particular, are uncommon. Like any thoughtful research or business endeavor, a good QI project begins with a written proposal that explains the problem, hypoth...

No abstract

No abstract

No abstract

Validity evidence supports MERIT as a meaningful measure of resident reflection on QI opportunities. Our findings suggest that dimensions of resident reflection on QI opportunities may include personal, system and Problem of Merit factors. Additionally, residents may be more effective at reflecting on 'problems of merit' than personal and systems factors.