Quality evaluation reports: Can a faculty development program make a difference?

Abstract

This study demonstrates that this FD workshop had a positive impact upon the quality of the participants' evaluation reports suggesting that faculty have the potential to be trained with regards to trainee assessment. This adds to the literature which suggests that FD is an important component in improving assessment quality.

Residents' narrative feedback on teaching performance of clinical teachers: analysis of the content and phrasing of suggestions for improvement

“…Furthermore, we suggest that residents add examples of daily practice, a location or a situational description, give a practical tip or an example, and explain the effect of the expected behaviour when this tip would be put into practice. Dudek and colleagues have shown positive results of feedback training on the quality of in-training evaluation reports on student and resident performance 29. We anticipate that when residents phrase the suggestions for improvement more specifically, this will result in more useful feedback for teachers.…”

Daily Encounter Cards—Evaluating the Quality of Documented Assessments

“…The CCERR has been used as a program evaluation tool to assess the impact of faculty development initiatives aimed at improving the quality of ITERs. 13,14 Our study suggests that it can be used in a similar manner for DECs. Additionally, because reliable scores can be obtained with a single rater and DEC quality can be evaluated relatively quickly (approximately 15 DECs per hour in this study), a program director could use the CCERR to provide individual feedback to clinical supervisors about the quality of their assessments documented on DECs.…”

“…9 It has repeatedly demonstrated excellent reliability and validity in determining ITER quality and has been used to evaluate interventions designed to improve ITER completion. 9,13,14 The items on the CCERR appear to be applicable to DECs, which share a similar structure to ITERs (list of items with a rating scale and written comments). However, the CCERR has thus far only been used to assess ITER quality.…”

“…However, the CCERR has thus far only been used to assess ITER quality. 9,13,14,19 Therefore, the purpose of this study is to provide validity evidence to support using the CCERR to assess the quality of DEC completion.…”

“…[9][10][11][12] Interventions to improve the quality of completion of WBAs have been designed, including faculty development workshops and rater training initiatives. [13][14][15] To evaluate the effectiveness of these interventions, a tool is needed that can measure outcomes beyond participant satisfaction. 16,17 This tool should be capable of measuring changes in assessor behavior that correspond to Kirkpatrick's third level of program evaluation.…”