Reliability of a grading rubric designed to evaluate the reflective ability of predoctoral dental students

Abstract

Introduction Reflection and self‐assessment are critical skills for healthcare providers. Identification of gaps in knowledge, skills and attitudes, along with the ability to critically think and problem solve to fill gaps, is the ultimate outcome for lifelong learning. The aims of this study were to (a) refine an instrument used for measuring reflective ability, and conduct comprehensive reliability testing, and (b) describe a process for rater calibration. Materials and Methods Students develop e‐portfolios over a four‐year span with assignments that require reflection and self‐assessment. The final piece of the portfolio includes a global reflection written the last semester of the programme. Three faculty raters independently evaluated 106 dental students’ global reflections using the revised grading rubric. An intraclass correlation coefficient measured the level of agreement between the three raters. Results Analysis of the 318 faculty ratings (106/rater) resulted in an intraclass correlation of .708. Based on a 5‐point grading scale (0 = does not respond to the assignment to 5 = reconstructing), the ratings of the 106 global reflections ranged from 1.3 to 5.0 (M = 3.1, SD =0.66). Discussion This study provides confidence in the reliability of a grading rubric designed to assess reflective ability, along with suggestions for calibration. An overall mean of 3.1 (Level 3 = relating—includes evidence of lessons learned) illustrates the complexity of teaching reflection and self‐assessment. Conclusion Use of a reliable grading rubric for assessing reflective writing could assist schools interested in incorporating reflection and self‐assessment into the curriculum, ultimately supporting lifelong and enhanced health care.

A reflective model: e‐Portfolios in undergraduate periodontology

“…To ensure grading consistency and facilitate learning, we developed a rubric composed of four performance categories (Figure 3) with behavioral anchors at each grade level and standardized templates to provide scaffolding. 4 With each case students provided proof of their measurement of radiographic bone loss and articulated all evidence considered in their staging and grading of the patient. Their case disposition summary and reflection included reasoning for determining treatment outcomes and future recommendations for their patient and for improving their performance.…”

“…Within Bulb, a homepage was designed to serve as a central repository for information including templates, videos, and validated resources (Figure 2). To ensure grading consistency and facilitate learning, we developed a rubric composed of four performance categories (Figure 3) with behavioral anchors at each grade level and standardized templates to provide scaffolding 4 . With each case students provided proof of their measurement of radiographic bone loss and articulated all evidence considered in their staging and grading of the patient.…”