Use of an Analytical Grading Rubric for Self‐Assessment: A Pilot Study for a Periodontal Oral Competency Examination in Predoctoral Dental Education

Abstract

While educators agree that using self-assessment in education is valuable, a major challenge is the poor agreement often found between faculty assessment and student self-assessment. The aim of this study was to determine if use of a predeined grading rubric would improve reliability between faculty and dental student assessment on a periodontal oral competency examination. Faculty members used the grading rubric to assess students' performance on the exam. Immediately after taking the exam, students used the same rubric to self-assess their performance on it. Data were collected from all third-and/or fourth-year students in four classes at one U.S. dental school from 2011 to 2014. Since two of the four classes took the exam in both the third and fourth years, those data were compared to determine if those students' self-assessment skills improved over time. Statistical analyses were performed to determine agreement between the two faculty graders and between the students' and faculty assessments on each criterion in the rubric and the overall grade. Data from the upper and lower performing quartiles of students were sub-analyzed. The results showed that faculty reliability for the overall grades was high (K=0.829) and less so for individual criteria, while student-faculty reliability was weak to moderate for both overall grades (Spearman's rho=0.312) and individual criteria. Students in the upper quartile self-evaluated themselves more harshly than the faculty (p<0.0001), while the lower quartile students overestimated their performance (p=0.0445) compared to faculty evaluation. No signiicant improvement was found in assessment over time in the students who took the exam in the third and fourth years. This study found only limited support for the hypothesis that a grading rubric used by both faculty and students would increase correspondence between faculty and student assessment and points to a need to reexamine the rubric and instructional strategies to help students improve their ability to selfassess their work.

Agreement between faculty member assessments and student self‐assessments in a preclinical endodontic programme

“…The reliability of self‐assessment is the confusing issue in health science education . Consistent with the results of other studies, we found the reliability of students’ assessment to be weak compared to that of staff members . The drawback of the self‐assessment in the current study is the weakness of reliability in all steps that evaluated by students (Table ) that may be associated with a very little experience in the evaluation root canal treatment procedures.…”

Rubric system for evaluation of crown preparation performed by dental students

“…However, it was evident that majority of the examiners had some agreement/consistency amongst their awarded grades. This finding of the current study is in line with the findings of Satheesh et al who reported increased reliability of 90.2% with the use of analytic rubrics in their research study.…”

“…Since the mid‐1990s, there is a call for change in the way dental education is provided which is evident from a report published in 1995 by the Institute of Medicine, calling attention to the need for change in dental education …”

Introduction of integrated dental training jaw models and rubric criteria

“…Compared with the instructors, the students tended to underestimate their performance in both years. These results showed that the self‐assessments did not agree with those by the instructors, which is similar to findings in previous reports . Self‐assessment requires practice and experience .…”

“…Self‐assessment requires practice and experience . Satheesh et al previously reported that student‐faculty reliability was weak, while Abe et al reported that self‐assessment scores using rubric sheets were higher the second time compared with the first and that experience led to higher self‐assessment scores . The training course described in the present study represented the first opportunity for undergraduate students in 2014 and 2015 to assess their own performance using rubric sheets.…”