Self- and peer assessment may not be an accurate measure of PBL tutorial process

Abstract

BackgroundUniversidade Cidade de São Paulo adopted a problem-based learning (PBL) strategy as the predominant method for teaching and learning medicine. Self-, peer- and tutor marks of the educational process are taken into account as part of the final grade, which also includes assessment of content. This study compared the different perspectives (and grades) of evaluators during tutorials with first year medical students, from 2004 to 2007 (n = 349), from seven semesters.MethodsThe tutorial evaluation method was comprised of the students' self assessment (SA) (10%), tutor assessment (TA) (80%) and peer assessment (PA) (10%) to calculate a final educational process grade for each tutorial. We compared these three grades from each tutorial for seven semesters using ANOVA and a post hoc test.ResultsA total of 349 students participated with 199 (57%) women and 150 (42%) men. The SA and PA scores were consistently greater than the TA scores. Moreover, the SA and PA groups did not show statistical difference in any semester evaluated, while both differed from tutor assessment in all semesters (Kruskal-Wallis, Dunn's test). The Spearman rank order showed significant (p < 0.0001) and positive correlation for the SA and PA groups (r = 0.806); this was not observed when we compared TA with PA (r = 0.456) or TA with SA (r = 0.376).ConclusionPeer- and self-assessment marks might be reliable but not valid for PBL tutorial process, especially if these assessments are used for summative assessment, composing the final grade. This article suggests reconsideration of the use of summative assessment for self-evaluation in PBL tutorials.

Self-assessment: how do third year medical students rate their performance during problem-based learning?

“…28 Clearly self-scoring, doesn't always discriminate low and high-achievers. use of self-assessment scores as final summative scores, could promote more generous self-scoring, 4 affecting productivity of PBL. Perhaps it would be useful for our students, to have this formative exercise mid-course, not end of course.…”

“…1,9 Machado showed that, self-assessment scores only correlated with peer-assessment scores, not tutor scores: concluding that students' scores are not useful for summative assessment, but should only be used for formative purposes. 4 In Das et al study, 75% of students, and 83% of tutors echoed that, students must be able to grade their work accurately, for their grades to be reliable. 1,9 Reports on Falchikov and Boud's study of 1989, showed correlations between self-assessment and teacher-assessments ranged from -0.05 to 0.82 (mean 0.39) and Gordon's study on health professionals, showed correlations between self-assessment and actual factual knowledge ranged from 0.02 to 0.65.…”

“…13,21,22 Luciano even suggested a lack of transparency in assessments, or possibly unfamiliarity with the PBL process. 4 However, unlike in Luciano where the analysis included students from 1 st /2 nd years, this study, only involved third-year students who had done PBL for first year. 4 Students can be taught to assess themselves.…”

“…4 However, unlike in Luciano where the analysis included students from 1 st /2 nd years, this study, only involved third-year students who had done PBL for first year. 4 Students can be taught to assess themselves. KelberlauBerks showed that with regular reflection and guidance by teachers, even seventh-graders were accurate in selfassessment.…”

Self-monitoring in third year medical students’ haematology clerkships

“…High achieving students tend to under-rate themselves where-as low achieving students over-rate themselves in exercises where students have to rate their performance. [25][26][27][28][29][30] For quiz-responders, the mean scores were higher than non-responders in both the EOC-spotter and the PDQ. For the responders the correlation was higher between the quiz and free-response PDQ than between quiz and spotter (MCQ).…”