Assessment of Learner Metacognition in a Professional Pharmacy Elective Course

Abstract

Objective. To investigate the effect of strategic feedback and metacognitive processes on learners' ability to predict performance and improve self-awareness. Methods. Strategic faculty and peer feedback, as well as self-assessments, were implemented in a professional pharmacy elective course throughout the semester, focused on three case-based oral presentations. After each presentation, students utilized an objective rubric to determine self-predicted and peer-predicted scores. Actual scores from faculty were compared to students' predicted scores. Results. Students' ability to predict presentation scores did not improve over time; however, students were able to accurately estimate performance in certain rubric sections on individual presentations (depth of problem, presentation). Students were generally overconfident in predicting their performance. When broken down into tertiles, top performing students were more accurate in their self-assessments compared to bottom performing students. Bottom performing students were highly overconfident in their assessment. Conclusion. Self-awareness is essential for professionals, though difficult to cultivate and improve in one semester. Incorporating longitudinal, continuous feedback and metacognitive skills may help learners become more aware of their own performance and devise a plan for enhancement.

Changes in Pharmacy Students’ Metacognition Through Self-Evaluation During Advanced Pharmacy Practice Experiences

“…1 In several studies, pharmacy learners have been found to overestimate their performance when compared to the results on evaluations conducted by faculty members or preceptors. [4][5][6][7] This is consistent with findings in medical and nursing students. 8,9 Additionally, top performers have been found to more accurately predict or evaluate their performance than bottom (third or fourth quartile) performers in studies of pharmacy and medical students, and underperformers tend to highly overestimate their knowledge and skills.…”

“…Previous studies have shown that learners frequently have poor self-assessment skills and thus, likely poor metacognition skills. [4][5][6][7]13 This is often demonstrated by a poor correlation between student self-assessments and preceptor evaluations and has been demonstrated in many of the health professions, including medicine, nursing, and pharmacy. 4,10,13,14 There are limited data assessing the development of metacognition over time, particularly during APPEs.…”

“…8,9 Additionally, top performers have been found to more accurately predict or evaluate their performance than bottom (third or fourth quartile) performers in studies of pharmacy and medical students, and underperformers tend to highly overestimate their knowledge and skills. 4,5,8 Findings in some studies suggest that while overall performance scores might be similar between instructor and learner, particular areas of rubrics tend to have larger discrepancies in scores. Examples of rubric items reported with higher score discrepancies include empathy, team relationships, industriousness and enthusiasm, personal attitudes, and global knowledge.…”

“…Despite the improvement seen, this study found that one semester was not enough time to fully realize the effects of interventions on metacognition. 4 Mort and Hansen studied the impact of video review on first-year student pharmacists' self-assessment of communication skills following a patient counseling encounter. 6 Using the same rubric as faculty evaluators, student pharmacists evaluated their encounter prior-to and after viewing the recording.…”

A Student-Led Elective Provides Quality Improvement Feedback for a Required Compounding Course

“…There is a body of evidence that has shown knowledge acquisition and retention can be enhanced if a course is designed to include elements crucial to the development and practice of metacognition, [14][15][16][17][18][19][20] including critical thinking, self-directed activities, repetition, [21][22][23] problem solving, 16,18 and reflection. [24][25][26][27][28] In addition, research suggests people become experts in a subject by learning to solve problems that become progressively more complex over time. 29 The inclusion of higher-order thinking elements prepares the students much more effectively for the role of a compounding pharmacist.…”