Using video-cases to assess student reflection: Development and validation of an instrument

Abstract

BackgroundReflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners’ verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection.MethodsFourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students’ reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS®). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets.ResultsReflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff’s alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and ≥ 6 cases from 5th year students.ConclusionUse of StARS® to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.

Reclaiming a theoretical orientation to reflection in medical education research: a critical narrative review

“…The use of reflective assignments and portfolios has become more widespread in recent years, but the experimental evidence for their generalisable efficacy is limited . Perhaps because reflection is difficult to evaluate, another trend in the literature refers to reflection rubrics developed to facilitate assessment of portfolios …”

Learning styles and approaches: Can reflective strategies encourage deep learning?

“…2,69 For example, reflecting on activities such as counseling and dispensing, particularly with self-reflection through media such as video, can empower a student to view first hand his/her approaches to the task. 16 This type of self-reflection allows an "introspective" process 121 that can highlight both strengths and weaknesses of the developing student's skills and along with reflective writing; they can utilize problems encountered to develop their reflective thinking skills.…”

“…2 If learning outcomes and assessments are the driving force for a student to adopt a certain approach and deep learners are the students who achieve a level of mastery and reflection, then perhaps assessment requirements and teaching methods need to be modified to address this problem. A solution to foster and develop students to adopt a deep approach to learning would be (i) to incorporate reflective practices 73,[79][80][81][82] in pharmacy curricula 11 ; (ii) constructively align assessments to include a reflective component such as a reflective essay, 70,83 reflective statement, 71,84 reflective video, 16,73,[85][86][87] reflective blog, [88][89][90][91][92][93] reflective portfolio, 94-109 reflective journal, [110][111][112][113][114][115] or reflective diary 116,117 ; (iii) apply teaching methods to foster student engagement in learning tasks; (iv) for educators to provide clearly stated academic expectations and timely feedback. This will (i) encourage surface and strategic learners to develop skills in reflective practice, 73,118 (ii) foster their reflective thinking, 75,76 (iii) enhance application and integration of course material in practice, 11 (iv) improve clinical competency through clinical reasoning skills, and (v) enhance a deeper understanding of crucial concepts in pharmacy education, through enhanced student engagement.…”

The Effect of Reflective Activities on Reflective Thinking Ability in an Undergraduate Pharmacy Curriculum

“…Videos are an effective tool in health professions education to enhance communication skills and self and/or peer reflection. [79][80][81][82][83] Video podcasts may be a useful way for students to improve their counseling and clinical competency skills by reflecting on their own counseling shortcomings, as well as their strengths and those of a peer. Moreover, addressing their inner thoughts about and challenges from the whole task (including the video task) through reflective writing allows students to develop strategies to overcome barriers and concerns to improve future tasks.…”