Instruments to measure the ability to self‐reflect: A systematic review of evidence from workplace and educational settings including health care

Ireland, Tony; Warman, Sheena M; Cake, Martin; Dymock, David; Fowler, Ellayne; Baillie, Sarah

doi:10.1111/eje.12445

Cited by 7 publications

(13 citation statements)

References 139 publications

(392 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rubrics are used in education to decrease rater bias and increase validity and reliability of ratings when there are multiple raters used to evaluate an assignment. This research is in line with a recent systematic review looking at how to measure the ability to self‐reflect, with grading rubrics identified as one of three types of recommended assessment instruments 15 . The faculty raters commented frequently about the students’ writing style.…”

Section: Discussionsupporting

confidence: 82%

Reliability of a grading rubric designed to evaluate the reflective ability of predoctoral dental students

Heathman

Mitchell

Ness

et al. 2021

Eur J Dental Education

View full text Add to dashboard Cite

Introduction Reflection and self‐assessment are critical skills for healthcare providers. Identification of gaps in knowledge, skills and attitudes, along with the ability to critically think and problem solve to fill gaps, is the ultimate outcome for lifelong learning. The aims of this study were to (a) refine an instrument used for measuring reflective ability, and conduct comprehensive reliability testing, and (b) describe a process for rater calibration. Materials and Methods Students develop e‐portfolios over a four‐year span with assignments that require reflection and self‐assessment. The final piece of the portfolio includes a global reflection written the last semester of the programme. Three faculty raters independently evaluated 106 dental students’ global reflections using the revised grading rubric. An intraclass correlation coefficient measured the level of agreement between the three raters. Results Analysis of the 318 faculty ratings (106/rater) resulted in an intraclass correlation of .708. Based on a 5‐point grading scale (0 = does not respond to the assignment to 5 = reconstructing), the ratings of the 106 global reflections ranged from 1.3 to 5.0 (M = 3.1, SD =0.66). Discussion This study provides confidence in the reliability of a grading rubric designed to assess reflective ability, along with suggestions for calibration. An overall mean of 3.1 (Level 3 = relating—includes evidence of lessons learned) illustrates the complexity of teaching reflection and self‐assessment. Conclusion Use of a reliable grading rubric for assessing reflective writing could assist schools interested in incorporating reflection and self‐assessment into the curriculum, ultimately supporting lifelong and enhanced health care.

show abstract

Section: Discussionsupporting

confidence: 82%

Reliability of a grading rubric designed to evaluate the reflective ability of predoctoral dental students

Heathman

Mitchell

Ness

et al. 2021

Eur J Dental Education

View full text Add to dashboard Cite

show abstract

“…PRISM was found to be superior against numeric scale regarding self-reflection in the current pilot study. A numeric scale (or scoring) as rubric tool for self-reflection has been evaluated in several studies [ 7 ]. A systematic review did not confirm a superiority of any instrument [ 7 ].…”

Section: Discussionmentioning

confidence: 99%

“…Different approaches are available, e.g., self-reflection combined with peer-feedback, video-watching, or expert-feedback [ 9 , 10 ]. Furthermore, a recent systematic review reported on the available instruments to measure the ability of self-reflection [ 7 ]. Thereby, rubrics (or scoring guides), self-reported scales, and observed behavior were differentiated, and it has been concluded that none of these tools can be recommended as single measure [ 7 ].…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, a recent systematic review reported on the available instruments to measure the ability of self-reflection [ 7 ]. Thereby, rubrics (or scoring guides), self-reported scales, and observed behavior were differentiated, and it has been concluded that none of these tools can be recommended as single measure [ 7 ]. Thus, based on this systematic review and to the authors' knowledge, there is no self-reflection method widely acknowledged as the ‘gold standard.'…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PRISM: A Novel Visual Instrument to Facilitate Self-Reflection and Learning Progress in Undergraduate Dental Education

Schmalz

Sensky

Kullmann

et al. 2022

BioMed Research International

View full text Add to dashboard Cite

Objectives. PRISM (Pictorial Representation of Illness and Self-Measure) is a simple visual tool that has been successfully used as a visual metaphor in medicine. In this pilot study, PRISM was used for the first time to test its potential to support self-reflection and expectations of learning in dental students. Methods. Dental student volunteers (25 3rd year, 10 4th year, and 10 5th year) participated. Using both quantitative and qualitative methods, PRISM interviews were compared with a numerical scale in assessing learning objectives concerning theoretical knowledge, practical skills, interest, and training need in the field of conservative dentistry. Results. Overall, 71% of total student group stated that they would draw personal consequences for their studies due to participating in the PRISM interviews. Compared to the numeric scales, PRISM was rated as more helpful regarding appraisal of students’ theoretical knowledge ( p = 0.02 ), practical skills ( p < 0.01 ), training needs ( p < 0.01 ), importance of dental subspecialties ( p < 0.01 ), and facilitating self-reflection ( p = 0.02 ). In focus groups, students commented that PRISM fostered the development of a trusting relationship with their teacher. Strengths of PRISM mentioned by the students included being able to observe and manipulate a visual summary of their individual learning needs and seeing their different learning needs in relation to one another. Conclusion. In this pilot study, dental students evaluated PRISM to be superior against numeric scales. Furthermore, it ameliorated the communication with teachers. The PRISM task is both simple and brief and warrants further exploration as a useful tool for self-reflection in dental education.

show abstract

“…However, this merit is based on a consensus on the concept of reflection in order to subject it to solid empirical testing. From the dynamic perspective, this consensus still remains an ideal (Williams et al, 2019 )–or rather, an impossibility. To further our understanding of both positions, we juxtapose the technicist and dynamic approaches.…”

Section: Discussionmentioning

confidence: 99%

Is reflection like soap? a critical narrative umbrella review of approaches to reflection in medical education research

Schaepkens

Veen

Croix

2021

Adv in Health Sci Educ

View full text Add to dashboard Cite

Reflection is a complex concept in medical education research. No consensus exists on what reflection exactly entails; thus far, cross-comparing empirical findings has not resulted in definite evidence on how to foster reflection. The concept is as slippery as soap. This leaves the research field with the question, ‘how can research approach the conceptual indeterminacy of reflection to produce knowledge?’. The authors conducted a critical narrative umbrella review of research on reflection in medical education. Forty-seven review studies on reflection research from 2000 onwards were reviewed. The authors used the foundational literature on reflection from Dewey and Schön as an analytical lens to identify and critically juxtapose common approaches in reflection research that tackle the conceptual complexity. Research on reflection must deal with the paradox that every conceptualization of reflection is either too sharp or too broad because it is entrenched in practice. The key to conceptualizing reflection lies in its use and purpose, which can be provided by in situ research of reflective practices.

show abstract

Instruments to measure the ability to self‐reflect: A systematic review of evidence from workplace and educational settings including health care

Cited by 7 publications

References 139 publications

Reliability of a grading rubric designed to evaluate the reflective ability of predoctoral dental students

Reliability of a grading rubric designed to evaluate the reflective ability of predoctoral dental students

PRISM: A Novel Visual Instrument to Facilitate Self-Reflection and Learning Progress in Undergraduate Dental Education

Is reflection like soap? a critical narrative umbrella review of approaches to reflection in medical education research

Contact Info

Product

Resources

About