Difficulty giving feedback on underperformance undermines the educational value of multi-source feedback

Ingram, John R.; Anderson, Elizabeth J.; Pugsley, Lesley

doi:10.3109/0142159x.2013.804910

Cited by 11 publications

(7 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When students, for example, respected the individual for his or her clinical competence, they would more readily accept the feedback. Despite some research suggesting that people eschew providing non-anonymous feedback on a student’s underperformance for fear that it will affect their working relationship (Castanelli and Kitto 2011 ; Ingram et al 2013 ), both research by Watling et al ( 2012 ) and the present research demonstrate that it is vital to know the feedback source in order to be able to assess the information’s credibility.…”

Section: Discussioncontrasting

confidence: 75%

From aggregation to interpretation: how assessors judge complex data in a competency-based portfolio

Pool

Govaerts

Jaarsma

et al. 2017

Adv in Health Sci Educ

View full text Add to dashboard Cite

While portfolios are increasingly used to assess competence, the validity of such portfolio-based assessments has hitherto remained unconfirmed. The purpose of the present research is therefore to further our understanding of how assessors form judgments when interpreting the complex data included in a competency-based portfolio. Eighteen assessors appraised one of three competency-based mock portfolios while thinking aloud, before taking part in semi-structured interviews. A thematic analysis of the think-aloud protocols and interviews revealed that assessors reached judgments through a 3-phase cyclical cognitive process of acquiring, organizing, and integrating evidence. Upon conclusion of the first cycle, assessors reviewed the remaining portfolio evidence to look for confirming or disconfirming evidence. Assessors were inclined to stick to their initial judgments even when confronted with seemingly disconfirming evidence. Although assessors reached similar final (pass–fail) judgments of students’ professional competence, they differed in their information-processing approaches and the reasoning behind their judgments. Differences sprung from assessors’ divergent assessment beliefs, performance theories, and inferences about the student. Assessment beliefs refer to assessors’ opinions about what kind of evidence gives the most valuable and trustworthy information about the student’s competence, whereas assessors’ performance theories concern their conceptualizations of what constitutes professional competence and competent performance. Even when using the same pieces of information, assessors furthermore differed with respect to inferences about the student as a person as well as a (future) professional. Our findings support the notion that assessors’ reasoning in judgment and decision-making varies and is guided by their mental models of performance assessment, potentially impacting feedback and the credibility of decisions. Our findings also lend further credence to the assertion that portfolios should be judged by multiple assessors who should, moreover, thoroughly substantiate their judgments. Finally, it is suggested that portfolios be designed in such a way that they facilitate the selection of and navigation through the portfolio evidence.

show abstract

Section: Discussioncontrasting

confidence: 75%

From aggregation to interpretation: how assessors judge complex data in a competency-based portfolio

Pool

Govaerts

Jaarsma

et al. 2017

Adv in Health Sci Educ

View full text Add to dashboard Cite

show abstract

“…Based on the widespread recognition of the utility of the Rasch model in the medical education literature, 16 - 18 it is also possible that while Rasch analysis was rarely aligned with a particular assessment style via the response matrices, it was still recognized as a useful means of evaluating the performance of an established standard setting method within a given school. For example, one respondent chose to distinguish between their “mainstream” standard setting method and the use of Rasch analysis “to look at equating/anchor/how well our estimates fit with ability over time.” Such practice at other schools may not have been fully captured by the response data.…”

Section: Discussionmentioning

confidence: 99%

Variation in assessment and standard setting practices across UK undergraduate medicine and the need for a benchmark

MacDougall

2015

Int J Med Educ

View full text Add to dashboard Cite

ObjectivesThe principal aim of this study is to provide an account of variation in UK undergraduate medical assessment styles and corresponding standard setting approaches with a view to highlighting the importance of a UK national licensing exam in recognizing a common standard.MethodsUsing a secure online survey system, response data were collected during the period 13 - 30 January 2014 from selected specialists in medical education assessment, who served as representatives for their respective medical schools.ResultsAssessment styles and corresponding choices of standard setting methods vary markedly across UK medical schools. While there is considerable consensus on the application of compensatory approaches, individual schools display their own nuances through use of hybrid assessment and standard setting styles, uptake of less popular standard setting techniques and divided views on norm referencing.ConclusionsThe extent of variation in assessment and standard setting practices across UK medical schools validates the concern that there is a lack of evidence that UK medical students achieve a common standard on graduation. A national licensing exam is therefore a viable option for benchmarking the performance of all UK undergraduate medical students.

show abstract

“…Developing such team collaboration skills, however, is difficult, and requires long-term ongoing dedication and reflection (Ezziane et al 2012;Chassin & Loeb 2013). Successfully developing and maintaining such team skills may help to overcome the difficulties most physicians experience in expressing intercollegial feedback, in particular on underperformance (Ingram et al 2013). …”

Section: Patient Safety-related Communicationmentioning

confidence: 99%

Preparedness for practice: A systematic cross-specialty evaluation of the alignment between postgraduate medical education and independent practice

et al. 2014

View full text Add to dashboard Cite

show abstract

Difficulty giving feedback on underperformance undermines the educational value of multi-source feedback

Abstract: A relative lack of developmental feedback limits the formative educational gains from MSF and could provide false reassurance that might reinforce negative behaviours.

Cited by 11 publications

References 32 publications

From aggregation to interpretation: how assessors judge complex data in a competency-based portfolio

From aggregation to interpretation: how assessors judge complex data in a competency-based portfolio

Variation in assessment and standard setting practices across UK undergraduate medicine and the need for a benchmark

Preparedness for practice: A systematic cross-specialty evaluation of the alignment between postgraduate medical education and independent practice

Contact Info

Product

Resources

About