Desire paths for workplace assessment in postgraduate anaesthesia training: analysing informal processes to inform assessment redesign

Weller, Jennifer; Coomber, Ties; Chen, Yan; Castanelli, Damian

doi:10.1016/j.bja.2022.03.013

Cited by 5 publications

(1 citation statement)

References 43 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Згідно з даними авторів [18], аналіз неформальних процесів для перепланування оцінювання навичок по-винен вирішувати такі дилеми у прагненні отримати автентичну та репрезентативну інформацію про стажера: 1) як процес збору та документування інформації може фільтрувати, трансформувати або обмежувати вихідний результат; 2) визначення можливих відхилень результатів стажера від норм продуктивності; 3) як відкритий чи таємний збір інформації впливає на інформацію, яку нададуть супервайзери, і як контроль над оцінюванням розподіляється між стажером і супервайзером.…”

Section: вступunclassified

Workplace-based assessment for anesthesia residents: efficacy and competence issues

Bielka,

Kuchyn,

Fomina

et al. 2023

ЕМ

View full text Add to dashboard Cite

Background. Workplace-based assessment (WPBA) is a new technology for evaluating medical residents that is believed to be more effective than traditional multiple-choice tests or exam-based assessment methods. The aim of the study was to investigate the WPBA efficacy in final-year medical residents: multi-source feedback (360°) assessment, direct procedure assessment (Anaesthesia-Clinical Evaluation Exercise (A-CEX)) and clinical case-based assessment, and to identify difficult competencies, which graduate residents acquire the least successfully. Materials and methods. From April 1 to June 15, 2023, 36 residents were evaluated at the workplace: the multi-source feedback 360°-evaluation form was filled out by three representatives of the medical team where the resident worked during the on-site part of the residency (1 anesthesiologist, 1 nurse and 1 other specialist); the form of direct procedure assessment was filled out by the teacher on face-to-face bases directly during the resident’s performance of the general anesthesia in low-risk patients (American Society of Anesthesiologists (ASA) I or II), preoperative examination of low-risk patients (ASA I or II) and neuraxial anesthesia in low-risk patients (ASA I or II) with the following feedback; assessment based on a clinical case was carried out by teachers during the conference of residents, where each of them presented a clinical case of anesthetic management or treatment of a critically ill patient. Results. According to the results of the multi-source feedback (360°) evaluation, most of the residents have received 7 or more points out of 10 possible. Communication with the medical team (odds ratio (OR) 1.9 [1.05–3.5], p = 0.048) and management in a critical situation (OR 2 [1.14–3.83], p = 0.024) were most difficult competencies during the multi-source feedback (360°) assessment. During direct procedure of general anesthesia, the most difficult competencies were: 1) management in a critical situation — knowledge/recognition of risks and how to avoid and treat them (OR 5.29 [1.9–14.4], p = 0.001 compared to documentation); 2) solving problems, making decisions (OR 12.6 [1.5–10,4], p = 0.007 compared to documentation); 3) interaction in the team (OR 2.7 [1.1–6.5], p = 0.049). No problematic competencies were identified during the assessment of neuraxial anesthesia competencies and preoperative examination. Conclusions. WPBA techniques such as multi-source feedback (360°) assessment, direct procedure evaluation (A-CEX) and clinical case-based assessment are effective in final year residents, improving their communication skills, readiness to work independently, help identify skill deficiencies. Communication with the medical team and management in a critical situation were difficult competencies during the multi-source feedback (360°) evaluation. Difficult competences during the direct evaluation of general anesthesia turned out to be the management in a critical situation; solving problems, making decisions; interaction in the team. No problematic competencies were identified during the assessment of neuraxial anesthesia competencies and preoperative examination.

show abstract

Section: вступunclassified

Workplace-based assessment for anesthesia residents: efficacy and competence issues

Bielka,

Kuchyn,

Fomina

et al. 2023

ЕМ

View full text Add to dashboard Cite

show abstract

Assessing trainee's performance using reported observations: Perceptions of nurse meta-assessors

Blanchette

Poitras

St‐Onge

2023

Nurse Education Today

View full text Add to dashboard Cite

“Praise in Public; Criticize in Private”: Unwritable Assessment Comments and the Performance Information That Resists Being Written

Gingerich,

Lingard,

Sebok-Syer

et al. 2024

Acad Med

View full text Add to dashboard Cite

Purpose Written assessment comments are needed to archive feedback and inform decisions. Regrettably, comments are often impoverished, leaving performance-relevant information undocumented. Research has focused on content and supervisor’s ability and motivation to write it but has not sufficiently examined how well the undocumented information lends itself to being written as comments. Because missing information threatens the validity of assessment processes, this study examined the performance information that resists being written. Method Two sequential data collection methods and multiple elicitation techniques were used to triangulate unwritten assessment comments. Between November 2022 and January 2023, physicians in Canada were recruited by email and social media to describe experiences with wanting to convey assessment information but feeling unable to express it in writing. Fifty supervisors shared examples via survey. From January to May 2023, a subset of 13 participants were then interviewed to further explain what information resisted being written and why it seemed impossible to express in writing and to write comments in response to a video prompt or for their own “unwritable” example. Constructivist grounded theory guided data collection and analysis. Results Not all performance-relevant information was equally writable. Information resisted being written as assessment comments when it would require an essay to be expressed in writing, belonged in a conversation and not in writing, or was potentially irrelevant and unverifiable. In particular, disclosing sensitive information discussed in a feedback conversation required extensive recoding to protect the learner and supervisor-learner relationship. Conclusions When written comment requests to document performance information are viewed as an act of disclosure, it becomes clear why supervisors may feel compelled to leave some comments unwritten. Although supervisors can be supported in writing better assessment comments, their failure to write invites a reexamination of expectations for documenting feedback and performance information as written comments on assessment forms.

show abstract

Desire paths for workplace assessment in postgraduate anaesthesia training: analysing informal processes to inform assessment redesign

Cited by 5 publications

References 43 publications

Workplace-based assessment for anesthesia residents: efficacy and competence issues

Workplace-based assessment for anesthesia residents: efficacy and competence issues

Assessing trainee's performance using reported observations: Perceptions of nurse meta-assessors

“Praise in Public; Criticize in Private”: Unwritable Assessment Comments and the Performance Information That Resists Being Written

Contact Info

Product

Resources

About