Variations in assessment and certification in postgraduate anaesthesia training: a European survey

Jonker, Gersten; Manders, Lieke Ananda; Marty, Adrian; Kalkman, Cor J.; Cate, Olle ten; Gessel, Elisabeth F. Van; Hoff, Reinier G.

doi:10.1093/bja/aex196

Cited by 30 publications

(14 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…16 A survey of anaesthesia training in the USA, UK, Canada, Japan, Denmark, and Switzerland showed that, whilst the surveyed countries purported to adhere to competency-based curricula, they had quite different structures and training programmes. 17 Similar findings were reported in a survey of 36 European countries, 18 with concerns that variable anaesthesia training in Europe resulted in variable graduate outcomes and a potentially detrimental effect on regional standards.…”

Section: Developing a Consensus On An Achievable Cbme Curriculumsupporting

confidence: 69%

Systematic review and narrative synthesis of competency-based medical education in anaesthesia

Weller

Naik

Diego

2020

British Journal of Anaesthesia

View full text Add to dashboard Cite

Background: Competency-based medical education (CBME) addresses the accountability of postgraduate training programmes to graduate specialists capable of independent practice. Methods: We undertook a systematic review and narrative synthesis of the published CBME literature in anaesthesia training programmes to identify current practices and areas requiring further exploration. Results: We grouped the 23 studies that met our inclusion criteria into the following categories: demonstrating outcomes of CBME, developing a consensus on an achievable CBME curriculum, CBME curriculum framework, design and implementation of workplace-based assessment (WBA) tools, trainee self-assessment, perceptions of trainees and supervisors on WBA tools, and technological solutions for assessment and feedback. Included studies reported variable success in reaching consensus in competency outcome frameworks for sequenced progression and limited research on approaches to curriculum delivery, whilst the majority of studies focused on workplace assessment. Studies supported the use of entrustment scales, where assessors make a judgement on the extent to which the trainee can manage a case independently. While evidence supported the reliability of WBA tools, and predicted the numbers needed for high-stakes decisions, areas of concern related to factors influencing the value WBA tools in promoting trainee learning, and variable perceptions of their value in making decisions on progression. Conclusions: Evidence on outcomes of CBME was limited to acquisition of specific competencies during training. The large number of unanswered questions and the dearth of studies across the core components of CBME suggest that we need a collaborative approach to create the evidence required to implement CBME wisely and cost effectively, to have positive impacts on patients, trainees, and healthcare systems.

show abstract

Section: Developing a Consensus On An Achievable Cbme Curriculumsupporting

confidence: 69%

Systematic review and narrative synthesis of competency-based medical education in anaesthesia

Weller

Naik

Diego

2020

British Journal of Anaesthesia

View full text Add to dashboard Cite

show abstract

“…The present study has identified some major inconsistencies in the types of assessments used, both for summative and formative assessment in respiratory medicine training. Similar variability across Europe was recently reported for anaesthetic training [15]. There are a number of deficiencies in our study: it is incomplete, sampling only just over half of the target respondents, although it has information from 73% of the target countries, representing 88% of the population.…”

Section: Discussionmentioning

confidence: 69%

Assessment methods in respiratory medicine training in Europe: current status and future needs

Primhak

2020

Breathe

View full text Add to dashboard Cite

There is a perceived need for harmonisation of training standards in medicine across Europe. Assessment methodology is a fundamental part of this harmonisation. Assessment may be: summative, measuring competency at the end of training, usually in a pass/fail mode; or formative, assessing strengths and weaknesses during training with a view to optimising performance. A survey of experienced respiratory medicine trainers in Europe was undertaken to discover the formative and summative assessment methods being used, and to explore the use of workplace-based assessments (WBAs). Structured interviews were sought with experienced trainers in adult and paediatric respiratory medicine. 35 trainers from 22 (71%) out of 31 countries were interviewed.The number of types of summative assessments required at the end of training varied from 0 to 4 including clinical (25%), written (44%) and oral (47%) examinations. Four respondents required a research thesis as a proof of clinical competence.WBA was not commonly used. Only 14% of respondents reported using a formal case-based discussion, 20% used a form of multisource feedback and 25% described some form of formal assessment of procedural skills. However, 77% of all respondents expressed a wish to have access to case-based discussion and 72% wanted to have a facility for multisource feedback. The majority also wanted training in the use of these tools. Almost half of the respondents had received no formal training in educational supervision and 80% of all respondents expressed a wish to receive such training.The findings suggest that there is no adequate process of ensuring uniform standards for specialist accreditation in Europe, and demonstrate a need and desire among trainers for more WBA tools and training in their use to be made available.

show abstract

“…The introduction of an OSCE to the clinical component follows the international trend towards a more competencybased certification process. 12 To our knowledge this will be the first postgraduate orthopaedic surgery exit examination in Africa to include an OSCE as part of the certifying process. Currently the essay-style questions are in a process of being phased out of the written component of the FC (Orth)SA final examination and have been replaced by the more reliable and reproducible MCQ format.…”

Section: Discussionmentioning

confidence: 99%

The FC Orth(SA) final examination: how effective is the written component?

et al. 2018

View full text Add to dashboard Cite

Background: To determine the pass rate of the final exit examination of the College of Orthopaedic Surgeons of South Africa [FC Orth(SA)] and to assess the correlation between the written component with the clinical and oral component. Methods: Results of candidates who participated in the FC Orth(SA) final examination during a 12-year period from March 2005 through to November 2016 were assessed retrospectively. Pass rates and component averages were analysed using descriptive and inferential statistics. Spearman's rho test was used to determine the correlation between the components. Results: A total of 399 candidates made 541 attempts at the written component of the examination; 71.5% of attempts were successful and 387 candidates were invited to the clinical and oral component, of which 341 (88%) candidates were certified. The second-attempt pass rate for those candidates who wrote the written component again was 42%. The average annual increase in the number of certified candidates was 8.5%. The overall certifying rate increased by 1.5% for this period. Invited candidates who scored less than 54% for the written component were at significant risk of failing the clinical and oral component. The written component showed weak correlation with the clinical and oral component (r=0.48). Conclusion: While the written component was found to be an effective gatekeeper, as evidenced by a high eventual certifying rate, the results of this component of the FC Orth(SA) final examination did not correlate strongly with the performance in the clinical and oral component. This finding confirms the value of the written component as part of a comprehensive assessment for the quality of orthopaedic surgeons.

show abstract

Variations in assessment and certification in postgraduate anaesthesia training: a European survey

Cited by 30 publications

References 17 publications

Systematic review and narrative synthesis of competency-based medical education in anaesthesia

Systematic review and narrative synthesis of competency-based medical education in anaesthesia

Assessment methods in respiratory medicine training in Europe: current status and future needs

The FC Orth(SA) final examination: how effective is the written component?

Contact Info

Product

Resources

About