Assessment methods in surgical training in the United Kingdom

Abstract

A career in surgery in the United Kingdom demands a commitment to a long journey of assessment. The assessment methods used must ensure that the appropriate candidates are selected into a programme of study or a job and must guarantee public safety by regulating the progression of surgical trainees and the certification of trained surgeons. This review attempts to analyse the psychometric properties of various assessment methods used in the selection of candidates to medical school, job selection, progression in training, and certification. Validity is an indicator of how well an assessment measures what it is designed to measure. Reliability informs us whether a test is consistent in its outcome by measuring the reproducibility and discriminating ability of the test. In the long journey of assessment in surgical training, the same assessment formats are frequently being used for selection into a programme of study, job selection, progression, and certification. Although similar assessment methods are being used for different purposes in surgical training, the psychometric properties of these assessment methods have not been examined separately for each purpose. Because of the significance of these assessments for trainees and patients, their reliability and validity should be examined thoroughly in every context where the assessment method is being used.

Training, assessment and accreditation in surgery

“…2 Summative assessments are used for selection and certification. Figure 4 highlights various points of assessment during surgical training in the UK, the most important of which are the Member of the Royal College of Surgeons (MRCS) and Fellow of the Royal College of Surgeons (FRCS) examinations.…”

“…2 Research has shown that the mini-clinical evaluation exercise (mini-CEX), case-based discussion (CbD), direct observation of procedural skills (DOPS) and the mini-peer assessment tool (mini-PAT) are feasible, reliable and valid assessment methods 50–52. Observational tools such as the use of procedure-based assessments (PBA) during real procedures and objective structured assessment of technical skills (OSATS) during simulated procedures have also shown good validity and reliability 53…”

“…Surgical certification examinations have been established in order to safeguard patients and ensure high standards for practising surgeons 59. Of these, the MRCS examination assesses the acquisition of knowledge, skills and attributes required for the completion of core surgical training, while the FRCS assesses these for completion of specialist training 2. Similarly, in Australia and New Zealand, after completion of all board requirements, including demonstration of clinical and operative expertise (via a logbook) and a research project, trainees are certified with the award of Fellowship of the Royal Australasian College of Surgeons (FRACS) 60.…”

An audit of the outcomes of the College of Surgeons general surgery final examinations

“…The manner in which both undergraduate and postgraduate medical graduates are assessed has been subject to extensive scrutiny and review in recent years. [1][2][3][4][5] The traditional format of examination with written essay-type papers, clinical cases and oral viva voce examinations have been criticised by educationalists and found to be significantly flawed. 6,7 These formats of examination are unfortunately extremely subjective and lack the psychometric properties of a good examination such as reliability, validity, applicability and acceptability.…”

Methodological shortcomings in the literature evaluating the role and applications of 3D training for surgical trainees

“…Surgical performance, competence, and skill acquisition can be measured through various assessment methods, broadly categorised into formative and summative, where former is aimed at providing feedback on the performance and latter is used to formally assess the acquisition of the skills required for completion of the training (Evgeniou 2013). The majority of assessment tools in surgical training are subjective and most of these tools are checklists (Alkhayal et al 2012), for example the Structured Technical Skills Assessment Form (STSAF) (Winckel et al 1994).…”