Analysis of Tools Used in Assessing Technical Skills and Operative Competence in Trauma and Orthopaedic Surgical Training

James, Hannah; Chapman, Anna W.P.; Pattison, Giles; Fisher, Joanne; Griffin, Damian R.

doi:10.2106/jbjs.rvw.19.00167

Cited by 25 publications

(28 citation statements)

References 116 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, the use of assessment tools through the spectrum of surgical skills development (simulated through live surgery) allows competency benchmarks to be defined as having been reached or not reached (step 4 of CBME development). However, a recent systematic review showed that 85% of assessment tools are limited to use in the simulation environment only 25 . These tools use a variety of methods, and a separate review by Atesok et al detailed specific strengths and weaknesses of a range of assessment tools 26 .…”

Section: Cbmementioning

confidence: 99%

Contemporary Issues in the Acquisition of Orthopaedic Surgical Skills During Residency

Myers

Marsh

Nicandri

et al. 2021

Journal of Bone and Joint Surgery

View full text Add to dashboard Cite

ä Orthopaedic education should produce surgeons who are competent to function independently and can obtain and maintain board certification.ä Contemporary orthopaedic training programs exist within a fixed 5-year time frame, which may not be a perfect match for each trainee.ä Most modern orthopaedic residencies have not yet fully adopted objective, proficiency-based, surgical skill training methods despite nearly 2 decades of evidence supporting the use of this methodology.ä Competency-based medical education backed by surgical simulation rooted in proficiency-based progression has the potential to address surgical skill acquisition challenges in orthopaedic surgery.Disclosure: The Disclosure of Potential Conflicts of Interest forms are provided with the online version of the article (http://links.lww.com/JBJS/G773).

show abstract

Section: Cbmementioning

confidence: 99%

Contemporary Issues in the Acquisition of Orthopaedic Surgical Skills During Residency

Myers

Marsh

Nicandri

et al. 2021

Journal of Bone and Joint Surgery

View full text Add to dashboard Cite

show abstract

“…Procedure time has been chosen as an outcome measure as there is evidence in the literature that procedure time is inversely related to experience, and so can be used as a surrogate measure of technical proficiency. 14…”

Section: Clinical Outcomesmentioning

confidence: 99%

“…There is evidence that with increasing experience and skill, surgeons use less intraoperative X-rays to adjust the position of the fracture and implant. 14 Hemiarthroplasty does not require fluoroscopy so this will not be used as an outcome measure for substudy 2.…”

Section: Outcome Measuresmentioning

confidence: 99%

Cadaveric simulation versus standard training for postgraduate trauma and orthopaedic surgical trainees: protocol for the CAD:TRAUMA study multicentre randomised controlled educational trial

et al. 2020

View full text Add to dashboard Cite

IntroductionThe quantity and quality of surgical training in the UK has been negatively affected by reduced working hours and National Health Service (NHS) financial pressures. Traditionally surgical training has occurred by the master-apprentice model involving a process of graduated responsibility, but a modern alternative is to use simulation for the early stages of training. It is not known if simulation training for junior trainees can safeguard patients and improve clinical outcomes. This paper details the protocol for a multicentre randomised controlled educational trial of a cadaveric simulation training intervention versus standard training for junior postgraduate orthopaedic surgeons-in-training. This is the first study to assess the effect of cadaveric simulation training for open surgery on patient outcome. The feasibility of delivering cadaveric training, use of radiographic and clinical outcome measures to assess impact and the challenges of upscaling provision will be explored.Methods and analysisWe will recruit postgraduate orthopaedic surgeons-in-training in the first 3 years (of 8) of the specialist training programme. Participants will be block randomised and allocated to either cadaveric simulation or standard ‘on-the-job’ training, learning three common orthopaedic procedures, each of which is a substudy within the trial. The procedures are (1) dynamic hip screw, (2) hemiarthroplasty and (3) ankle fracture fixation. These procedures have been selected as they are very common procedures which are routinely performed by junior surgeons-in-training. A pragmatic approach to sample size is taken in lieu of a formal power calculation as this is novel exploratory work with no a priori estimate of effect size to reference. The primary outcome measure is the technical success of the surgery performed on patients by the participating surgeons-in-training during the follow-up period for the three substudy procedures, as measured by the implant position on the postoperative radiograph. The secondary outcome measures are procedure time, postoperative complication rate and patient health state at 4 months postoperation (EQ-5D—substudies 1 and 2 only).Ethics, registration and disseminationNational research ethics approval was granted for this study by the NHS Research Authority South Birmingham Research Ethics Committee (15/WM/0464). Confidentiality Advisory Group approval was granted for accessing radiographic and outcome data without patient consent on 27 February 2017 (16/CAG/0125). The results of this trial will be submitted to a peer-reviewed journal and will inform educational and clinical practice.Trial registration numberISRCTN20431944

show abstract

“…Measurement of technical skill in the real-world orthopaedic theatre is fraught with methodological challenges, and a recent systematic review showed that none of the technical skills assessment tools in current use in orthopaedic training around the world satisfy the Norcini criteria for effective assessment. 13 Most have not been validated beyond the simulated environment, and are unsuitable for use in real life due to reliance on simulator-derived metrics for assessment of technical skill.…”

Section: Introductionmentioning

confidence: 99%

Assessment of technical skill in hip fracture surgery using the postoperative radiograph

et al. 2020

Self Cite

View full text Add to dashboard Cite

Aims To develop a core outcome set of measurements from postoperative radiographs that can be used to assess technical skill in performing dynamic hip screw (DHS) and hemiarthroplasty, and to validate these against Van der Vleuten’s criteria for effective assessment. Methods A Delphi exercise was undertaken at a regional major trauma centre to identify candidate measurement items. The feasibility of taking these measurements was tested by two of the authors (HKJ, GTRP). Validity and reliability were examined using the radiographs of operations performed by orthopaedic resident participants (n = 28) of a multicentre randomized controlled educational trial (ISRCTN20431944). Trainees were divided into novice and intermediate groups, defined as having performed < ten or ≥ ten cases each for DHS and hemiarthroplasty at baseline. The procedure-based assessment (PBA) global rating score was assumed as the gold standard assessment for the purposes of concurrent validity. Intra- and inter-rater reliability testing were performed on a random subset of 25 cases. Results In total, 327 DHS and 248 hemiarthroplasty procedures were performed by 28 postgraduate year (PGY) 3 to 5 orthopaedic trainees during the 2014 to 2015 surgical training year at nine NHS hospitals in the West Midlands, UK. Overall, 109 PBAs were completed for DHS and 80 for hemiarthroplasty. Expert consensus identified four ‘final product analysis’ (FPA) radiological parameters of technical success for DHS: tip-apex distance (TAD); lag screw position in the femoral head; flushness of the plate against the lateral femoral cortex; and eight-cortex hold of the plate screws. Three parameters were identified for hemiarthroplasty: leg length discrepancy; femoral stem alignment; and femoral offset. Face validity, content validity, and feasibility were excellent. For all measurements, performance was better in the intermediate compared with the novice group, and this was statistically significant for TAD (p < 0.001) and femoral stem alignment (p = 0.023). Concurrent validity was poor when measured against global PBA score. This may be explained by the fact that they are measuring difference facets of competence. Intra-and inter-rater reliability were excellent for TAD, moderate for lag screw position (DHS), and moderate for leg length discrepancy (hemiarthroplasty). Use of a large multicentre dataset suggests good generalizability of the results to other settings. Assessment using FPA was time- and cost-effective compared with PBA. Conclusion Final product analysis using post-implantation radiographs to measure technical skill in hip fracture surgery is feasible, valid, reliable, and cost-effective. It can complement traditional workplace-based assessment for measuring performance in the real-world operating room . It may have particular utility in competency-based training frameworks and for assessing skill transfer from the simulated to live operating theatre. Cite this article: Bone Joint Open 2020;1-9:594–604.

show abstract

Analysis of Tools Used in Assessing Technical Skills and Operative Competence in Trauma and Orthopaedic Surgical Training

Cited by 25 publications

References 116 publications

Contemporary Issues in the Acquisition of Orthopaedic Surgical Skills During Residency

Contemporary Issues in the Acquisition of Orthopaedic Surgical Skills During Residency

Cadaveric simulation versus standard training for postgraduate trauma and orthopaedic surgical trainees: protocol for the CAD:TRAUMA study multicentre randomised controlled educational trial

Assessment of technical skill in hip fracture surgery using the postoperative radiograph

Contact Info

Product

Resources

About