Relationship between objective assessment of technical skills and subjective in-training evaluations in surgical residents

Feldman, Liane S.; Hagarty, Sarah E; Ghitulescu, Gabriela; Stanbridge, D.; Fried, Gerald M.

doi:10.1016/j.jamcollsurg.2003.08.020

Cited by 136 publications

(92 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…16 In contrast, our results suggest that ITER scores can discriminate resident technical performance. We found that a failing score on the OSATS in PGY-1 was associated with significantly poorer technical skills in PGY-2, based on ITER data.…”

Section: Bottom Linecontrasting

confidence: 74%

“…One is the use of ITER data, which have been criticized for being poor at identifying below-average residents. 16 However, while the reliability of the ITER can be low with a single rater and a single evaluation, aggregated ITER data (as used in our study) with multiple evaluators and across multiple rotations have been shown to have good reliability and predictive validity. [31][32][33] The second limitation is its retrospective nature.…”

Section: Bottom Linementioning

confidence: 71%

“…Progression within a surgery program often relies on ITER evaluations, which are poor at identifying residents with below-average technical skills. 16 Implementing an objective assessment of technical skills early in surgical training may be instrumental in identifying underperformers and introducing early educational interventions for effective remediation, and could help to address the failure to fail phenomenon. 17,18 The OSATS examination, originally developed as a technical skills assessment, 19 was used in the present study to investigate the predictive ability of an objective, standardized, performance-based assessment.…”

Section: Bottom Linementioning

confidence: 99%

See 2 more Smart Citations

Passing a Technical Skills Examination in the First Year of Surgical Residency Can Predict Future Performance

Montbrun

Louridas

Grantcharov

2017

Journal of Graduate Medical Education

View full text Add to dashboard Cite

show abstract

Section: Bottom Linecontrasting

confidence: 74%

Section: Bottom Linementioning

confidence: 71%

Section: Bottom Linementioning

confidence: 99%

See 1 more Smart Citation

Passing a Technical Skills Examination in the First Year of Surgical Residency Can Predict Future Performance

Montbrun

Louridas

Grantcharov

2017

Journal of Graduate Medical Education

View full text Add to dashboard Cite

show abstract

“…However, for practical and effective use in the surgical curriculum, tasks and metrics incorporated in the simulator need to be tested for objectivity and reliability. Consequently, each new training device must be assessed and validated.In the literature, the validation of training tools and their effectiveness are described using different theoretical models, but exact definitions vary among different authors [2,5,11]. One important step in evaluating new training tools is to assess construct validity.…”

mentioning

confidence: 99%

“…In the literature, the validation of training tools and their effectiveness are described using different theoretical models, but exact definitions vary among different authors [2,5,11]. One important step in evaluating new training tools is to assess construct validity.…”

mentioning

confidence: 99%

Construct validity and assessment of the learning curve for the SIMENDO endoscopic simulator

et al. 2007

View full text Add to dashboard Cite

The SIMENDO is an affordable virtual reality simulator designed to train basic psychomotor skills for endoscopic surgery. This study aimed first to establish construct validity by determining which parameters can discriminate groups with different experience levels, and second to establish the extent to which training is useful by determining when inexperienced groups reach expert level. Methods: The study participants were divided into four groups according to their experience with endoscopic procedures: experienced group (group A, >50 procedures performed, n = 15), intermediate group (group B, 1-50 procedures performed, n = 18), endoscope navigation group (group C, endoscope navigation experience, n = 14), and novice group (group D, no endoscopic experience, n = 14). Each participant performed three repetitions of six consecutive exercises. The parameters studied were task time, path length of the instruments, and number of errors (collisions). Some participants continued training up to 10 repetitions to get insight in the learning curve. Results: Group A (expert) outperformed all the other groups (B, C, and D) in terms of total median task time (p < 0.05), groups C and D in terms of path length, and group D in terms of collision frequency in the first two repetitions. Group B (intermediate) outperformed group D (novice) in total time and endoscope path length for all repetitions, and group C (camera navigation) outperformed group D (novice) in the first repetition. Less experienced groups D and C did not reach expert level for the task time within 10 repetitions, and group B reached it after the eighth repetition (p < 0.05). Conclusion: The study was able to establish construct validity for the training program with the simulator under study. The learning curve showed that training with this simulator is useful for subjects with or without limited endoscopic experience. Furthermore, previous endoscopic camera navigation already improves motor skills to more than the basic level.Key words: Construct validity -LaparoscopyMotor skills -Simulator -Training -Virtual reality Endoscopic virtual reality (VR) trainers have become an attractive and valuable tool for training surgeons in a nonpatient environment. The aim of a simulator is to shorten the learning curve of the trainee for the real job.The SIMENDO (DelltaTech, Delft, The Netherlands) is a recently developed affordable simulator (5-8 thousand euro for software and hardware) aimed at training hand-eye coordination motor skills needed to perform endoscopic surgery. This simulator is designed to provide an easy-to-use plug-and-play system for surgical trainees. It features abstract tasks and simultaneously measures the performance of subjects using various parameters for objective assessment. However, for practical and effective use in the surgical curriculum, tasks and metrics incorporated in the simulator need to be tested for objectivity and reliability. Consequently, each new training device must be assessed and validated.In the literature, the validation ...

show abstract

Correlation of Laparoscopic Experience With Differential Functional Brain Activation

Duty¹,

Andonian²,

Ma³

et al. 2012

Arch Surg

View full text Add to dashboard Cite

We compared novice and expert laparoscopists using positron emission tomography (PET) during the peg transfer task of the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) protocol. The first scan (rest) was performed with the subject's eyes closed. The second scan (video 1) was performed while watching a peg transfer video. The third scan (peg 1) was acquired during the peg transfer task. The forth scan (peg 2) was performed after practicing 15 minutes. The fifth scan (video 2) was performed after peg 2 as the subject watched a video clip of a laparoscopic partial nephrectomy. The sixth scan (peg 3) was conducted during the final peg transfer task after 15 more minutes of practice.

show abstract

Relationship between objective assessment of technical skills and subjective in-training evaluations in surgical residents

Cited by 136 publications

References 10 publications

Passing a Technical Skills Examination in the First Year of Surgical Residency Can Predict Future Performance

Passing a Technical Skills Examination in the First Year of Surgical Residency Can Predict Future Performance

Construct validity and assessment of the learning curve for the SIMENDO endoscopic simulator

Correlation of Laparoscopic Experience With Differential Functional Brain Activation

Contact Info

Product

Resources

About