Developing a robust suturing assessment: validity evidence for the intracorporeal suturing assessment tool

Anton, Nicholas E.; Sawyer, John Mark; Korndorffer, James R.; DuCoin, Christopher; McRary, Graham; Timsina, Lava; Bacon, Simon L.

doi:10.1016/j.surg.2017.10.029

Cited by 9 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, the ESs only had the first 5 min of the videos to rate, and this could have given them a harder time giving an accurate rating even though they had received rater training before. This is not in accordance with a previous study, where ESs showed agreement for videos edited to the first part of the procedure [ 14 ]. The use of videos from a simulated environment instead of real-life surgeries made it possible to completely standardize the procedures and allowed all participants to perform in an independent and unsupervised (“real”) fashion.…”

Section: Discussioncontrasting

confidence: 96%

“…We used video recordings on three different modules: bladder neck dissection (BND), neurovascular bundle dissection (NVBD), and urethrovesical anastomosis (UVA), we edited videos to include the first 5 min to standardize them. We anticipated that the total time for video assessment for the CWs would be too long [ 14 ] as full-length videos were up to 43 min long. All CWs and ESs were blinded to the identity and skill level of the surgeon on the recorded video.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Laypersons versus experienced surgeons in assessing simulated robot-assisted radical prostatectomy

Olsen,

Konge,

Hayatzaki

et al. 2023

World J Urol

View full text Add to dashboard Cite

Background Feedback is important for surgical trainees but it can be biased and time-consuming. We examined crowd-sourced assessment as an alternative to experienced surgeons’ assessment of robot-assisted radical prostatectomy (RARP). Methods We used video recordings (n = 45) of three RARP modules on the RobotiX, Simbionix simulator from a previous study in a blinded comparative assessment study. A group of crowd workers (CWs) and two experienced RARP surgeons (ESs) evaluated all videos with the modified Global Evaluative Assessment of Robotic Surgery (mGEARS). Results One hundred forty-nine CWs performed 1490 video ratings. Internal consistency reliability was high (0.94). Inter-rater reliability and test–retest reliability were low for CWs (0.29 and 0.39) and moderate for ESs (0.61 and 0.68). In an Analysis of Variance (ANOVA) test, CWs could not discriminate between the skill level of the surgeons (p = 0.03–0.89), whereas ES could (p = 0.034). Conclusion We found very low agreement between the assessments of CWs and ESs when they assessed robot-assisted radical prostatectomies. As opposed to ESs, CWs could not discriminate between surgical experience using the mGEARS ratings or when asked if they wanted the surgeons to perform their robotic surgery.

show abstract

Section: Discussioncontrasting

confidence: 96%

Section: Methodsmentioning

confidence: 99%

Laypersons versus experienced surgeons in assessing simulated robot-assisted radical prostatectomy

Olsen,

Konge,

Hayatzaki

et al. 2023

World J Urol

View full text Add to dashboard Cite

show abstract

“…We showed identical inter-rater reliability for ES when assessing full-length videos and 5-minute videos. This is in line with Anton et al 7 who showed ES could assess videos of laparoscopic suturing of 30 seconds just as good as full-length videos. When we look at the ability of the ES to discriminate between the surgical skill levels of the surgeons, we find that they can distinguish well between novice surgeons and experienced surgeons and intermediate surgeons and experienced surgeons in both full-length videos and 5-minute videos (Fig.…”

Section: Discussionsupporting

confidence: 89%

“…[2][3][4][5][6] In order to reduce the time and effort needed for video assessments, shortened or segmented video clips could potentially be used instead of full-length videos. 7 Crowdsourced assessment is a relatively new trend that uses anonymous individuals to complete small, precise tasks as an alternative method for evaluation. 8,9 It has previously been used to solve a range of different problems, eg, decipher complex protein structure folding or help blind users find their mobile phones.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Medical Students Cannot Assess Robotic Surgeons Performing Radical Prostatectomy

Olsen

Konge²,

Hayatzaki³

et al. 2024

Sim Healthcare

View full text Add to dashboard Cite

Introduction Medical students have previously been shown to be just as effective for video rating as experts. We want to compare medical students to experienced surgeons as video assessors of simulated robot-assisted radical prostatectomy (RARP) performance. Materials and Methods Video recordings of three RARP modules on the RobotiX (formerly Simbionix) simulator from a previous study were used. Five novice surgeons, five experienced robotic surgeons, and five experienced robotic surgeons in RARP performed a total of 45 video-recorded procedures. The videos were assessed with the modified Global Evaluative Assessment of Robotic Skills tool as both full-length and an edited edition that only included the first 5 minutes of the procedure. Results Fifty medical students and two experienced RARP surgeons (ES) performed a total of 680 video ratings of full-length videos and 5-minute videos (2–9 ratings per video). Medical students and ES showed poor agreement for both full-length videos and 5-minute videos (0.29 and −0.13, respectively). Medical students could not discriminate between the skill level of the surgeons in either full-length videos or 5-minute videos (P = 0.053–0.36 and P = 0.21–0.82), whereas ES could discriminate between novice surgeons and experienced surgeons (full-length, P < 0.001, and 5 minutes, P = 0.007) and intermediate and experienced surgeons (full-length, P = 0.001, and 5 minutes, P = 0.01) in both full-length videos and 5-minute videos. Conclusion We found that medical students cannot be used to assess RARP because they showed poor agreement with the ES rating for both full-length videos and 5-minute videos. Medical students could not discriminate between surgical skill levels.

show abstract

Simulation platforms to assess laparoscopic suturing skills: a scoping review

et al. 2019

View full text Add to dashboard Cite

Developing a robust suturing assessment: validity evidence for the intracorporeal suturing assessment tool

Cited by 9 publications

References 16 publications

Laypersons versus experienced surgeons in assessing simulated robot-assisted radical prostatectomy

Laypersons versus experienced surgeons in assessing simulated robot-assisted radical prostatectomy

Medical Students Cannot Assess Robotic Surgeons Performing Radical Prostatectomy

Simulation platforms to assess laparoscopic suturing skills: a scoping review

Contact Info

Product

Resources

About