Machine learning analyses of automated performance metrics during granular sub-stitch phases predict surgeon experience

Chen, Andrew B.; Liang, Siqi; Nguyen, Jessica; Liu, Yan; Hung, Andrew J.

doi:10.1016/j.surg.2020.09.020

Cited by 26 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Objective performance indicator data provides new information to understand and classify surgeon and trainee techniques and skill level during both individual surgical tasks and the entire procedure. 3,9 This contrasts with today's gold standard: live observation or video review in combination with competency assessment tools (i.e.,, GEARS and OSATS). Assessment tools are inherently subjective and limited by observer bias, time and energy constraints, poor scalability, and limited actionable feedback.…”

Section: Discussionmentioning

confidence: 99%

Ratio of Economy of Motion: A New Objective Performance Indicator to Assign Consoles During Dual-Console Robotic Proctectomy

Devin

Gillani

Shields

et al. 2023

The American Surgeon™

View full text Add to dashboard Cite

Background Our group investigates objective performance indicators (OPIs) to analyze robotic colorectal surgery. Analyses of OPI data are difficult in dual-console procedures (DCPs) as there is currently no reliable, efficient, or scalable technique to assign console-specific OPIs during a DCP. We developed and validated a novel metric to assign tasks to appropriate surgeons during DCPs. Methods A colorectal surgeon and fellow reviewed 21 unedited, dual-console proctectomy videos with no information to identify the operating surgeons. The reviewers watched a small number of random tasks and assigned “attending” or “trainee” to each task. Based on this sampling, the remainder of task assignments for each procedure was extrapolated. In parallel, we applied our newly developed OPI, ratio of economy of motion (rEOM), to assign consoles. Results from the 2 methods were compared. Results A total of 1811 individual surgical tasks were recorded during 21 proctectomy videos. A median of 6.5 random tasks (137 total) were reviewed during each video, and the remainder of task assignments were extrapolated based on the 7.6% of tasks audited. The task assignment agreement was 91.2% for video review vs rEOM, with rEOM providing ground truth. It took 2.5 hours to manually review video and assign tasks. Ratio of economy of motion task assignment was immediately available based on OPI recordings and automated calculation. Discussion We developed and validated rEOM as an accurate, efficient, and scalable OPI to assign individual surgical tasks to appropriate surgeons during DCPs. This new resource will be useful to everyone involved in OPI research across all surgical specialties.

show abstract

Section: Discussionmentioning

confidence: 99%

Ratio of Economy of Motion: A New Objective Performance Indicator to Assign Consoles During Dual-Console Robotic Proctectomy

Devin

Gillani

Shields

et al. 2023

The American Surgeon™

View full text Add to dashboard Cite

show abstract

“…In addition, AI has the potential to distinguish surgeon experience and to provide access to standard surgical solutions that are independent of individuals’ experience and day-to-day performance changes. Chen et al’s study demonstrated that machine learning can accurately classify surgeon experience based on individual stitches and sub-stitches in the vesico-urethral anastomosis of a robot-assisted radical prostatectomy [ 46 ]. Saeidi et al achieved the enhanced autonomy necessary to perform robotic laparoscopic anastomosis of the small bowel using the Smart Tissue Autonomous Robot (STAR) and they found that autonomous robotic laparoscopic surgery outperforms expert surgeons’ manual technique and robot-assisted surgery technique in terms of consistency and accuracy during laparoscopic small bowel anastomosis experiments [ 47 ].…”

Section: Discussionmentioning

confidence: 99%

Effect of annualized surgeon volume on major surgical complications for abdominal and laparoscopic radical hysterectomy for cervical cancer in China, 2004–2016: a retrospective cohort study

Liang

Liu

et al. 2023

BMC Women's Health

View full text Add to dashboard Cite

Background Previous studies have suggested that higher surgeon volume leads to improved perioperative outcomes for oncologic surgery; however, the effect of surgeon volumes on surgical outcomes might differ according to the surgical approach used. This paper attempts to evaluate the effect of surgeon volume on complications or cervical cancer in an abdominal radical hysterectomy (ARH) cohort and laparoscopic radical hysterectomy (LRH) cohort. Methods We conducted a population-based retrospective study using the Major Surgical Complications of Cervical Cancer in China (MSCCCC) database to analyse patients who underwent radical hysterectomy (RH) from 2004 to 2016 at 42 hospitals. We estimated the annualized surgeon volumes in the ARH cohort and in the LRH cohort separately. The effect of the surgeon volume of ARH or LRH on surgical complications was examined using multivariable logistic regression models. Results In total, 22,684 patients who underwent RH for cervical cancer were identified. In the abdominal surgery cohort, the mean surgeon case volume increased from 2004 to 2013 (3.5 to 8.7 cases) and then decreased from 2013 to 2016 (8.7 to 4.9 cases). The mean surgeon case volume number of surgeons performing LRH increased from 1 to 12.1 cases between 2004 and 2016 (P < 0.01). In the abdominal surgery cohort, patients treated by intermediate-volume surgeons were more likely to experience postoperative complications (OR = 1.55, 95% CI = 1.11–2.15) than those treated by high-volume surgeons. In the laparoscopic surgery cohort, surgeon volume did not appear to influence the incidence of intraoperative or postoperative complications (P = 0.46; P = 0.13). Conclusions The performance of ARH by intermediate-volume surgeons is associated with an increased risk of postoperative complications. However, surgeon volume may have no effect on intraoperative or postoperative complications after LRH.

show abstract

“…These metrics enable thorough quantification of key aspects like precision, recall, and specificity for the sentiment models. The metrics assess the model's exactness, completeness, and effectiveness in identifying the sentiments correctly (Chen et al, 2021 ). Assessing these characteristics provides comprehensive insights into each model's capabilities and limitations in sentiment analysis, guiding the selection of most suitable models for analyzing tweets.…”

Section: Methodsmentioning

confidence: 99%

Sentiment analysis of COP9-related tweets: a comparative study of pre-trained models and traditional techniques

Elmitwalli,

Mehegan

2024

Front. Big Data

View full text Add to dashboard Cite

IntroductionSentiment analysis has become a crucial area of research in natural language processing in recent years. The study aims to compare the performance of various sentiment analysis techniques, including lexicon-based, machine learning, Bi-LSTM, BERT, and GPT-3 approaches, using two commonly used datasets, IMDB reviews and Sentiment140. The objective is to identify the best-performing technique for an exemplar dataset, tweets associated with the WHO Framework Convention on Tobacco Control Ninth Conference of the Parties in 2021 (COP9).MethodsA two-stage evaluation was conducted. In the first stage, various techniques were compared on standard sentiment analysis datasets using standard evaluation metrics such as accuracy, F1-score, and precision. In the second stage, the best-performing techniques from the first stage were applied to partially annotated COP9 conference-related tweets.ResultsIn the first stage, BERT achieved the highest F1-scores (0.9380 for IMDB and 0.8114 for Sentiment 140), followed by GPT-3 (0.9119 and 0.7913) and Bi-LSTM (0.8971 and 0.7778). In the second stage, GPT-3 performed the best for sentiment analysis on partially annotated COP9 conference-related tweets, with an F1-score of 0.8812.DiscussionThe study demonstrates the effectiveness of pre-trained models like BERT and GPT-3 for sentiment analysis tasks, outperforming traditional techniques on standard datasets. Moreover, the better performance of GPT-3 on the partially annotated COP9 tweets highlights its ability to generalize well to domain-specific data with limited annotations. This provides researchers and practitioners with a viable option of using pre-trained models for sentiment analysis in scenarios with limited or no annotated data across different domains.

show abstract

Machine learning analyses of automated performance metrics during granular sub-stitch phases predict surgeon experience

Cited by 26 publications

References 11 publications

Ratio of Economy of Motion: A New Objective Performance Indicator to Assign Consoles During Dual-Console Robotic Proctectomy

Ratio of Economy of Motion: A New Objective Performance Indicator to Assign Consoles During Dual-Console Robotic Proctectomy

Effect of annualized surgeon volume on major surgical complications for abdominal and laparoscopic radical hysterectomy for cervical cancer in China, 2004–2016: a retrospective cohort study

Sentiment analysis of COP9-related tweets: a comparative study of pre-trained models and traditional techniques

Contact Info

Product

Resources

About