Automated video-based assessment of surgical skills for training and evaluation in medical schools

Abstract

Our evaluations show that frequency features perform better than motion texture features, which in-turn perform better than symbol-/word-based features. Put succinctly, skill classification accuracy is positively correlated with motion granularity as demonstrated by our results on two challenging video datasets.

What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment

“…Skills assessment: Zia et al [33] extract spatio-temporal interest points (STIP's) in the frequency domain to classify a sample into novice, intermediate or expert skills level. Instead of using handcrafted STIP's Doughty et al [4] learn and use convolutional features with ranking loss as their objective function to evaluate surgical, drawing, chopstick use and dough rolling skills.…”

“…Answering these questions involves the quantification of the quality of the action -determining how well the action was carried out, also known as action quality assessment (AQA). Existing AQA [18,16,26,13,25] and skills assessment [4,10,31,32,33] approaches use a single label, known as a final score or skill-level, to train the system using some kind of regression or ranking loss function. However, the performance of these systems is limited and it seems that a single score is not sufficient to characterize a complicated action.…”

Manipulation-Skill Assessment from Videos with Spatial Attention Network

“…Skill assessment has been extensively researched in the context of computer-assisted surgical training since the current process of surgical skill assessment sorely relies on subjective evaluation by human experts which is highly labor intensive. [13,22,30,40,53,39,54,55,57,56]. Jin et al [22] proposed to automatically assess surgeon performance by tracking and analyzing tool movements in surgical videos, which hints us that the model for skill assessment should pay attention to the task-related regions in video rather than treat visual information in every region equally.…”

Action Quality Assessment Across Multiple Actions

“…For e.g., AQA can be employed in low-cost at-home physiotherapy to manage diseases like cerebral palsy [15,27]. High quality AQA can be incorporated into medical training to assess the surgical skills of a student [28,5]. AQA systems can be used to mimic Olympic judges during sports training [17,16,23,28]; or provide a second opinion in light of recent judging scandals [3,24].…”