Deep Learning in Surgical Workflow Analysis: A review

Abstract

<p><em>Objective</em>: In the last two decades, there has been a growing interest in exploring surgical procedures with statistical models to analyze operations at different semantic levels. This information is necessary for developing context-aware intelligent systems, which can assist the physicians during operations, eval- uate procedures afterward or help the management team to effectively utilize the operating room. The objective is to extract reliable patterns from surgical data for the robust estimation of surgical ac- tivities performed during operations. This paper reviews the state- of-the-art methods published after 2018 for the surgical workflow analysis using deep learning methods. <em>Methods</em>: Three databases, IEEE Xplore, Scopus, and PubMed are searched, and additional studies are added through a manual search. After the database search, 343 studies are screened and a total of 41 studies are selected for this review. <em>Conclusion</em>: Utilizing temporal information is essential for recognizing current surgical actions. Contemporary methods used mainly RNNs, hierarchical CNNs models, and Transformers to preserve long-distance temporal relations. The lack of large publicly available datasets for various procedures is a great challenge for the development of new and robust models. As supervised learning strategies used to show proof-of-concept, self-supervised, semi-supervised, or active learning methods are used to mitigate dependency on annotated data. <em>Significance</em>: The present study provides a comprehensive review of recent methods in surgical workflow analysis, summarizes commonly used architectures, datasets, and discusses challenges.</p>

Multi-level effective surgical workflow recognition in robotic left lateral sectionectomy with deep learning: Experimental research

“…Automatic workflow indexing of raw surgical videos can make it more efficient for surgeons to review the surgical process and share the essence of the surgery for education and communication 9 . Currently, researches on surgical workflow recognition have made significant strides and have achieved relatively high accuracy and precision across a range of surgical procedures 10 . However, much of this research has focused on relatively coarse-grained recognition, such as the surgical phase.…”