State-of-the-art of situation recognition systems for intraoperative procedures

Junger, Denise; Frommer, Sina; Burgert, Oliver

doi:10.1007/s11517-022-02520-4

Cited by 11 publications

(6 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further feasibility testing of the analyzed use cases of prototypical CAS utilizing the specified interface architecture is still needed. It is also important to note that context-aware behavior is dependent on the situation recognition capabilities of the SRS [22]. Despite these limitations, the use of a DICOM network architecture to communicate contextual information has laid the foundation for a higher degree of automation of surgical assistance systems, aligning with the growing trend towards image-guided surgery and the integration of interdependent devices [23].…”

Section: Discussionmentioning

confidence: 99%

A proposal for communication of intraoperative contextual information via DICOM Unified Procedure Steps

Beyersdorffer

Junger

Burgert

2023

Medical Imaging 2023: Imaging Informatics for Healthcare, Research, and Applications

View full text Add to dashboard Cite

[Motivation] In order to enable context-aware behavior of surgical assistance systems, the acquisition of various information about the current intraoperative situation is crucial. To achieve this, the complex task of situation recognition can be delegated to a specialized system. Consequently, a standardized interface is required for the seamless transfer of the recognized contextual information to the assistance systems, enabling them to adapt accordingly. [Methods] Our group analyzed four medical interface standards to determine their suitability for exchanging intraoperative contextual information. The assessment was based on a harmonized data and service model derived from the requirements of expected context-aware use cases. The Digital Imaging and Communications in Medicine (DICOM) and IEEE 11073 for Service-oriented Device Connectivity (SDC) were identified as the most appropriate standards. [Results] We specified how DICOM Unified Procedure Steps (UPS), can be used to effectively communicate contextual information. We proposed the inclusion of attributes to formalize different granularity levels of the surgical workflow. [Conclusions] DICOM UPS SOP classes can be used for the exchange of intraoperative contextual information between a situation recognition system and surgical assistance systems. This can pave the way for vendor-independent context awareness in the OR, leading to targeted assistance of the surgical team and an improvement of the surgical workflow.

show abstract

Section: Discussionmentioning

confidence: 99%

A proposal for communication of intraoperative contextual information via DICOM Unified Procedure Steps

Beyersdorffer

Junger

Burgert

2023

Medical Imaging 2023: Imaging Informatics for Healthcare, Research, and Applications

View full text Add to dashboard Cite

show abstract

“…First steps in this endeavor have been made in the preoperative setting using ML to automatically identify high risk surgical patients [19]. Intraoperatively, ML has been widely studied for the recognition of surgical phases [20,21], for the automatic assessment of the surgeon's skill level [22], or for predicting postoperative adverse events and distractions [23]. With the OR black box, Jung et al presented a first comprehensive approach of quantitatively identifying intraoperative events and distractions by collecting intraoperative data [24].…”

Section: Graphical Abstractmentioning

confidence: 99%

Surgomics: personalized prediction of morbidity, mortality and long-term outcome in surgery using machine learning on multimodal data

et al. 2022

View full text Add to dashboard Cite

Background Personalized medicine requires the integration and analysis of vast amounts of patient data to realize individualized care. With Surgomics, we aim to facilitate personalized therapy recommendations in surgery by integration of intraoperative surgical data and their analysis with machine learning methods to leverage the potential of this data in analogy to Radiomics and Genomics. Methods We defined Surgomics as the entirety of surgomic features that are process characteristics of a surgical procedure automatically derived from multimodal intraoperative data to quantify processes in the operating room. In a multidisciplinary team we discussed potential data sources like endoscopic videos, vital sign monitoring, medical devices and instruments and respective surgomic features. Subsequently, an online questionnaire was sent to experts from surgery and (computer) science at multiple centers for rating the features’ clinical relevance and technical feasibility. Results In total, 52 surgomic features were identified and assigned to eight feature categories. Based on the expert survey (n = 66 participants) the feature category with the highest clinical relevance as rated by surgeons was “surgical skill and quality of performance” for morbidity and mortality (9.0 ± 1.3 on a numerical rating scale from 1 to 10) as well as for long-term (oncological) outcome (8.2 ± 1.8). The feature category with the highest feasibility to be automatically extracted as rated by (computer) scientists was “Instrument” (8.5 ± 1.7). Among the surgomic features ranked as most relevant in their respective category were “intraoperative adverse events”, “action performed with instruments”, “vital sign monitoring”, and “difficulty of surgery”. Conclusion Surgomics is a promising concept for the analysis of intraoperative data. Surgomics may be used together with preoperative features from clinical data and Radiomics to predict postoperative morbidity, mortality and long-term outcome, as well as to provide tailored feedback for surgeons. Graphical abstract

show abstract

“…Garrow et al [20] have provided an overview of the latest algorithms and data sources for surgical phase recognition in 2018. Junger et al [21] have investigated 58 studies published between 2010 and 2019 in different granularity levels with a focus on applicability and transferability. Amsterdam et al [16] have focused on low-level granularity and reviewed gesture recognition techniques in robotic surgery.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning in Surgical Workflow Analysis: A Review of Phase and Step Recognition

Demir¹,

Schieber²,

Roth³

et al. 2023

Preprint

View full text Add to dashboard Cite

Objective: In the last decades, there has been a growing interest in exploring surgical procedures with statistical models to analyze operations in different se?mantic levels. This information is necessary for developing context-aware intelligent systems, which can assist the physicians during operations, evaluate procedures after?ward or help the management team to effectively utilize the operating room. The objective is to extract reliable patterns from surgical data for the robust estimation of surgical activities performed during operations. This paper reviews the state-of-the-art methods for the recognition of highest?level surgical activities, i.e., phases, and focuses on data and learning algorithms. Methods: Three databases, IEEE Xplore, Scopus, and PubMed are searched, and additional studies are added through a manual search. After the database search, 173 studies are screened and a total of 33 studies are selected for the review. Conclusion: The de?velopment of robust surgical phase recognition algorithms requires large and diverse datasets, as the task is highly variable and complex. Much progress has been made, but publicly available data are still limited in volume and biased to endoscopic videos. Unsupervised or semi-supervised learning approaches and active learning methods are pro?posed to lessen the consequences of these challenges. Additionally, other potential data sources in the operating room and using different modalities together could be in?vestigated in the future. Significance: The present study provides a comprehensive review of surgical phase recog?nition algorithms, their generalizability, and point under?investigated areas for possible improvements

show abstract

State-of-the-art of situation recognition systems for intraoperative procedures

Cited by 11 publications

References 59 publications

A proposal for communication of intraoperative contextual information via DICOM Unified Procedure Steps

A proposal for communication of intraoperative contextual information via DICOM Unified Procedure Steps

Surgomics: personalized prediction of morbidity, mortality and long-term outcome in surgery using machine learning on multimodal data

Deep Learning in Surgical Workflow Analysis: A Review of Phase and Step Recognition

Contact Info

Product

Resources

About