Active learning using deep Bayesian networks for surgical workflow analysis

Abstract

Purpose For many applications in the field of computer-assisted surgery, such as providing the position of a tumor, specifying the most probable tool required next by the surgeon or determining the remaining duration of surgery, methods for surgical workflow analysis are a prerequisite. Often machine learning based approaches serve as basis for analyzing the surgical workflow. In general, machine learning algorithms, such as convolutional neural networks (CNN), require large amounts of labeled data. While data is often available in abundance, many tasks in surgical workflow analysis need annotations by domain experts, making it difficult to obtain a sufficient amount of annotations.Methods The aim of using active learning to train a machine learning model is to reduce the annotation effort. Active learning methods determine which unlabeled data points would provide the most information according to some metric, such as prediction uncertainty. Experts will then be asked to only annotate these data points. The model is then retrained with the new data and used to select further data for annotation. Recently, active learning has been applied to CNN by means of Deep Bayesian Networks (DBN). These networks make it possible to assign uncertainties to predictions. In this paper, we present a DBN-based active learning approach adapted for image-based surgical workflow analysis task. Furthermore, by using a recurrent architecture,

Computer Vision in the Surgical Operating Room

“…AI techniques learn patterns and temporal interconnections of the sub-task sequences from combinations of robot kinematics and surgical video and detect and temporally localize each sub-task [19][20][21][22][23][24]. Recently, AI models for activity recognition have been developed and tested on annotated datasets from real cases of robotic-assisted radical prostatectomy and ocular microsurgery [18][19][20]. Future work in this area should focus on investigating the ability of AI methods for surgical workflow analysis to generalize with rigorous validation on multicenter annotated datasets of real procedures [20].…”

“…Recently, AI models for activity recognition have been developed and tested on annotated datasets from real cases of robotic-assisted radical prostatectomy and ocular microsurgery [18][19][20]. Future work in this area should focus on investigating the ability of AI methods for surgical workflow analysis to generalize with rigorous validation on multicenter annotated datasets of real procedures [20].…”

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions

“…In the recent years, a large body of work has focused on recognizing the surgical steps of a procedure directly from the videos [94]- [99]. This has notably been the case in cholecystectomy, a common procedure consisting in removing the gallbladder, which is frequently used in research due to its high frequency of occurrence and well-standardized protocol [100].…”

Artificial Intelligence-Assisted Surgery: Potential and Challenges

“…Several approaches to reducing the annotation effort have been proposed, such as active learning [8][9][10], where only the most informative data points are selected and then are annotated, as well as crowdsourcing, where the "wisdom of the crowd" can be utilized for certain clinical tasks [11,12]. A promising pathway for overcoming the lack of annotated data is to generate realistic synthetic images based on a simple simulation by using generative adversarial networks [13] (Fig.…”