More Unlabelled Data or Label More Data? A Study on Semi-supervised Laparoscopic Image Segmentation

Fu, Yunguan; Robu, Maria; Koo, Bong-Jin; Schneider, C.; Laarhoven, Stijn van; Stoyanov, Danail; Davidson, Brian R; Clarkson, Matthew J.; Hu, Yipeng

doi:10.1007/978-3-030-33391-1_20

Cited by 17 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To mitigate data requirements, the recognition problem can be tackled in a semi-supervised or unsupervised manner, where labels are only necessary for model testing. Even if not yet able to outperform supervised methods, these have a lot of potential to capitalise on large amounts of unlabelled data [89].…”

Section: Translational Researchmentioning

confidence: 99%

Gesture Recognition in Robotic Surgery: A Review

Amsterdam

Clarkson²,

Stoyanov³

2021

IEEE Trans. Biomed. Eng.

Self Cite

100

View full text Add to dashboard Cite

Surgical activity recognition is a fundamental step in computer-assisted interventions. This paper reviews the state-of-the-art in methods for automatic recognition of fine-grained gestures in robotic surgery focusing on recent data-driven approaches and outlines the open questions and future research directions. Methods: An article search was performed on 5 bibliographic databases with combinations of the following search terms: robotic, robot-assisted, JIGSAWS, surgery, surgical, gesture, fine-grained, surgeme, action, trajectory, segmentation, recognition, parsing. Selected articles were classified based on the level of supervision required for training and divided into different groups representing major frameworks for time series analysis and data modelling. Results: A total of 52 articles were reviewed. The research field is showing rapid expansion, with the majority of articles published in the last 4 years. Deep-learning-based temporal models with discriminative feature extraction and multi-modal data integration have demonstrated promising results on small surgical datasets. Currently, unsupervised methods perform significantly less well than the supervised approaches. Conclusion: The development of large and diverse open-source datasets of annotated demonstrations is essential for development and validation of robust solutions for surgical gesture recognition. While new strategies for discriminative feature extraction and knowledge transfer, or unsupervised and semi-supervised approaches, can mitigate the need for data and labels, they have not yet been demonstrated to achieve comparable performance. Important future research directions include detection and forecast of gesture-specific errors and anomalies. Significance: This paper is a comprehensive and structured analysis of surgical gesture recognition methods aiming to summarize the status of this rapidly evolving field.

show abstract

Section: Translational Researchmentioning

confidence: 99%

Gesture Recognition in Robotic Surgery: A Review

Amsterdam

Clarkson²,

Stoyanov³

2021

IEEE Trans. Biomed. Eng.

Self Cite

100

View full text Add to dashboard Cite

show abstract

“…Hypothetically the coarser, more lobulated surface of the porcine liver may be more amenable to stereoscopic surface reconstruction because it contains more features to distinguish it from surrounding structures. As demonstrated on this data set, stereoscopic surface reconstruction for the purpose of semi-automatic registration of the human liver is feasible if the liver surface is automatically segmented prior to registration [35,45]. To the best of our knowledge this is the first clinical study to compare accuracy of manual and semi-automatic registration in a group of patients.…”

Section: Discussionmentioning

confidence: 92%

Comparison of manual and semi-automatic registration in augmented reality image-guided liver surgery: a clinical feasibility study

et al. 2020

View full text Add to dashboard Cite

Background The laparoscopic approach to liver resection may reduce morbidity and hospital stay. However, uptake has been slow due to concerns about patient safety and oncological radicality. Image guidance systems may improve patient safety by enabling 3D visualisation of critical intra- and extrahepatic structures. Current systems suffer from non-intuitive visualisation and a complicated setup process. A novel image guidance system (SmartLiver), offering augmented reality visualisation and semi-automatic registration has been developed to address these issues. A clinical feasibility study evaluated the performance and usability of SmartLiver with either manual or semi-automatic registration. Methods Intraoperative image guidance data were recorded and analysed in patients undergoing laparoscopic liver resection or cancer staging. Stereoscopic surface reconstruction and iterative closest point matching facilitated semi-automatic registration. The primary endpoint was defined as successful registration as determined by the operating surgeon. Secondary endpoints were system usability as assessed by a surgeon questionnaire and comparison of manual vs. semi-automatic registration accuracy. Since SmartLiver is still in development no attempt was made to evaluate its impact on perioperative outcomes. Results The primary endpoint was achieved in 16 out of 18 patients. Initially semi-automatic registration failed because the IGS could not distinguish the liver surface from surrounding structures. Implementation of a deep learning algorithm enabled the IGS to overcome this issue and facilitate semi-automatic registration. Mean registration accuracy was 10.9 ± 4.2 mm (manual) vs. 13.9 ± 4.4 mm (semi-automatic) (Mean difference − 3 mm; p = 0.158). Surgeon feedback was positive about IGS handling and improved intraoperative orientation but also highlighted the need for a simpler setup process and better integration with laparoscopic ultrasound. Conclusion The technical feasibility of using SmartLiver intraoperatively has been demonstrated. With further improvements semi-automatic registration may enhance user friendliness and workflow of SmartLiver. Manual and semi-automatic registration accuracy were comparable but evaluation on a larger patient cohort is required to confirm these findings.

show abstract

“…The idea is to construct an image pyramid using different sizes from a full view input image for scaleinvariance. Liver segmentation has also been studied by Fu et al [53]. In this work, they study the effect of adding more labelled or unlabelled data for improving segmentation tasks, particularizing in liver segmentation.…”

Section: ) Segmentation Of Anatomical Structuresmentioning

confidence: 99%

A Review on Deep Learning in Minimally Invasive Surgery

et al. 2021

View full text Add to dashboard Cite

In the last five years, deep learning has attracted great interest in computer-assisted systems for Minimally Invasive Surgery. The straightforward accessibility to images in surgical interventions makes deep neural networks enormously powerful for solving classification problems in complex surgical scenarios. The objective of this work is to provide readers a survey on deep learning models applied to minimally invasive surgery, identifying the different architectures used depending on the application, the results achieved until now, and the publicly available surgical datasets that can be used for validating new studies. A total of 85 publications have been extracted from manual research from four databases (IEEE Xplorer, Springer Link, Science Direct, and ACM Digital Library). After analyzing all these studies, they have been classified into four applications: surgical image analysis, surgical task analysis, surgical skill assessment, and automation of surgical tasks. This work provides a technical description of these works and a comparison among them. Finally, promising research directions to advance in this field are identified.

show abstract

More Unlabelled Data or Label More Data? A Study on Semi-supervised Laparoscopic Image Segmentation

Cited by 17 publications

References 11 publications

Gesture Recognition in Robotic Surgery: A Review

Gesture Recognition in Robotic Surgery: A Review

Comparison of manual and semi-automatic registration in augmented reality image-guided liver surgery: a clinical feasibility study

A Review on Deep Learning in Minimally Invasive Surgery

Contact Info

Product

Resources

About