Deep Attention Based Semi-Supervised 2D-Pose Estimation for Surgical Instruments

Abstract

For many practical problems and applications, it is not feasible to create a vast and accurately labeled dataset, which restricts the application of deep learning in many areas. Semi-supervised learning algorithms intend to improve performance by also leveraging unlabeled data. This is very valuable for 2D-pose estimation task where data labeling requires substantial time and is subject to noise. This work aims to investigate if semi-supervised learning techniques can achieve acceptable performance level that makes using these algorithms during training justifiable. To this end, a lightweight network architecture is introduced and mean teacher, virtual adversarial training and pseudo-labeling algorithms are evaluated on 2D-pose estimation for surgical instruments. For the applicability of pseudo-labelling algorithm, we propose a novel confidence measure, total variation. Experimental results show that utilization of semi-supervised learning improves the performance on unseen geometries drastically while maintaining high accuracy for seen geometries. For RMIT benchmark, our lightweight architecture outperforms state-of-the-art with supervised learning. For Endovis benchmark, pseudolabelling algorithm improves the supervised baseline achieving the new state-of-the-art performance.

Surgical Tool Datasets for Machine Learning Research: A Survey

“…Du et al (2018) added detailed annotations to existing labels for the RMIT and EndoVis 2015 datasets, and tested a framework with a fully convolutional detection-regression network for articulated multi-instrument 2-D pose estimation. Kayhan et al (2019) proposed a lightweight deep attention based network architecture and evaluated three SSL algorithms for a deep attention based semi-supervised 2D-pose estimation method for urgical instruments: mean teacher, virtual adversarial training and pseudo-labelling. Analysis was conducted n the RMIT and EndoVis 2015 datasets.…”

“…For example, Sahu et al (2020) tested the Endo-Sim2Real model for instrument segmentation across two datasets-Cholec80 and EndoVis 2015, Zhao et al (2019a) tested their method on the EndoVis Challenge dataset and the ATLAS Dione dataset, and Kalavakonda et al (2019) evaluated three different deep architectures-U-Net, VGG16 and MobileNetV2-on their NeuroID dataset and on the EndoVis 2017 dataset. Du et al (2018) and Kayhan et al (2019) developed machine learning solutions and tested them on the RMIT and EndoVis 2015 datasets. More research initiatives across datasets to evaluate issues such as how accuracy or performance changes from one dataset to another, or the dependence of performance on camera or image quality, is essential.…”