An Efficient One-Stage Detector for Real-Time Surgical Tools Detection in Robot-Assisted Surgery

Yang, Yu; Zhao, Zijian; Shi, Pan; Hu, Sanyuan

doi:10.1007/978-3-030-80432-9_2

Cited by 9 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They proposed a two-stage detector, an attention-guided convolutional neural network with coarse and refined modules, to achieve high inference time (55.5 FPS) and mAP (91.65%). Cholec80-location subset was also used on a one-stage detector by Yang et al [30], adding modifications to the backbone and neck of the architecture. In the backbone, they used a GoshtNet architecture and cross-stage partial connections to increase inference time and enhance the learning process.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Real‐time surgical tool detection with multi‐scale positional encoding and contrastive learning

Loza,

Valdastri,

Ali

2023

Healthcare Tech Letters

View full text Add to dashboard Cite

Real‐time detection of surgical tools in laparoscopic data plays a vital role in understanding surgical procedures, evaluating the performance of trainees, facilitating learning, and ultimately supporting the autonomy of robotic systems. Existing detection methods for surgical data need to improve processing speed and high prediction accuracy. Most methods rely on anchors or region proposals, limiting their adaptability to variations in tool appearance and leading to sub‐optimal detection results. Moreover, using non‐anchor‐based detectors to alleviate this problem has been partially explored without remarkable results. An anchor‐free architecture based on a transformer that allows real‐time tool detection is introduced. The proposal is to utilize multi‐scale features within the feature extraction layer and at the transformer‐based detection architecture through positional encoding that can refine and capture context‐aware and structural information of different‐sized tools. Furthermore, a supervised contrastive loss is introduced to optimize representations of object embeddings, resulting in improved feed‐forward network performances for classifying localized bounding boxes. The strategy demonstrates superiority to state‐of‐the‐art (SOTA) methods. Compared to the most accurate existing SOTA (DSSS) method, the approach has an improvement of nearly 4% on mAP50 and a reduction in the inference time by 113%. It also showed a 7% higher mAP50 than the baseline model.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Cholec80‐location subset was also used on a one‐stage detector by Yang et al. [ 30 ], adding modifications to the backbone and neck of the architecture. In the backbone, they used a GoshtNet architecture and cross‐stage partial connections to increase inference time and enhance the learning process.…”

Section: Related Workmentioning

confidence: 99%

Real‐time surgical tool detection with multi‐scale positional encoding and contrastive learning

Loza,

Valdastri,

Ali

2023

Healthcare Tech Letters

View full text Add to dashboard Cite

show abstract

“…In another study (Zhang et al 2020), irrigator can be observed as worst performing instrument with average precision (AP) of 41.6%, followed by grasper with 54.1% in a supervised setting at intersection-over-union (IoU) threshold of 50%. A ghost feature map-based pipeline was used to reduce the computational burden for tool detection in Yang et al (2021). A CNN-based hidden Markov model was proposed by Twinanda et al (2016a) for surgical tool detection from laparoscopic videos.…”

Section: Related Workmentioning

confidence: 99%

A semi-supervised Teacher-Student framework for surgical tool detection and localization

Teevno

Ochoa-Ruiz

Ali

2022

Computer Methods in Biomechanics and Biomedical Engineering: Im

View full text Add to dashboard Cite

Surgical tool detection in minimally invasive surgery is an essential part of computer-assisted interventions. Current approaches are mostly based on supervised methods requiring large annotated datasets. However, labelled datasets are often scarce. Semi-supervised learning (SSL) has recently emerged as a viable alternative showing promise in producing models retaining competitive performance to supervised methods. Therefore, this paper introduces an SSL framework in the surgical tool detection paradigm, which aims to mitigate training data scarcity and data imbalance problems through a knowledge distillation approach. In the proposed work, we train a model with labelled data which initialises the Teacher-Student joint learning, where the Student is trained on Teacher-generated pseudo-labels from unlabelled data. We also propose a multi-class distance with a margin-based classification loss function in the region-of-interest head of the detector to segregate the foreground-background region effectively. Our results on m2cai16-tool-locations dataset indicates the superiority of our approach on different supervised data settings (1%, 2%, 5% and 10% of annotated data) where our model achieves overall improvements of 8%, 12%, and 27% in mean average precision on 1% labelled data over the state-of-theart SSL methods and the supervised baseline, respectively. The code is available at https://github.com/ Mansoor-at/Semi-supervised-surgical-tool-detection.

show abstract

“…For example RCNN‐based models [ 24 , 25 , 26 ] struggle to meet the frame rate of laparoscopic videos even on powerful servers, due to their two‐stage process of object localization and classification. In contrast, YOLO‐based models [ 27 , 28 , 29 ], which are one‐stage, are faster and show similar detection accuracy compared with RCNN‐based models, making them more suitable for laparoscopic tool detection on embedded devices.…”

Section: Introductionmentioning

confidence: 99%

YOLOv7‐RepFPN: Improving real‐time performance of laparoscopic tool detection on embedded systems

Liu,

Hayashi,

Oda

et al. 2024

Healthcare Tech Letters

View full text Add to dashboard Cite

This study focuses on enhancing the inference speed of laparoscopic tool detection on embedded devices. Laparoscopy, a minimally invasive surgery technique, markedly reduces patient recovery times and postoperative complications. Real‐time laparoscopic tool detection helps assisting laparoscopy by providing information for surgical navigation, and its implementation on embedded devices is gaining interest due to the portability, network independence and scalability of the devices. However, embedded devices often face computation resource limitations, potentially hindering inference speed. To mitigate this concern, the work introduces a two‐fold modification to the YOLOv7 model: the feature channels and integrate RepBlock is halved, yielding the YOLOv7‐RepFPN model. This configuration leads to a significant reduction in computational complexity. Additionally, the focal EIoU (efficient intersection of union) loss function is employed for bounding box regression. Experimental results on an embedded device demonstrate that for frame‐by‐frame laparoscopic tool detection, the proposed YOLOv7‐RepFPN achieved an mAP of 88.2% (with IoU set to 0.5) on a custom dataset based on EndoVis17, and an inference speed of 62.9 FPS. Contrasting with the original YOLOv7, which garnered an 89.3% mAP and 41.8 FPS under identical conditions, the methodology enhances the speed by 21.1 FPS while maintaining detection accuracy. This emphasizes the effectiveness of the work.

show abstract

An Efficient One-Stage Detector for Real-Time Surgical Tools Detection in Robot-Assisted Surgery

Cited by 9 publications

References 21 publications

Real‐time surgical tool detection with multi‐scale positional encoding and contrastive learning

Real‐time surgical tool detection with multi‐scale positional encoding and contrastive learning

A semi-supervised Teacher-Student framework for surgical tool detection and localization

YOLOv7‐RepFPN: Improving real‐time performance of laparoscopic tool detection on embedded systems

Contact Info

Product

Resources

About