Exploring Segment-level Semantics for Online Phase Recognition from Surgical Videos

Ding, Xinpeng; Li, Yongming

doi:10.48550/arxiv.2111.11044

Cited by 1 publication

(2 citation statements)

References 31 publications

(58 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use the ResNet50 [11] as the backbone to extract features for each frame. After that, following [24,28,6], a multi-stage temporal convolution (MS-TCN) [8] is used to extracted temporal relations for frame features and predicts phase results. Following the same evaluation protocols [24,13,14,15], we employ four commonly-used metrics, i.e., accuracy (AC), precision (PR), recall (RE), and Jaccard (JA) to evaluate the phase prediction accuracy.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…on the target datasets, has achieved great success for image recognition and video understanding [10,4,2,19]. This paper focuses on designing self-supervised learning methods for surgical video understanding with a downstream tasksurgical phase recognition, which aims to predict what phase is occurring for each frame in a video [1,29,13,14,15,6,9,25,24,28]. Self-supervised learning has been widely applied into various medical images, such as X-ray [30], fundus images [16,17], CT [34] and MRI [32,30].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Free Lunch for Surgical Video Understanding by Distilling Self-Supervisions

Ding¹,

Liu²,

Li³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Self-supervised learning has witnessed great progress in vision and NLP; recently, it also attracted much attention to various medical imaging modalities such as X-ray, CT, and MRI. Existing methods mostly focus on building new pretext self-supervision tasks such as reconstruction, orientation, and masking identification according to the properties of medical images. However, the publicly available self-supervision models are not fully exploited. In this paper, we present a powerful yet efficient self-supervision framework for surgical video understanding. Our key insight is to distill knowledge from publicly available models trained on large generic datasets 4 to facilitate the self-supervised learning of surgical videos. To this end, we first introduce a semantic-preserving training scheme to obtain our teacher model, which not only contains semantics from the publicly available models, but also can produce accurate knowledge for surgical data. Besides training with only contrastive learning, we also introduce a distillation objective to transfer the rich learned information from the teacher model to self-supervised learning on surgical data. Extensive experiments on two surgical phase recognition benchmarks show that our framework can significantly improve the performance of existing self-supervised learning methods. Notably, our framework demonstrates a compelling advantage under a low-data regime. Our code is available at https://github.com/xmed-lab/DistillingSelf.

show abstract