Characterizing Limits of Vision-Based Force Feedback in Simulated Surgical Tool-Tissue Interaction

Huang, Kevin; Chitrakar, Digesh; Mitra, Rahul; Subedi, Divas; Su, Yun-Hsuan

doi:10.1109/embc44109.2020.9176658

Cited by 7 publications

(1 citation statement)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The designed experiments do provide a guideline for systematically quantifying the usability and requirements of synthetic training data for other applications. Enhancing surgical tool segmentation can enable broader research efforts in multicamera surgical reconstruction [ 102 , 103 ] within the context of vision-based force estimation [ 104 , 105 , 106 ] and other robot-assisted medical procedures.…”

Section: Discussionmentioning

confidence: 99%

Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation

Jiang

Chitrakar

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

Accurate semantic image segmentation from medical imaging can enable intelligent vision-based assistance in robot-assisted minimally invasive surgery. The human body and surgical procedures are highly dynamic. While machine-vision presents a promising approach, sufficiently large training image sets for robust performance are either costly or unavailable. This work examines three novel generative adversarial network (GAN) methods of providing usable synthetic tool images using only surgical background images and a few real tool images. The best of these three novel approaches generates realistic tool textures while preserving local background content by incorporating both a style preservation and a content loss component into the proposed multi-level loss function. The approach is quantitatively evaluated, and results suggest that the synthetically generated training tool images enhance UNet tool segmentation performance. More specifically, with a random set of 100 cadaver and live endoscopic images from the University of Washington Sinus Dataset, the UNet trained with synthetically generated images using the presented method resulted in 35.7% and 30.6% improvement over using purely real images in mean Dice coefficient and Intersection over Union scores, respectively. This study is promising towards the use of more widely available and routine screening endoscopy to preoperatively generate synthetic training tool images for intraoperative UNet tool segmentation.

show abstract