Stable Image Registration for In-Vivo Fetoscopic Panorama Reconstruction

Gaisser, Floris; Peeters, Suzanne; Lenseigne, Boris; Jonker, Pieter; Oepkes, Dick

doi:10.3390/jimaging4010024

Cited by 22 publications

(23 citation statements)

References 18 publications

(21 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed approach may also be integrated with recent work, which deals with vessel segmentation from placenta images 1,34 , stitching of fetoscopy images to build placental panoramic image 12,44 and classification of TTTS surgical phases 38 .…”

Section: Discussionmentioning

confidence: 99%

Inter-foetus Membrane Segmentation for TTTS Using Adversarial Networks

et al. 2019

View full text Add to dashboard Cite

Twin-to-Twin Transfusion Syndrome (TTTS) is commonly treated with minimally invasive laser surgery in fetoscopy. The inter-foetal membrane is used as a reference to find abnormal anastomoses. Membrane identification is a challenging task due to small field of view of the camera, presence of amniotic liquid, foetus movement, illumination changes and noise. This paper aims at providing automatic and fast membrane segmentation in fetoscopic images. We implemented an adversarial network consisting of two Fully-Convolutional Neural Networks (FCNNs). The former (the segmentor) is a segmentation network inspired by U-Net and integrated with residual blocks, whereas the latter acts as critic and is made only of the encoding path of the segmentor. A dataset of 900 images acquired in 6 surgical cases was collected and labelled to validate the proposed approach. The adversarial networks achieved a median Dice similarity coefficient of 91.91% with Inter-Quartile Range (IQR) of 4.63%, overcoming approaches based on U-Net (82.98%-IQR : 14.41%) and U-Net with residual blocks (86.13%-IQR : 13.63%). Results proved that the proposed architecture could be a valuable and robust solution to assist surgeons in providing membrane identification while performing fetoscopic surgery.

show abstract

Section: Discussionmentioning

confidence: 99%

Inter-foetus Membrane Segmentation for TTTS Using Adversarial Networks

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Such feature descriptors are attractive initial options for mosaicking algorithms, as they are easy to describe mathematically and do not require application-specific training datasets. They have poor repeatability when applied to fetoscopic video frames, however, which makes it difficult to consistently extract enough frame-to-frame correspondences to estimate the motion of the fetoscope [13]. Deep-learned vessel segmentations have the potential to be invariant to many of the visual distractors present in fetoscopic video frames and therefore have the potential to serve as more robust and more repeatable features for panorama construction.…”

Section: Discussionmentioning

confidence: 99%

“…Gaisser et al [13], for example, simulated ex vivo and in vivo settings using a placental phantom and found that the performance of various feature detection algorithms could fall dramatically in the translation to in vivo. A feature detector could detect as many as 73% fewer features in images acquired in vivo settings as opposed to images acquired in ex vivo settings.…”

Section: Introductionmentioning

confidence: 99%

“…These blood vessels are too small to be reliably resolved on ultrasound. Gaisser et al [13] made an important step by demonstrating that it is possible to use a region-based convolutional neural network (R-CNN) to detect stable center points within placental blood vessels. This deep-learned approach far outperformed standard feature detection algorithms when applied to in vivo images in which the amniotic fluid had a yellow coloration, but the performance significantly degraded when the amniotic fluid had a green coloration, suggesting that the algorithm was not robust to variations that might be seen in the intrauterine environment.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep-learned placental vessel segmentation for intraoperative video enhancement in fetoscopic surgery

et al. 2018

View full text Add to dashboard Cite

Introduction Twin-to-twin transfusion syndrome (TTTS) is a potentially lethal condition that affects pregnancies in which twins share a single placenta. The definitive treatment for TTTS is fetoscopic laser photocoagulation, a procedure in which placental blood vessels are selectively cauterized. Challenges in this procedure include difficulty in quickly identifying placental blood vessels due to the many artifacts in the endoscopic video that the surgeon uses for navigation. We propose using deep-learned segmentations of blood vessels to create masks that can be recombined with the original fetoscopic video frame in such a way that the location of placental blood vessels is discernable at a glance. Methods In a process approved by an institutional review board, intraoperative videos were acquired from ten fetoscopic laser photocoagulation surgeries performed at Yale New Haven Hospital. A total of 345 video frames were selected from these videos at regularly spaced time intervals. The video frames were segmented once by an expert human rater (a clinician) and once by a novice, but trained human rater (an undergraduate student). The segmentations were used to train a fully convolutional neural network of 25 layers. Results The neural network was able to produce segmentations with a high similarity to ground truth segmentations produced by an expert human rater (sensitivity=92.15%±10.69%) and produced segmentations that were significantly more accurate than those produced by a novice human rater (sensitivity=56.87%±21.64%; p < 0.01). Conclusion A convolutional neural network can be trained to segment placental blood vessels with near-human accuracy and can exceed the accuracy of novice human raters. Recombining these segmentations with the original fetoscopic video frames can produced enhanced frames in which blood vessels are easily detectable. This has significant implications for aiding fetoscopic surgeons—especially trainees who are not yet at an expert level.

show abstract

“…This method generates noticeable results (back projection error of 0.6 pixel), but cannot be applied to real time tracking: it needs 3.4sec for every pair of images. In [12], the authors apply a novel approach based on the use of convolutional neural networks, with the need of a first learning phase. The used images come from a simulated placenta, with better visual conditions than in real scenarios.…”

Section: Related Workmentioning

confidence: 99%

Vision Based Robot Assistance in TTTS Fetal Surgery

Sayols

Hernansanz

Parra

et al. 2019

2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

View full text Add to dashboard Cite

This paper presents an accurate and robust tracking vision algorithm for Fetoscopic Laser Photo-coagulation (FLP) surgery for Twin-Twin Transfusion Syndrome (TTTS). The aim of the proposed method is to assist surgeons during anastomosis localization, coagulation and review using a teleoperated robotic system. The algorithm computes the relative position of the fetoscope tool tip with respect to the placenta, via local vascular structure registration. The algorithm uses image features (local superficial vascular structures of the placenta's surface) to automatically match consecutive fetoscopic images. It is composed of three sequential steps: image processing (filtering, binarization and vascular structures segmentation); relevant Points Of Interest (POIs) seletion; and image registration between consecutive images. The algorithm has to deal with the low quality of fetoscopic images, the liquid and dirty environment inside the placenta jointly with the thin diameter of the fetoscope optics and low amount of environment light reduces the image quality. The obtained images are blurred, noisy and with very poor color components. The tracking system has been tested using real video sequences of FLP surgery for TTTS. The computational performance enables real time tracking, locally guiding the robot over the placenta's surface with enough accuracy.

show abstract

Stable Image Registration for In-Vivo Fetoscopic Panorama Reconstruction

Cited by 22 publications

References 18 publications

Inter-foetus Membrane Segmentation for TTTS Using Adversarial Networks

Inter-foetus Membrane Segmentation for TTTS Using Adversarial Networks

Deep-learned placental vessel segmentation for intraoperative video enhancement in fetoscopic surgery

Vision Based Robot Assistance in TTTS Fetal Surgery

Contact Info

Product

Resources

About