Surgical workflow image generation based on generative adversarial networks

Chen, Yu‐Wen; Zhong, Kunhua; Wang, Fei; Wang, Shaoyi; Zhao, Xueliang

doi:10.1109/icaibd.2018.8396171

Cited by 7 publications

(7 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In many studies, GANs were primarily used to generate various imaging modalities such as X-ray, CT, magnetic resonance, positron emission tomography, histopathology images, retinal images, and surgical videos. [56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71] Generated images in the studies were mainly used for data augmentation to have a more balanced dataset for training neural networks of classification or segmentation. With the synthetic images, classification or segmentation accuracies were significant increases than those with the imbalanced dataset.…”

Section: Image Generationmentioning

confidence: 99%

Deep Learning in Medical Imaging

et al. 2019

View full text Add to dashboard Cite

The artificial neural network (ANN), one of the machine learning (ML) algorithms, inspired by the human brain system, was developed by connecting layers with artificial neurons. However, due to the low computing power and insufficient learnable data, ANN has suffered from overfitting and vanishing gradient problems for training deep networks. The advancement of computing power with graphics processing units and the availability of large data acquisition, deep neural network outperforms human or other ML capabilities in computer vision and speech recognition tasks. These potentials are recently applied to healthcare problems, including computer-aided detection/diagnosis, disease prediction, image segmentation, image generation, etc. In this review article, we will explain the history, development, and applications in medical imaging

show abstract

Section: Image Generationmentioning

confidence: 99%

Deep Learning in Medical Imaging

et al. 2019

View full text Add to dashboard Cite

show abstract

“…The use of GANs, where a generator produces synthetic images to resemble a real image distribution while a discriminator is trained in parallel to distinguish between fake and real data, has been applied to surgical workflow image synthesis [15]. However, it was not feasible to produce any labels associated with the synthetic images, making the approach unsuitable for training supervised models.…”

Section: Literature Review: I2i In Surgerymentioning

confidence: 99%

SSIS-Seg: Simulation-Supervised Image Synthesis for Surgical Instrument Segmentation

Colleoni¹,

Psychogyios²,

Amsterdam³

et al. 2022

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Surgical instrument segmentation can be used in a range of computer assisted interventions and automation in surgical robotics. While deep learning architectures have rapidly advanced the robustness and performance of segmentation models, most are still reliant on supervision and large quantities of labelled data. In this paper, we present a novel method for surgical image generation that can fuse robotic instrument simulation and recent domain adaptation techniques to synthesize artificial surgical images to train surgical instrument segmentation models. We integrate attention modules into well established image generation pipelines and propose a novel cost function to support supervision from simulation frames in model training.We provide an extensive evaluation of our method in terms of segmentation performance along with a validation study on image quality using evaluation metrics. Additionally, we release a novel segmentation dataset from real surgeries that will be shared for research purposes. Both binary and semantic segmentation have been considered, and we show the capability of our synthetic images to train segmentation models compared with the latest methods from the literature.

show abstract

“…Similarly, adversarial network with ConvGRU as the core was proposed, mainly to solve the problem of ConvGRU's inability to achieve multi-modal data modelling (Tian et.al 2020). There are also researchers based on the idea of a four-level pyramid convolution structure, and proposed four pairs of models to generate an adversarial network for radar echo prediction (Chen et al 2019). It should be noted that the traditional GAN network has the problem of unstable training, 75 which will cause the model unable to learn.…”

Section: Gan-based Radar Echo Extrapolation 65mentioning

confidence: 99%

A Generative Adversarial Model for Radar Echo Extrapolation Based on Convolutional Recurrent Units

Zheng

Liu

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Abstract. Precipitation nowcasting play a vital role in preventing meteorological disasters and doppler radar data acts as an important input for nowcasting models. The traditional extrapolation method is difficult to model highly nonlinear echo movements. The key challenge of the nowcasting mission lies in achieving high-precision radar echo extrapolation. In recent years, machine learning has made a great progress in the extrapolation of weather radar echoes. However, most of models neglect the multi-modal characteristics of radar echo data, resulting in blurred and unrealistic prediction images. This paper aims to solve this problem by utilizing the feature of the GAN that can enhance the multi-modal distribution modelling, and design the radar echo extrapolation model of GAN-argcPredNet. The model composed of argcPredNet generator and a convolutional neural network discriminator. In generator, a gate control data memory and output are designed in the rgcLSTM prediction unit of the generator, thereby reducing the loss of spatiotemporal information. In discriminator, model uses a dual-channel input method, which enables it to strictly score according to the true echo distribution, and has a more powerful discrimination ability. Through experiments on the radar data set of Shenzhen, China, the results show that the radar echo hit rate (POD) and critical success index (CSI) increased by 5.5 % and %10.4 % compared with rgcPredNet, the false alarm rate (FAR) is reduced by 15 %~20 %. From the comparison of the result graph and the evaluation index, we also found a problem. The recursive prediction method will produce the phenomenon that the prediction result will gradually deviate from the true value over time. In addition, the accuracy of high-intensity echo extrapolation is relatively low. This is a question worthy of further investigation. In the future, we will continue to conduct research from these two directions.

show abstract

Surgical workflow image generation based on generative adversarial networks

Cited by 7 publications

References 6 publications

Deep Learning in Medical Imaging

Deep Learning in Medical Imaging

SSIS-Seg: Simulation-Supervised Image Synthesis for Surgical Instrument Segmentation

A Generative Adversarial Model for Radar Echo Extrapolation Based on Convolutional Recurrent Units

Contact Info

Product

Resources

About