DeformableGAN: Generating Medical Images With Improved Integrity for Healthcare Cyber Physical Systems

Shen, Zhangyi; Ding, Feng; Jolfaei, Alireza; Yadav, Kusum; Vashisht, Sahil; Yu, Keping

doi:10.1109/tnse.2022.3190765

Cited by 28 publications

(13 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, we showed that artifacts are prevalent (in two out of every three cases) in CGANsimulated mammograms, as previous studies have reported for different domains and tasks. [8][9][10]18 From the simulated mammograms generated by our CGAN algorithm, we identified one network design artifact, which was the checkerboard artifact, and three application-specific artifacts, which were the breast boundary, nipple-areolar complex, and black spot artifacts. We found that these artifacts could appear in any mammograms regardless of the existence of MO cancer, but checkerboard and breast boundary artifacts were more common than the other two artifacts.…”

Section: Discussionmentioning

confidence: 99%

“…Shen et al 9 studied where and why checkerboard artifacts happen in medical images (vessel segmentation in retinal images). They found that checkerboard artifacts could be generated anywhere in the image, when the length of stride was smaller than the size of the convolution kernel.…”

Section: Discussionmentioning

confidence: 99%

“…It can cause so-called checkerboard artifacts when its stride value is smaller (but greater than 2) than its kernel size. 9 In addition to this checkerboard artifact, other common signatures that a single model could easily identify exist. 8 The second category is an application-specific artifact for the case of the resulting GAN simulation not following the typical characteristics (or distribution) of the target images/objects.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Impact of GAN artifacts for simulating mammograms on identifying mammographically occult cancer

Lee,

Mustafaev,

Nishikawa

2023

J. Med. Imag.

View full text Add to dashboard Cite

.PurposeGenerative adversarial networks (GANs) can synthesize various feasible-looking images. We showed that a GAN, specifically a conditional GAN (CGAN), can simulate breast mammograms with normal, healthy appearances and can help detect mammographically-occult (MO) cancer. However, similar to other GANs, CGANs can suffer from various artifacts, e.g., checkerboard artifacts, that may impact the quality of the final synthesized image, as well as the performance of detecting MO cancer. We explored the types of GAN artifacts that exist in mammogram simulations and their effect on MO cancer detection.ApproachWe first trained a CGAN using digital mammograms (FFDMs) of 1366 women with normal/healthy breasts. Then, we tested the trained CGAN on an independent MO cancer dataset with 333 women with dense breasts (97 MO cancers). We trained a convolutional neural network (CNN) on the MO cancer dataset, in which real and simulated mammograms were fused, to identify women with MO cancer. Then, a radiologist who was independent of the development of the CGAN algorithms evaluated the entire MO cancer dataset to identify and annotate artifacts in the simulated mammograms.ResultsWe found four artifact types, including checkerboard, breast boundary, nipple-areola complex, and black spots around calcification artifacts, with an overall incidence rate over 69% (the individual incident rate ranged from 9% to 53%) from both normal and MO cancer samples. We then evaluated their potential impact on MO cancer detection. Even though various artifacts existed in the simulated mammogram, we found that it still provided complementary information for MO cancer detection when it was combined with the real mammograms.ConclusionsWe found that artifacts were pervasive in the CGAN-simulated mammograms. However, they did not negatively affect our MO cancer detection algorithm; the simulated mammograms still provided complementary information for MO cancer detection when combined with real mammograms.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Impact of GAN artifacts for simulating mammograms on identifying mammographically occult cancer

Lee,

Mustafaev,

Nishikawa

2023

J. Med. Imag.

View full text Add to dashboard Cite

show abstract

“…However, most feeding systems are based on human experience or computerized feeding frequency and times. The feeding frequency of the bait can change depending on the type of bait, and thus the feeding frequency cannot be obtained accurately 29 . The solution of feeding time and feeding amount is the key method to improve breeding efficiency.…”

Section: Preliminariesmentioning

confidence: 99%

(Retracted) Dynamic scene images-assisted intelligent control method for industrialized feeding through deep vision learning

et al. 2022

View full text Add to dashboard Cite

In a recirculating aquaculture system (RAS), feeding is an important factor affecting the growth of breeding objects. The traditional feeding methods relied on manual experience, which resulted in high labor costs and bait waste. To deal with these challenges, this paper proposes a dynamic scene images-assisted intelligent control method for industrial feeding through deep vision learning. First, a feeding video is processed according to the interframe difference method to obtain the image of the feeding state of the fish. Then, a modified VGG16 model is developed to determine the feeding state of the fish, transform it into a binary classification problem, and calculate the feeding frequency of the fish. After that, residual bait detection is deployed by adapting the YOLOv5 model. The results of the feeding frequency and the residual bait detection are then used to develop an intelligent feeding strategy to improve the growth rate of the fish and the conversion rate of the bait. Experimental tests on real-world scene images showed that the accuracy of identifying the feeding state by the modified VGG16 model reaches 92.4%. Through the verification of the medium-size RAS, compared with the traditional feeding method, the intelligent feeding method significantly saves manpower and reduce 15% of bait waste.

show abstract

“…In agriculture, 3D reconstruction facilitates the improvement of vehicle navigation, crop, and animal husbandry [ 5 ]. In medical imaging, reconstruction produces enhanced visualizations which are used to assist diagnostics [ 6 ].…”

Section: Introductionmentioning

confidence: 99%

A Sequential Color Correction Approach for Texture Mapping of 3D Meshes

Dal’Col

Coelho

Madeira

et al. 2023

Sensors

View full text Add to dashboard Cite

Texture mapping can be defined as the colorization of a 3D mesh using one or multiple images. In the case of multiple images, this process often results in textured meshes with unappealing visual artifacts, known as texture seams, caused by the lack of color similarity between the images. The main goal of this work is to create textured meshes free of texture seams by color correcting all the images used. We propose a novel color-correction approach, called sequential pairwise color correction, capable of color correcting multiple images from the same scene, using a pairwise-based method. This approach consists of sequentially color correcting each image of the set with respect to a reference image, following color-correction paths computed from a weighted graph. The color-correction algorithm is integrated with a texture-mapping pipeline that receives uncorrected images, a 3D mesh, and point clouds as inputs, producing color-corrected images and a textured mesh as outputs. Results show that the proposed approach outperforms several state-of-the-art color-correction algorithms, both in qualitative and quantitative evaluations. The approach eliminates most texture seams, significantly increasing the visual quality of the textured meshes.

show abstract

DeformableGAN: Generating Medical Images With Improved Integrity for Healthcare Cyber Physical Systems

Cited by 28 publications

References 46 publications

Impact of GAN artifacts for simulating mammograms on identifying mammographically occult cancer

Impact of GAN artifacts for simulating mammograms on identifying mammographically occult cancer

(Retracted) Dynamic scene images-assisted intelligent control method for industrialized feeding through deep vision learning

A Sequential Color Correction Approach for Texture Mapping of 3D Meshes

Contact Info

Product

Resources

About