UMGAN: Generative adversarial network for image unmosaicing using perceptual loss

Javed, Kamran; Din, Nizam Ud; Bae, Seho; Maharjan, Rahul Singh; Seo, Donghwan; Yi, Juneho

doi:10.23919/mva.2019.8757902

Cited by 11 publications

(12 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The reason is that the objective of image editing techniques is to synthesize realistic-looking content rather than the exact same content as the original image. Therefore, we argue that as reported by many other works [7], [40], quantitative analysis may not be the most effective measure of the image editing task.…”

Section: ) Quantitative Comparisonmentioning

confidence: 66%

A Novel GAN-Based Network for Unmasking of Masked Face

et al. 2020

Self Cite

View full text Add to dashboard Cite

Recent deep learning based image editing methods have achieved promising results for removing object in an image but fail to generate plausible results for removing large objects of complex nature, especially in facial images. The objective of this work is to remove mask objects in facial images. This problem is challenging because (1) most of the time facial masks cover quite a large region of face that even extends beyond the actual face boundary below chin, and (2) facial image pairs with and without mask object do not exist for training. We break the problem into two stages: mask object detection and image completion of the removed mask region. The first stage of our model automatically produces binary segmentation for the mask region. Then, the second stage removes the mask and synthesizes the affected region with fine details while retaining the global coherency of face structure. For this, we have employed a GAN-based network using two discriminators where one discriminator helps learn the global structure of the face and then another discriminator comes in to focus learning on the deep missing region. To train our model in a supervised manner, we create a paired synthetic dataset using publicly available CelebA dataset and evaluated on real world images collected from the Internet. Our model outperforms others representative state-of-the-art approaches both qualitatively and quantitatively. INDEX TERMS Generative adversarial network, object removal, image editing.

show abstract

Section: ) Quantitative Comparisonmentioning

confidence: 66%

A Novel GAN-Based Network for Unmasking of Masked Face

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Quantitative comparison: Table 1 shows a quantitative comparison of our model with state-of-the art methods. As mentioned in [8] and [38], there is no good quantitative metric available to evaluate image editing results because the goal of these methods is to not to generate exact content, but to produce realistic-looking content. Nonetheless, we measure the quantitative performance using four popular metrics: 1) Structural SIMilarity (SSIM) [44], 2) Peak Signal to Noise Ratio (PSNR), 3) Naturalness Image Quality Evaluator (NIQE) [47], and 4) Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) [48].…”

Section: Results and Comparisonmentioning

confidence: 99%

“…A symmetric shape allows a network to have a large number of feature maps in the expansive path, facilitating the transfer of more information. Due to its simple architecture and better performance, other work has also exploited the U-Net architecture with minor modifications for image inpainting [17], multiple sketch styles generation [37], image unmosaicing [11], [38], object removal [9] and object detection in facial images [16]. Nizam et al [16] employed a simple U-Net architecture to efficiently detect medical masks in facial images.…”

Section: Related Workmentioning

confidence: 99%

“…Nizam et al [16] employed a simple U-Net architecture to efficiently detect medical masks in facial images. Some models [9], [11], [37], [38] improved performance by using a discriminator along with the U-Net architecture. We also use the U-Net architecture along with a discriminator and target object label encoder to detect multiple types of nonface objects in facial images.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Effective Removal of User-Selected Foreground Object From Facial Images Using a Novel GAN-Based Network

et al. 2020

Self Cite

View full text Add to dashboard Cite

This research features a user-friendly method for face de-occlusion in facial images where the user has control of which object to remove. Our system removes one object at a time, however, it is capable of removing multiple objects through repeated application. Although we show the effectiveness of our system on five commonly occurring occluding objects including hands, a medical mask, microphone, sunglasses, and eyeglasses, more types of object can be considered based on the proposed methodology. Our model learns to detect a user-selected, possibly distracting, object in the first stage. Then, the second stage removes the object using the object detection information from the first stage as guidance. To achieve this, we employ GAN-based networks in both stages. Specifically, in the second stage, we integrate both partial and vanilla convolution operations in the generator part of the GAN network. We show that by using this integration, the proposed network can learn a well-incorporated structure and also overcome the problem of visual discrepancies in the affected region of the face. To train our network, we produce a paired synthetic face-occluded dataset. Our model is evaluated using real world images collected from the Internet and publicly available CelebA and CelebA-HQ datasets. Experimental results confirm our model's effectiveness in removing challenging foreground non-face objects from facial images as compared to the existing representative state-of-the-art approaches.

show abstract

“…The BCE loss function tries to maximize the difference of the probability distribution between two classes, in this case, lesion and nonlesion voxels [ 7 ]. SSIM, on the other hand, is a perception-based loss function that quantifies the similarity between two images [ 8 ].…”

Section: Introductionmentioning

confidence: 99%

Multiple Sclerosis Lesion Segmentation in Brain MRI Using Inception Modules Embedded in a Convolutional Neural Network

Ansari

Javed

Qaisar

et al. 2021

Journal of Healthcare Engineering

Self Cite

View full text Add to dashboard Cite

Multiple sclerosis (MS) is a chronic and autoimmune disease that forms lesions in the central nervous system. Quantitative analysis of these lesions has proved to be very useful in clinical trials for therapies and assessing disease prognosis. However, the efficacy of these quantitative analyses greatly depends on how accurately the MS lesions have been identified and segmented in brain MRI. This is usually carried out by radiologists who label 3D MR images slice by slice using commonly available segmentation tools. However, such manual practices are time consuming and error prone. To circumvent this problem, several automatic segmentation techniques have been investigated in recent years. In this paper, we propose a new framework for automatic brain lesion segmentation that employs a novel convolutional neural network (CNN) architecture. In order to segment lesions of different sizes, we have to pick a specific filter or size 3 × 3 or 5 × 5. Sometimes, it is hard to decide which filter will work better to get the best results. Google Net has solved this problem by introducing an inception module. An inception module uses 3 × 3 , 5 × 5 , 1 × 1 and max pooling filters in parallel fashion. Results show that incorporating inception modules in a CNN has improved the performance of the network in the segmentation of MS lesions. We compared the results of the proposed CNN architecture for two loss functions: binary cross entropy (BCE) and structural similarity index measure (SSIM) using the publicly available ISBI-2015 challenge dataset. A score of 93.81 which is higher than the human rater with BCE loss function is achieved.

show abstract

UMGAN: Generative adversarial network for image unmosaicing using perceptual loss

Cited by 11 publications

References 13 publications

A Novel GAN-Based Network for Unmasking of Masked Face

A Novel GAN-Based Network for Unmasking of Masked Face

Effective Removal of User-Selected Foreground Object From Facial Images Using a Novel GAN-Based Network

Multiple Sclerosis Lesion Segmentation in Brain MRI Using Inception Modules Embedded in a Convolutional Neural Network

Contact Info

Product

Resources

About