Towards Vivid and Diverse Image Colorization with Generative Color Prior

Wu, Yanze; Wang, Xintao; Liu, Yu; Zhang, Honglun; Zhao, Xun; Shan, Ying

doi:10.1109/iccv48922.2021.01411

Cited by 86 publications

(44 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The vast majority of colorization algorithms [ 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ] use regression loss functions. Cheng et al [ 9 ] extracted image features using a CNN and combined bilateral filtering to enhance colorization.…”

Section: Related Workmentioning

confidence: 99%

“…Su et al [ 19 ] cropped the objects in the image, constructed a multichannel CNN to color each object of the crop and the overall image, and fused multiple color images according to the weights to improve the colorization effect. Wu et al [ 20 ] used GANs to generate color images associated with grayscale images to guide the colorization of grayscale images. Jin et al [ 21 ] constructed a three-channel HistoryNet that contained image category, semantics, and colorization, using categorical and semantic information to guide colorization.…”

Section: Related Workmentioning

confidence: 99%

“…In recent years, with the rapid development of deep learning, a large number of automatic colorization algorithms based on convolutional neural networks (CNNs) have been proposed. However, most colorization algorithms use regression loss functions (such as L 1 and L 2 ) [ 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ]. These algorithms resolve the features of grayscale images and add color channels to achieve colorization.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Colorful Image Colorization with Classification and Asymmetric Feature Fusion

Wang

Yu²,

Li³

et al. 2022

Sensors

View full text Add to dashboard Cite

An automatic colorization algorithm can convert a grayscale image to a colorful image using regression loss functions or classification loss functions. However, the regression loss function leads to brown results, while the classification loss function leads to the problem of color overflow and the computation of the color categories and balance weights of the ground truth required for the weighted classification loss is too large. In this paper, we propose a new method to compute color categories and balance the weights of color images. In this paper, we propose a new method to compute color categories and balance weights of color images. Furthermore, we propose a U-Net-based colorization network. First, we propose a category conversion module and a category balance module to obtain the color categories and to balance weights, which dramatically reduces the training time. Second, we construct a classification subnetwork to constrain the colorization network with category loss, which improves the colorization accuracy and saturation. Finally, we introduce an asymmetric feature fusion (AFF) module to fuse the multiscale features, which effectively prevents color overflow and improves the colorization effect. The experiments show that our colorization network has peak signal-to-noise ratio (PSNR) and structure similarity index measure (SSIM) metrics of 25.8803 and 0.9368, respectively, for the ImageNet dataset. As compared with existing algorithms, our algorithm produces colorful images with vivid colors, no significant color overflow, and higher saturation.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Colorful Image Colorization with Classification and Asymmetric Feature Fusion

Wang

Yu²,

Li³

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Similarly, [Zhao et al, 2018] use mean IoU of segmentation results on the PASCAL VOC2012 dataset [Everingham et al, 2012]. [Wu et al, 2021] use a no-reference measure called colourfulness score [Hasler and Süsstrunk, 2003] which incorporates the means and standard deviations of the a* and b* channels of CIEL*a*b* in a parametric model to compute a measure of how colourful the image is. The parameters were learned from data based on psychophysical experiments .…”

Section: Introductionmentioning

confidence: 99%

“…[Górriz et al, 2019] compare L 1 distance between convolutional features in the VGG19 model [Simonyan and Zisserman, 2015] for ground-truth and colourised samples. Similarly, [Lee et al, 2020] and [Wu et al, 2021] use Fréchet Inception Distance [Heusel et al, 2017], which requires comparing the inception score for colourisations versus ground-truth for 50K samples. [Zhang et al, 2018], developed a perception measure based on the features of deep neural networks called the Learned Perceptual Image Patch Similarity (LPIPS) metric, and this has also been used for the measure of colourisation in [Su et al, 2020, Yoo et al, 2019, Kim et al, 2021.…”

Section: Introductionmentioning

confidence: 99%

Human vs Objective Evaluation of Colourisation Performance

Mullery¹,

Whelan²

2022

Preprint

View full text Add to dashboard Cite

Automatic colourisation of grey-scale images is the process of creating a full-colour image from the grey-scale prior. It is an ill-posed problem, as there are many plausible colourisations for a given grey-scale prior. The current SOTA in auto-colourisation involves image-to-image type Deep Convolutional Neural Networks with Generative Adversarial Networks showing the greatest promise. The end goal of colourisation is to produce full colour images that appear plausible to the human viewer, but human assessment is costly and time consuming. This work assesses how well commonly used objective measures correlate with human opinion. We also attempt to determine what facets of colourisation have the most significant effect on human opinion. For each of 20 images from the BSD dataset, we create 65 recolourisations made up of local and global changes. Opinion scores are then crowd sourced using the Amazon Mechanical Turk and together with the images this forms an extensible dataset called the Human Evaluated Colourisation Dataset (HECD). While we find statistically significant correlations between human-opinion scores and a small number of objective measures, the strength of the correlations is low. There is also evidence that human observers are most intolerant to an incorrect hue of naturally occurring objects.

show abstract

Real‐time coloring method of laser surgery video based on generative adversarial network

Pan¹,

Lian

et al. 2023

Int J Imaging Syst Tech

View full text Add to dashboard Cite

Lasers have become an important technology for the medical treatment of cancer. However, the high power of visible laser knives makes conventional endoscopic imaging light pollution severe. This phenomenon can cause the physician to be unable to determine the condition of the lesion site. In this article, a colorization method for laser surgery videos is proposed. The method performs real-time colorization of the laser surgery video under black-andwhite imaging. The main contribution of this work is to propose a new idea for solving laser contamination in medical surgery videos, along with real-time generative adversarial networks (RTGANs). The network takes advantage of generative adversarial networks and residual networks for fast colorization of black and white images. To evaluate the method, we conducted experiments using three datasets. The experimental results show that the RTGAN network takes only 0.026 s to colorize a single frame image. Coloring speed up to 67 times faster than advanced methods, and the Peak Signal to Noise Ratio reached a maximum of 35.736.

show abstract

Towards Vivid and Diverse Image Colorization with Generative Color Prior

Cited by 86 publications

References 45 publications

Colorful Image Colorization with Classification and Asymmetric Feature Fusion

Colorful Image Colorization with Classification and Asymmetric Feature Fusion

Human vs Objective Evaluation of Colourisation Performance

Real‐time coloring method of laser surgery video based on generative adversarial network

Contact Info

Product

Resources

About