Remote sensing image colorization using symmetrical multi-scale DCGAN in YUV color space

Wu, Min; Jin, Xin; Jiang, Qian; Lee, Shin-Jye; Liang, Wentao; Guo, Lin; Yao, Shaowen

doi:10.1007/s00371-020-01933-2

Cited by 23 publications

(12 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wu et al [9] combined multi-scale convolution with a squeezeexcitation network (SEnet) based on DCGAN to preserve effective image features during image generation and adjust the channel weights during training. Based on this work, Wu et al transferred the coloring task from RGB to YUV color space and used the multi-scale convolution to optimize the coloring effect [33]. Feng et.…”

Section: Automatic Coloring Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Remote Sensing Colorization Based on Bidirectional Macro-Micro Adaptive Enhancement Network

Wang

Yang

et al. 2022

IEEE Access

View full text Add to dashboard Cite

The demand for re-colorization of remote sensing images is urgent since image quality is extremely deteriorated by haze or other noises occurring in the atmospheric layer. The most challenging issue is to restore the color information with respect to preserving spatial consistency as well as to obtain object salience in context with extremely imbalanced space structure, where the former requires learning stable macroscopic semantics while the latter needs to recover microscopic pixels. In this paper, we propose a Bidirectional Macro-Micro Adaptive Enhancement (BMMAEnet) framework by adopting three modules, i.e., the Downward Micro Enhancement (DME) module, the Upward Adaptive Macro Enhancement(UAME) module, and Macro-Micro Balance (MMB) module. Firstly, the DME module is designed by adding micro details as well as suppressing macro context during the multi-branch downsampling process to supplement missing pixel details. Secondly, UAME is proposed by adaptive selecting proper level of features during multi-branch upsampling process to strengthen macro semantic constraints. In addition, MMB is designed by embedding attention-guided local details and global semantics into the decoding features to balance micro and macro information within each branch. Comprehensive comparison and ablation experiments are implemented and verify the proposed method performs overpass SOTA methods not only in pixel color value restoration performance but also in human perceptive understanding.

show abstract

Section: Automatic Coloring Methodsmentioning

confidence: 99%

“…e: Wu et al's Method [33] Use multi-scale DCGAN to color remote sensing images, and multi-scale is implemented by convolution operation with different kernel sizes in the generator.…”

Section: D: Amazon Mechanical Turk(amt) Perception Testmentioning

confidence: 99%

Remote Sensing Colorization Based on Bidirectional Macro-Micro Adaptive Enhancement Network

Wang

Yang

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Markchom et al [40] explored an algorithm to remove clouds in HSI color space, which eliminates clouds only in the intensity channel to avoid the influence on the original color. Wu et al [41] considered the transformation (or inverse transformation) between YUV and RGB is linear, realizing the remote sensing image colorization in YUV color space, which will not generate a nonlinear error in the processes of color transformation and prediction. Furthermore, the advantage of the YUV color space compared with the RGB color spaces is that it can represent luminance and chromatic information independently.…”

Section: Image Reconstruction In Different Color Spacesmentioning

confidence: 99%

“…RGB values can be transformed into YUV color space that considers color sensitivity through Equation (1). Additionally, YUV values can be transformed inversely into RGB color space through Equation ( 2) [41].…”

Section: Color Space Transformation and Inverse Transformationmentioning

confidence: 99%

Generative Adversarial Learning in YUV Color Space for Thin Cloud Removal on Satellite Imagery

Wen

Pan

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

Clouds are one of the most serious disturbances when using satellite imagery for ground observations. The semi-translucent nature of thin clouds provides the possibility of 2D ground scene reconstruction based on a single satellite image. In this paper, we propose an effective framework for thin cloud removal involving two aspects: a network architecture and a training strategy. For the network architecture, a Wasserstein generative adversarial network (WGAN) in YUV color space called YUV-GAN is proposed. Unlike most existing approaches in RGB color space, our method performs end-to-end thin cloud removal by learning luminance and chroma components independently, which is efficient at reducing the number of unrecoverable bright and dark pixels. To preserve more detailed features, the generator adopts a residual encoding–decoding network without down-sampling and up-sampling layers, which effectively competes with a residual discriminator, encouraging the accuracy of scene identification. For the training strategy, a transfer-learning-based method was applied. Instead of using either simulated or scarce real data to train the deep network, adequate simulated pairs were used to train the YUV-GAN at first. Then, pre-trained convolutional layers were optimized by real pairs to encourage the applicability of the model to real cloudy images. Qualitative and quantitative results on RICE1 and Sentinel-2A datasets confirmed that our YUV-GAN achieved state-of-the-art performance compared with other approaches. Additionally, our method combining the YUV-GAN with a transfer-learning-based training strategy led to better performance in the case of scarce training data.

show abstract

“…One of the earliest works using texture information for this task is [ 27 ]. In recent years, GANs (in particular cGANs) have become a popular approach for such a challenge, in particular, in the remote sensing domain [ 20 , 28 ]. In the image colorization task, cGANs take a condition that should be utilized for new image generation.…”

Section: Introductionmentioning

confidence: 99%

Generation of the NIR Spectral Band for Satellite Images with Convolutional Neural Networks

Illarionova

Shadrin

Trekin

et al. 2021

Sensors

View full text Add to dashboard Cite

The near-infrared (NIR) spectral range (from 780 to 2500 nm) of the multispectral remote sensing imagery provides vital information for landcover classification, especially concerning vegetation assessment. Despite the usefulness of NIR, it does not always accomplish common RGB. Modern achievements in image processing via deep neural networks make it possible to generate artificial spectral information, for example, to solve the image colorization problem. In this research, we aim to investigate whether this approach can produce not only visually similar images but also an artificial spectral band that can improve the performance of computer vision algorithms for solving remote sensing tasks. We study the use of a generative adversarial network (GAN) approach in the task of the NIR band generation using only RGB channels of high-resolution satellite imagery. We evaluate the impact of a generated channel on the model performance to solve the forest segmentation task. Our results show an increase in model accuracy when using generated NIR compared to the baseline model, which uses only RGB (0.947 and 0.914 F1-scores, respectively). The presented study shows the advantages of generating the extra band such as the opportunity to reduce the required amount of labeled data.

show abstract

Remote sensing image colorization using symmetrical multi-scale DCGAN in YUV color space

Cited by 23 publications

References 44 publications

Remote Sensing Colorization Based on Bidirectional Macro-Micro Adaptive Enhancement Network

Remote Sensing Colorization Based on Bidirectional Macro-Micro Adaptive Enhancement Network

Generative Adversarial Learning in YUV Color Space for Thin Cloud Removal on Satellite Imagery

Generation of the NIR Spectral Band for Satellite Images with Convolutional Neural Networks

Contact Info

Product

Resources

About