Enhanced Night-to-Day Image Conversion Using CycleGAN-Based Base-Detail Paired Training

Abstract

Numerous studies are underway to enhance the identification of surroundings in nighttime environments. These studies explore methods such as utilizing infrared images to improve night image visibility or converting night images into day-like representations for enhanced visibility. This research presents a technique focused on converting the road conditions depicted in night images to resemble daytime scenes. To facilitate this, a paired dataset is created by augmenting limited day and night image data using CycleGAN. The model is trained using both original night images and single-scale luminance transform (SLAT) day images to enhance the level of detail in the converted daytime images. However, the generated daytime images may exhibit sharpness and noise issues. To address these concerns, an image processing approach, inspired by the Stevens effect and local blurring, which align with visual characteristics, is employed to reduce noise and enhance image details. Consequently, this study contributes to improving the visibility of night images by means of day image conversion and subsequent image processing. The proposed night-to-day image translation in this study has a processing time of 0.81 s, including image processing, which is less than one second. Therefore, it is considered valuable as a module for daytime image translation. Additionally, the image quality assessment metric, BRISQUE, yielded a score of 19.8, indicating better performance compared to conventional methods. The outcomes of this research hold potential applications in fields such as CCTV surveillance systems and self-driving cars.

Review and Experimental Comparison of Generative Adversarial Networks for Synthetic Image Generation

“…The CycleGAN network belongs to the advanced category of generative adversarial networks (GANs) and is widely used for image transformations. Unlike Pix2Pix, CycleGAN utilizes two unrelated datasets, addressing the difficulty and costs associated with assembling paired training data (Son et al, 2023). For example, in tasks such as semantic image segmentation, only a few combined datasets currently exist, and even these datasets often lack sufficient data (Zhu et al, 2017).…”

“…Such network is well suited for medical imaging, the manipulation of objects within images, and the improvement of photo quality. However, challenges arise in modifying video geometry, and the generated images often closely resemble the originals, limiting diverse transformations (Son et al, 2023).…”

Nighttime road scene image enhancement based on cycle-consistent generative adversarial network

“…They also designed the Harr down-sampling layer to separate high and low-frequency signals 29 . Son et al proposed to utilize the Stevens effect and local blur map to process the enhanced night road images by the cycle-consistent generative adversarial network to reduce the noise and enhance detail information 30 . Chen et al proposed an improved generative adversarial network to enhance the image quality of nighttime images and rain images by introducing the attention mechanism modules and the multiscale feature fusion modules into the generator network and local discrimination strategy into the discriminator 31 .…”

A High-Quality Hybrid Mapping Model Based on Averaging Dense Sampling Parameters

“…WA is a technique which fuses the deep learning models and aims to improve the model. Some recent work [32] uses Stochastic Weighted Averaging (SWA) [33] in Cy-cleGANs [34,35], which enhances the capabilities of generators. Inspired by checkpoint averaging [36,37], Izmailov et al [33] propose SWA.…”