Generative Adversarial Network for Generating Synthetic Infrared Image from Visible Image

Abstract

One of the most important discoveries in the field of deep learning in recent years is the Generative Adversarial Networks (GAN). It offers great convenience and flexibility for image-to-image conversion processes. This study aims to obtain thermal images from visible band colour images by using Pix2Pix Network, which is a Conditionally Generative Adversarial Network (cGAN). For this purpose, a data set has been prepared by taking facial images at different angles in the visible and infrared bands. By applying image processing methods on this created dataset, pixel-by-pixel matching process was performed. Synthetic thermal face images were obtained thanks to this learning network fed with facial images consisting of visible and long wavelength infrared image (LWIR) pairs. In the generator and discriminator deep networks of the Pix2Pix GAN, Batch Normalization and Instance Normalization methods are applied and their effects on the outputs are examined. The same process has also been tested on the Google Maps dataset and thus its effects on different datasets has been demonstrated. Similarity values between synthetic outputs and real images of both studies has been calculated with several image quality metrics. The performance of this generated model in creating the details in the infrared band is also given in the conclusion section.