A Novel Multispectral Maritime Target classification based on ThermalGAN (RGB-to-Thermal Image Translation)

Abstract

Convolutional Neural Networks (CNN) for ship classification in multi-spectral images (RGB, IR, etc.) is proposed in this paper. Recent developments in deep learning have significantly advanced the field of ship recognition. However, since maritime light intensity is frequently disturbed, multispectral imaging is considered a more robust substitute for RGB imaging. The proposed architectures were fine-tuned after being trained from scratch on the publicly available dataset VAIS (RGB-IR pairs). Unfortunately, the classification results plateaued at 59.74% accuracy, which is unsatisfactory for most real-life applications. Such an accuracy wall was due to the small number of training images. In order to overcome the scarcity of IR ship images, we proposed a novel image data augmentation strategy that translates RGB images to IR. A Pix2Pix model and a Generative Adversarial Network (GAN) network were modified to carry out the generation process as an RGB to IR translator. The KAIST general-purpose RGB-IR image pairs dataset was used to train our RGB-to-IR image translator, whereas the VAIS dataset was held aside for validation purposes. Our proposed network improved the accuracy of the native network by 8% ( from 59.74% to 67.74%), which is fairly satisfactory in the field of ship recognition.

Thermal image analytics for industrial anomaly detection

“…They found that their proposed UNet model gave the best IoU results when compared with existing UNet models. In [79], a Pix2Pix based ThermalGAN has been proposed for translating maritime visual images to thermal images with the motivation of data augmentation and for creating a larger IR dataset for higher thermal image based classification accuracy. In the proposed model only six downsampling and upsampling blocks were used in the encoder and decoder respectively.…”