A Study on the Elimination of Thermal Reflections

Abstract

Recently, thermal cameras have been used in various surveillance and monitoring systems. In particular, in camera-based surveillance systems, algorithms are being developed for detecting and recognizing objects from images acquired in dark environments. However, it is difficult to detect and recognize an object due to the thermal reflections generated in the image obtained from a thermal camera. For example, thermal reflection often occurs on a structure or the floor near an object, similar to shadows or mirror reflections. In this case, the object and the areas of thermal reflection overlap or are connected to each other and are difficult to separate. Thermal reflection also occurs on nearby walls, which can be detected as artifacts when an object is not associated with this phenomenon. In addition, the size and pixel value of the thermal reflection area vary greatly depending on the material of the area and the environmental temperature. In this case, the patterns and pixel values of the thermal reflection and the object are similar to each other and difficult to differentiate. These problems reduce the accuracy of object detection and recognition methods. In addition, no studies have been conducted on the elimination of thermal reflection of objects under different environmental conditions. Therefore, to address these challenges, we propose a method of detecting reflections in thermal images based on deep learning and their elimination via post-processing. Experiments using a self-collected database (Dongguk thermal image database (DTh-DB), Dongguk items and vehicles database (DI&V-DB)) and an open database showed that the performance of the proposed method is superior compared to that of other state-of-the-art approaches. INDEX TERMS Thermal reflection, thermal imaging, deep learning, thermal reflection detection, thermal reflection removal.

Region-Based Removal of Thermal Reflection Using Pruned Fully Convolutional Network

“…In this section, the comparison results of the proposed method and the state-of-the-art methods are provided. For the comparison, the accuracies of seven types of methods i.e., CycleGAN [44], PLN [10], Mask R-CNN + PLN [12], SegNet [43]-based removal method [12], Mask R-CNN [18]based removal method [12], FCN_V1 [46], and FCN_V2 [46] are compared with the accuracy of the method proposed in this study. Based on the original parameters provided by authors, the optimal parameters of these seven types of methods were obtained by the further procedure of fine-tuning with the training dataset of our experimental data.…”

“…It can measure the temperature from -40 • C to +80 • C to make objects visible in both light and dark environments. The database (an image has the depth of 14 bits and the size of 640 × 480 pixels [12]) obtained using the thermal camera was used in the experiment. A mask region CNN (Mask R-CNN) was used to detect the approximate region (input region image in Figure 3) of thermal reflection in input images, and the detailed explanation is provided in [12].…”

“…In [12], a thermal reflection elimination method is proposed. The method is conducted using Mask R-CNN [18] to detect thermal reflection regions in thermal images.…”

“…The database [12] used in this study consists of thermal images of objects at close and far distances in both dark and bright environments. The database was collected in both indoor and outdoor environments.…”

“…Furthermore, the database also includes visible light images. The details of the database are shown in Tables 3 and 4, and Figures 8-11 in our previous work [12]. Figure 8 shows the examples of the images in the database.…”