<p><strong>Abstract.</strong> Nowadays methods based on deep neural networks show the best performance among image recognition and object detection algorithms. Nevertheless, such methods require to have large databases of multispectral images of various objects to achieve state-of-the-art results. Therefore the dataset generation is one of the major challenges for the successful training of a deep neural network. However, infrared image datasets that are large enough for successful training of a deep neural network are not available in the public domain. Generation of synthetic datasets using 3D models of various scenes is a time-consuming method that requires long computation time and is not very realistic. This paper is focused on the development of the method for thermal image synthesis using a GAN (generative adversarial network). The aim of the presented work is to expand and complement the existing datasets of real thermal images. Today, deep convolutional networks are increasingly used for the goal of synthesizing various images. Recently a new generation of such algorithms commonly called GAN has become a promising tool for synthesizing images of various spectral ranges. These networks show effective results for image-to-image translations. While it is possible to generate a thermal texture for a single object, generation of environment textures is extremely difficult due to the presence of a large number of objects with different emission sources. The proposed method is based on a joint approach that uses 3D modeling and deep learning. Synthesis of background textures and objects textures is performed using a generative-adversarial neural network and semantic and geometric information about objects generated using 3D modeling. The developed approach significantly improves the realism of the synthetic images, especially in terms of the quality of background textures.</p>

No abstract

Abstract. Situational awareness of the crew is critical for the safety of the air flight. Head-up display allows providing all required flight information in front of the pilot over the cockpit view visible through the cockpit’s front window. This device has been created for solving the problem of informational overload during piloting of an aircraft. While computer graphics such as scales and digital terrain model can be easily presented on the such display, errors in the Head-up display alignment for correct presenting of sensor data pose challenges. The main problem arises from the parallax between the pilot’s eyes and the position of the camera. This paper is focused on the development of an online calibration algorithm for conform projection of the 3D terrain and runway models on the pilot’s head-up display. The aim of our algorithm is to align the objects visible through the cockpit glass with their projections on the Head-up display. To improve the projection accuracy, we use an additional optical sensor installed on the aircraft. We combine classical photogrammetric techniques with modern deep learning approaches. Specifically, we use an object detection neural network model to find the runway area and align runway projection with its actual location. Secondly, we re-project the sensor’s image onto the 3D model of the terrain to eliminate errors caused by the parallax. We developed an environment simulator to evaluate our algorithm. Using the simulator we prepared a large training dataset. The dataset includes 2000 images of video sequences representing aircraft’s motion during takeoff, landing and taxi. The results of the evaluation are encouraging and demonstrate both qualitatively and quantitatively that the proposed algorithm is capable of precise alignment of the 3D models projected on a Head-up display.