Marker Based Thermal-Inertial Localization for Aerial Robots in Obscurant Filled Environments

Abstract

For robotic inspection tasks in known environments fiducial markers provide a reliable and low-cost solution for robot localization. However, detection of such markers relies on the quality of RGB camera data, which degrades significantly in the presence of visual obscurants such as fog and smoke. The ability to navigate known environments in the presence of obscurants can be critical for inspection tasks especially, in the aftermath of a disaster. Addressing such a scenario, this work proposes a method for the design of fiducial markers to be used with thermal cameras for the pose estimation of aerial robots. Our low cost markers are designed to work in the long wave infrared spectrum, which is not affected by the presence of obscurants, and can be affixed to any object that has measurable temperature difference with respect to its surroundings. Furthermore, the estimated pose from the fiducial markers is fused with inertial measurements in an extended Kalman filter to remove high frequency noise and error present in the fiducial pose estimates. The proposed markers and the pose estimation method are experimentally evaluated in an obscurant filled environment using an aerial robot carrying a thermal camera.

Low complexity visual UAV track navigation using long‐wavelength infrared

“…Using radiometric LWIR imagery only, Khattak et al (2018) created a thermally isolated fiducial marker to allow localization in a challenging but cooperative workshop scene. The ability to recognize the thermal difference between fiduciary markers and background machines in the scene was enabled by the radiometric measurements.…”

A Review of Modern Thermal Imaging Sensor Technology and Applications for Autonomous Aerial Navigation

“…The study by [ 48 ] utilised this concept by applying thermal markers made of thin 5 mm acrylic sheets over objects in smoke filled environments. Due to high emissivity values of acrylic at 0.88 [ 49 ], the thermal markers are distinct in the thermal frame, which later could be used for map-building SLAM techniques.…”

“…Another study by Khattak et al [ 48 ] used a LWIR sensor alone to detect low thermal conductivity fiducial markers in order to localise in a dark indoor scene. The team attached a thermal fiducial marker to fixed objects around the environment in an incremental manner.…”

“…In this case, thermal data were the only input for navigation algorithms. At first, re-scaled 8 bit thermal data were utilised for stereo odometry [ 61 , 83 ], SLAM [ 48 ] and optical flow [ 63 ]. The disadvantages of 8 bit re-scaled data have been discussed in Section 5.2.1 .…”

“…Firstly, the authors [ 84 ] utilised 8 bit thermal data with the extended Kalman filter framework for feature extraction and matching. Later, they experimented with thermal markers [ 48 ] with very high emissivity, as described in Section 4.3 . The thermal markers were applied throughout the test site as distinctive landmarks that are visible in thermal frames that could be used for odometry estimation.…”