Overcoming the Challenges of Thermal Infrared Orthomosaics Using a Swath-Based Approach to Correct for Dynamic Temperature and Wind Effects

Abstract

The miniaturization of thermal infrared sensors suitable for integration with unmanned aerial vehicles (UAVs) has provided new opportunities to observe surface temperature at ultra-high spatial and temporal resolutions. In parallel, there has been a rapid development of software capable of streamlining the generation of orthomosaics. However, these approaches were developed to process optical and multi-spectral image data and were not designed to account for the often rapidly changing surface characteristics inherent in the collection and processing of thermal data. Although radiometric calibration and shutter correction of uncooled sensors have improved, the processing of thermal image data remains difficult due to (1) vignetting effects on the uncooled microbolometer focal plane array; (2) inconsistencies between images relative to in-flight effects (wind-speed and direction); (3) unsuitable methods for thermal infrared orthomosaic generation. Here, we use thermal infrared UAV data collected with a FLIR-based TeAx camera over an agricultural field at different times of the day to assess inconsistencies in orthophotos and their impact on UAV-based thermal infrared orthomosaics. Depending on the wind direction and speed, we found a significant difference in UAV-based surface temperature (up to 2 °C) within overlapping areas of neighboring flight lines, with orthophotos collected with tail wind being systematically cooler than those with head wind. To address these issues, we introduce a new swath-based mosaicking approach, which was compared to three standard blending modes for orthomosaic generation. The swath-based mosaicking approach improves the ability to identify rapid changes of surface temperature during data acquisition, corrects for the influence of flight direction relative to the wind orientation, and provides uncertainty (pixel-based standard deviation) maps to accompany the orthomosaic of surface temperature. It also produced more accurate temperature retrievals than the other three standard orthomosaicking methods, with a root mean square error of 1.2 °C when assessed against in situ measurements. As importantly, our findings demonstrate that thermal infrared data require appropriate processing to reduce inconsistencies between observations, and thus, improve the accuracy and utility of orthomosaics.

Quinoa Phenotyping Methodologies: An International Consensus

“…However, camera effects such as vignetting, camera warming, and temperature drift as well as meteorological conditions such as ambient temperature, wind, and wind direction often affect the accuracy of UAV-derived at-surface temperature measurements [153,154]. Malbeteau et al, (accepted) [155] provide practical information and examples of how to overcome issues related to dynamic temperatures and wind effects during thermal UAV data collection to improve data consistency and accuracy of UAV-based orthomosaics of a field trial designed for phenotyping of tomato plants. UAV-based hyperspectral imagery contains hundreds of spectral bands that allow the collection of detailed spectral information on phenotypic traits [156].…”

Detecting Plant Stress Using Thermal and Optical Imagery From an Unoccupied Aerial Vehicle

“…This range may be inherent to the data collection method due to thermal drift or the creation of the orthomosaic. However, compared to previous research we applied a novel orthomosaic generation method by Malbéteau et al (2021) , wherein the temperature of overlapping pixels was averaged along each swath and normalized between-swath temperatures to reduce the impact of standard orthomosaic generation approaches (which integrate overlapping flight lines collected minutes apart and exposed to different wind directions).…”

“…The image alignment step also performs a bundle adjustment to estimate the camera positions, orientations, and lens calibration parameters. Hence, to recalculate the camera positions, the self-calibrating bundle adjustment computes three-dimensional point clouds from which thermal orthophotos were built ( Malbéteau et al, 2021 ).…”

“…Then, the second corrected swath was used for correcting the third swath and so forth. Adjusting the temperatures of each swath one by one and starting with the first swath of the flight survey ensured that all swaths were also corrected for temperature variability experienced during the 17min of flight time ( Malbéteau et al, 2021 ). The normalization process of individual swaths allowed them to be merged to form an orthomosaic.…”

Quantifying Within-Flight Variation in Land Surface Temperature from a UAV-Based Thermal Infrared Camera

“…In the last decade, thermal sensors have seen significant improvement in their accuracy, weight, and size, making it possible to install a thermal camera onboard a UAV [11][12][13]. In parallel, UAV technology has become a mature means by which to collect information, especially where high spatial and temporal resolution information is required [14].…”