Unmanned ground vehicle perception using thermal infrared cameras

Abstract

The ability to perform off-road autonomous navigation at any time of day or night is a requirement for some unmanned ground vehicle (UGV) programs. Because there are times when it is desirable for military UGVs to operate without emitting strong, detectable electromagnetic signals, a passive only terrain perception mode of operation is also often a requirement. Thermal infrared (TIR) cameras can be used to provide day and night passive terrain perception. TIR cameras have a detector sensitive to either mid-wave infrared (MWIR) radiation (3-5µm) or long-wave infrared (LWIR) radiation (7-14µm). With the recent emergence of high-quality uncooled LWIR cameras, TIR cameras have become viable passive perception options for some UGV programs. The Jet Propulsion Laboratory (JPL) has used a stereo pair of TIR cameras under several UGV programs to perform stereo ranging, terrain mapping, tree-trunk detection, pedestrian detection, negative obstacle detection, and water detection based on object reflections. In addition, we have evaluated stereo range data at a variety of UGV speeds, evaluated dual-band TIR classification of soil, vegetation, and rock terrain types, analyzed 24 hour water and 12 hour mud TIR imagery, and analyzed TIR imagery for hazard detection through smoke. Since TIR cameras do not currently provide the resolution available from megapixel color cameras, a UGV's daytime safe speed is often reduced when using TIR instead of color cameras. In this paper, we summarize the UGV terrain perception work JPL has performed with TIR cameras over the last decade and describe a calibration target developed by General Dynamics Robotic Systems (GDRS) for TIR cameras and other sensors.

Infrared stereo calibration for unmanned ground vehicle navigation

“…Unlike the calibration pattern for color stereo cameras, which can utilize simple black and white checkerboard patterns for highly accurate calibration, the calibration pattern for two IR stereo cameras must be carefully selected, designed, and/or manufactured. Rankin et al 6 use small light bulbs behind a circular pattern and a grid of wires placed on a checkerboard pattern. Ursine et al 7 use a checkerboard pattern painted on a copper plate background.…”

“…One of the most extensive works on the subject is from Rankin et al, 6 where the authors the details and applications of their perception system which relies on thermal and IR cameras (which they refer to as TIR). Their first TIR stereo calibration target consisted of a board with a pattern of symmetric circles with a light bulb placed behind each plastic circle insert.…”

A comparison of IR stereo vision and LIDAR for use in fire environments

“…Far infrared stereo vision systems like this have been explored for locating and tracking people in normal atmospheric conditions [2], use in medical equipment [3], disparity mapping in clean environments [4,5], and outdoor unmanned guided vehicles [6]. The use of an IR stereo vision system has not been explored for accurately mapping out distances to surfaces indoors for navigation in thick, low visibility smoke.…”

Multispectral image analysis for object recognition and classification

“…Many of the sources found used synthetic visual images which often yields better classification results in comparison to real imagery. In the case of thermal imagery, many of the sources found were based on outdoor environment [25][26] [36] which significantly changes the thermal signature of the objects in comparison to an indoor setting and makes it difficult to compare the recognition rates. In the case of combined visual and thermal dataset, one publication [37] used the visual band imagery for daytime recognition while the thermal imagery was used for night time but the two were not combined in any way.…”