A Deep Learning Framework for Detection of Targets in Thermal Images to Improve Firefighting

Abstract

Intelligent detection and processing capabilities can be instrumental in improving the safety, efficiency, and successful completion of rescue missions conducted by firefighters in emergency first response settings. The objective of this research is to create an automated system that is capable of real-time, intelligent object detection and recognition and facilitates the improved situational awareness of firefighters during an emergency response. We have explored state-of-the-art machine/deep learning techniques to achieve this objective. The goal of this work is to enhance the situational awareness of firefighters by effectively exploiting the infrared video that is actively recorded by firefighters on the scene. To accomplish this, we use a trained deep Convolutional Neural Network (CNN) system to classify and identify objects of interest from thermal imagery in real-time. In the midst of those critical circumstances created by a structure fire, this system is able to accurately inform the decision-making process of firefighters with upto-date scene information by extracting, processing, and analyzing crucial information. Utilizing the new information produced by the framework, firefighters are able to make more informed inferences about the circumstances for their safe navigation through such hazardous and potentially catastrophic environments. INDEX TERMS Deep convolutional neural networks, infrared images, firefighting environment, firefighters, situational awareness.

The INUS Platform: A Modular Solution for Object Detection and Tracking from UAVs and Terrestrial Surveillance Assets

“…In the literature there are several studies that have design efficient schemes to enhance situational awareness [4,[25][26][27][28]. However, their limitations are that they: (i) implement only object detection or only object tracking processes, (ii) consider only UAV or only terrestrial assets, and (iii) employ only RGB or only thermal cameras.…”

A deep Q-Learning based Path Planning and Navigation System for Firefighting Environments

“…The DQN framework is built on top of a VGG-net-like framework [25] as a backbone and is shown in Figure 2. The backbone framework is used as a feature extractor that produces 4096-d features on a 224x224 infrared/thermal image.…”

“…Machine learning does however, perform well in rapid assessment and production of a decision given the current set of circumstances. In other research outside of the scope of this paper, [25], [31] have developed a machine learning based methodology that detects and tracks objects of interest such as doors, ladders, people and fire in the thermal imagery generated by firefighter's thermal cameras. Such information may be valuable to further improve the reinforcement learning algorithm's ability to understand aspects of the environment that may be used in navigation or escape.…”

TheLNet270v1 – A Novel Deep-Network Architecture for the Automatic Classification of Thermal Images for Greenhouse Plants

“…Another study reported by Zhang et al (2019) used a group of neurons termed as a capsule or vector for replacing traditional neurons and achieved equivariance by successfully encoding spatial information and properties of an input image. Bhattarai and Martínez-Ramón (2020) identified and classified objects in real time from thermal cameras carried by firefighters. The detection accuracy reported by authors varied from 70 to 95%, which depends on the depth of the convolution network layer.…”