Object Classification in Thermal Images using Convolutional Neural Networks for Search and Rescue Missions with Unmanned Aerial Systems

Rodin, Christopher Dahlin; Lima, Luciano Netto de; Andrade, Fabio; Haddad, Diego B.; Johansen, Tor Arne; Storvold, Rune

doi:10.1109/ijcnn.2018.8489465

Cited by 59 publications

(30 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The radius of the remote sensor’s range is equal to 200 m, which is half of the width of the sensor’s footprint. This footprint was chosen assuming that a computer vision algorithm, such as the one described by [31], can detect the target in images captured at 400 m of altitude by an infrared camera with 7.5 mm of lens focal length, 640×480 pixels of resolution and 17 μm of pixel size.…”

Section: Resultsmentioning

confidence: 99%

Autonomous Unmanned Aerial Vehicles in Search and Rescue Missions Using Real-Time Cooperative Model Predictive Control

Andrade

Hovenburg

Lima

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

Unmanned Aerial Vehicles (UAVs) have recently been used in a wide variety of applications due to their versatility, reduced cost, rapid deployment, among other advantages. Search and Rescue (SAR) is one of the most prominent areas for the employment of UAVs in place of a manned mission, especially because of its limitations on the costs, human resources, and mental and perception of the human operators. In this work, a real-time path-planning solution using multiple cooperative UAVs for SAR missions is proposed. The technique of Particle Swarm Optimization is used to solve a Model Predictive Control (MPC) problem that aims to perform search in a given area of interest, following the directive of international standards of SAR. A coordinated turn kinematic model for level flight in the presence of wind is included in the MPC. The solution is fully implemented to be embedded in the UAV on-board computer with DUNE, an on-board navigation software. The performance is evaluated using Ardupilot’s Software-In-The-Loop with JSBSim flight dynamics model simulations. Results show that, when employing three UAVs, the group reaches 50% Probability of Success 2.35 times faster than when a single UAV is employed.

show abstract

Section: Resultsmentioning

confidence: 99%

Autonomous Unmanned Aerial Vehicles in Search and Rescue Missions Using Real-Time Cooperative Model Predictive Control

Andrade

Hovenburg

Lima

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…They had compared various tracking methods by utilizing visible spectrum images and thermal images and concluded that the best tracking methods for normal images and thermal images are (ASLA, SCM and DSST) on the basis of its dimensional structure and/ or scattered representation and (EDFT) on the basis of pixel value distribution respectively. Rodin et al (2018) had worked on thermal images captured by Unmanned Ariel Systems (UAS) for the sea surface detection and classification of objects. It was beneficial for searching and finding the marine objects.…”

Section: Related Workmentioning

confidence: 99%

Object Detection and Classification from Thermal Images Using Region based Convolutional Neural Network

Mittal¹,

Srivastava²,

Chawla³

2019

Journal of Computer Science

View full text Add to dashboard Cite

In recent years, object detection and classification has gained so much popularity in different application areas like face detection, selfdriving cars, pedestrian detection, security surveillance systems etc. The traditional detection methods like background subtraction, Gaussian Mixture Model (GMM), Support Vector Machine (SVM) have certain drawbacks like overlapping of objects, distortion due to smoke, fog, lightening conditions etc. In this paper, thermal images are used as thermal cameras capture the image by using the heat generated by the objects. Thermal camera images are not influenced by smoke and bad weather conditions which makes them a built-up apparatus in inquiry and safeguards or fire-fighting applications. These days, deep learning techniques are extensively used for detection and classification. In this paper, a comparative analysis has been done by applying Faster region based convolutional neural network on thermal images and visual spectrum images. The experimental results show that thermal camera images are better as compared to visible spectrum images.

show abstract

“…The output of the CNN is the classification result (object or nonobject) of the input image. This approach is demonstrated in Rodin et al (2018) for detection of boats and humans at sea in thermal images taken from a UAV. The major issue with the machine learning approaches is that their accuracy is greatly affected by the size and quality of the training set (Doherty & Rudol, 2007; Gaszczak, Breckon, & Han, 2011; Portmann, Lynen, Chli, & Siegwart, 2014; Viola & Jones, 2001).…”

Section: Introductionmentioning

confidence: 99%

Object detection, recognition, and tracking from UAVs using a thermal camera

Leira

Helgesen

Johansen

et al. 2020

Journal of Field Robotics

Self Cite

View full text Add to dashboard Cite

In this paper a multiple object detection, recognition, and tracking system for unmanned aerial vehicles (UAVs) has been studied. The system can be implemented on any UAVs platform, with the main requirement being that the UAV has a suitable onboard computational unit and a camera. It is intended to be used in a maritime object tracking system framework for UAVs, which enables a UAV to perform multiobject tracking and situational awareness of the sea surface, in real time, during a UAV operation. Using machine vision to automatically detect objects in the camera's image stream combined with the UAV's navigation data, the onboard computer is able to georeference each object detection to measure the location of the detected objects in a local North‐East (NE) coordinate frame. A tracking algorithm which uses a Kalman filter and a constant velocity motion model utilizes an object's position measurements, automatically found using the object detection algorithm, to track and estimate an object's position and velocity. Furthermore, a global‐nearest‐neighbor algorithm is applied for data association. This is achieved using a measure of distance that is based not only on the physical distance between an object's estimated position and the measured position, but also how similar the objects appear in the camera image. Four field tests were conducted at sea to verify the object detection and tracking system. One of the flight tests was a two‐object tracking scenario, which is also used in three scenarios with an additional two simulated objects. The tracking results demonstrate the effectiveness of using visual recognition for data association to avoid interchanging the two estimated object trajectories. Furthermore, real‐time computations performed on the gathered data show that the system is able to automatically detect and track the position and velocity of a boat. Given that the system had at least 100 georeferenced measurements of the boat's position, the position was estimated and tracked with an accuracy of 5–15 m from 400 m altitude while the boat was in the camera's field of view (FOV). The estimated speed and course would also converge to the object's true trajectories (measured by Global Positioning System, GPS) for the tested scenarios. This enables the system to track boats while they are outside the FOV of the camera for extended periods of time, with tracking results showing a drift in the boat's position estimate down to 1–5 m/min outside of the FOV of the camera.

show abstract

Object Classification in Thermal Images using Convolutional Neural Networks for Search and Rescue Missions with Unmanned Aerial Systems

Cited by 59 publications

References 27 publications

Autonomous Unmanned Aerial Vehicles in Search and Rescue Missions Using Real-Time Cooperative Model Predictive Control

Autonomous Unmanned Aerial Vehicles in Search and Rescue Missions Using Real-Time Cooperative Model Predictive Control

Object Detection and Classification from Thermal Images Using Region based Convolutional Neural Network

Object detection, recognition, and tracking from UAVs using a thermal camera

Contact Info

Product

Resources

About