Visible and Thermal Image-Based Trunk Detection with Deep Learning for Forestry Mobile Robotics

Abstract

Mobile robotics in forests is currently a hugely important topic due to the recurring appearance of forest wildfires. Thus, in-site management of forest inventory and biomass is required. To tackle this issue, this work presents a study on detection at the ground level of forest tree trunks in visible and thermal images using deep learning-based object detection methods. For this purpose, a forestry dataset composed of 2895 images was built and made publicly available. Using this dataset, five models were trained and benchmarked to detect the tree trunks. The selected models were SSD MobileNetV2, SSD Inception-v2, SSD ResNet50, SSDLite MobileDet and YOLOv4 Tiny. Promising results were obtained; for instance, YOLOv4 Tiny was the best model that achieved the highest AP (90%) and F1 score (89%). The inference time was also evaluated, for these models, on CPU and GPU. The results showed that YOLOv4 Tiny was the fastest detector running on GPU (8 ms). This work will enhance the development of vision perception systems for smarter forestry robots.

Sugar Beet Damage Detection during Harvesting Using Different Convolutional Neural Network Models

“…Comparing with the methodologies used for broken corn detection at a conveyor belt of a corn harvester based on different YOLO v3 models [15], the proposed YOLO v4 According to the performance of YOLO v4, it achieved a good balance between precision, recall, F1-score and speed which could be considered as the best model for sugar beet damage detection during harvesting. This finding is in line with reported studies that the YOLO networks could achieve higher speed and better overall performance e.g., [31,33,36]. Furthermore, our finding is in agreement with [37] who reported that YOLO models achieved higher speed and F1-score compared with SVM, Faster R-CNN for apple surface defect detection.…”

“…In this study, the YOLO v4 model shows better performance compared to the other developed networks. This finding is in line with previous studies conducted for detection of citrus in an orchard [30], apples in a farming complex environment [31], pests [32] and tree trunks in a forest [33]. However, the Faster R-CNN NAS model shows lower performance in this research.…”

“…In this context, results of a reported study by [34] showed that the Faster R-CNN NAS model had higher precision than Faster R-CNN Inception v2 for kiwi fruit detection; however, Faster R-CNN Inception v2 showed better recall compared to the Faster R-CNN NAS model. detection of citrus in an orchard [30], apples in a farming complex environment [31], pests [32] and tree trunks in a forest [33]. However, the Faster R-CNN NAS model shows lower performance in this research.…”

Unimodal and Multimodal Perception for Forest Management: Review and Dataset

“…The system was tested in a simulated and in a real environment: in the simulated environment, the UAV concluded 85% of the test flights without collisions, and in the real environment, the UAV concluded all test flights without collisions. Other studies focused on detecting tree trunks in street images using Deep Learning methods [29,30], in dense forests using visible and thermal imagery combined with Deep Learning [31], and even on the detection of stumps in harvested forests [32] to enhance the surrounding awareness of the operators and to endow machines with intelligent object avoidance systems.…”

“…Health and diseases Offline [4-8] Inventory and structure Offline [9-23] Navigation Online [24][25][26][27][28][29][30][31][32] The aim of the "Health and diseases" category is to monitor the health of forest lands and detect the existence of diseases that affect forest trees, destroying some forest cultures and ecosystems. Data from this category are most of the times processed offline-the data are collected in the field and are processed later.…”