A novel smartphone-based activity recognition modeling method for tracked equipment in forest operations

Abstract

Activity recognition modelling using smartphone Inertial Measurement Units (IMUs) is an underutilized resource defining and assessing work efficiency for a wide range of natural resource management tasks. This study focused on the initial development and validation of a smartphone-based activity recognition system for excavator-based mastication equipment working in Ponderosa pine (Pinus ponderosa) plantations in North Idaho, USA. During mastication treatments, sensor data from smartphone gyroscopes, accelerometers, and sound pressure meters (decibel meters) were collected at three sampling frequencies (10, 20, and 50 hertz (Hz)). These data were then separated into 9 time domain features using 4 sliding window widths (1, 5, 7.5 and 10 seconds) and two levels of window overlap (50% and 90%). Random forest machine learning algorithms were trained and evaluated for 40 combinations of model parameters to determine the best combination of parameters. 5 work elements (masticate, clear, move, travel, and delay) were classified with the performance metrics for individual elements of the best model (50 Hz, 10 second window, 90% window overlap) falling within the following ranges: area under the curve (AUC) (95.0% - 99.9%); sensitivity (74.9% - 95.6%); specificity (90.8% - 99.9%); precision (81.1% - 98.3%); F1-score (81.9% - 96.9%); balanced accuracy (87.4% - 97.7%). Smartphone sensors effectively characterized individual work elements of mechanical fuel treatments. This study is the first example of developing a smartphone-based activity recognition model for ground-based forest equipment. The continued development and dissemination of smartphone-based activity recognition models may assist land managers and operators with ubiquitous, manufacturer-independent systems for continuous and automated time study and production analysis for mechanized forest operations.

Recent Applications of Smart Technologies for Monitoring the Sustainability of Forest Operations

“…GNSS technology based on more userfriendly devices such as smartphones and smartwatches was tested as well with the aim of monitoring the working productivity of forest operations. These systems reached a working time prediction accuracy ranging from 65.9% to 99.6%, thus demonstrating their reliability in monitoring small-scale forest operations [67][68][69], even if the main limitation, as it often happens when applying GNSS in a forest environment, is signal loss due to canopy occlusion [70,71]. GNSS use in forestry is frequently impacted by multi-pathing error, which is caused when satellite signals are reflected or diffracted by surrounding objects or surfaces or blocked by the canopy or other solid objects [63,72,73].…”

Development of a Robust Machine Learning Model to Monitor the Operational Performance of Fixed-Post Multi-Blade Vertical Sawing Machines

“…Unless a given device holds the capabilities and can be used to collect data multimodally by integrating several sensors, the procurement of separate devices would incur more costs, limiting the economic efficiency of data collection. Nevertheless, by the use of a multimodal approach and RF algorithms, the classification outcomes were found to be similar to those provided by NN, with values of between 97.7 and 99.6% [23,26]. Therefore, it is obvious that when several machine learning algorithms enable classification over a given signal typology, several options need to be checked to evaluate their performance.…”

“…Most of them agree that general classification accuracies of up to 100% may be achieved depending on several factors such as the complexity of classification, signal quality and accuracy of data labeling. In contrast, some have opted for using RF machine learning algorithms for classification purposes [23,26], finding also highly accurate classifications when collecting data multimodally. Unless a given device holds the capabilities and can be used to collect data multimodally by integrating several sensors, the procurement of separate devices would incur more costs, limiting the economic efficiency of data collection.…”

“…Two machine learning algorithms were considered, namely the Artificial Neural Networks (hereafter, NN) and Random Forests (hereafter, RF). The choice was based on the popularity of these two machine learning techniques [4,[22][23][24]26,30,37] as well as on the capabilities and functionalities of the software used [34] to tune, train and test the models (Section 2.4). By the software used, NN models are implemented in the form of multilayer perceptrons with backpropagation [34,38].…”

“…A typical example is that of using accelerometer data loggers, which were found to be very sensitive to motion and vibration, making them very versatile in getting useful information in many disciplines. There are already many examples of studies using ac-celerometer data, which were implemented in forestry and other sectors with the aim of solving specific problems, mainly those related to operational activity recognition [4,[23][24][25][26][27][28][29][30], proving that acceleration data may be successfully used in many tasks. As a fact, electronics have increasingly been used in forestry [31] to find efficient solutions to the current challenges.…”